Gastrointestinal cancer pathological image segmentation method based on vit mechanism model and related equipment

Abstract

The present application belongs to the technical field of medical image processing, and particularly relates to a gastrointestinal cancer pathological image segmentation method based on a ViT mechanism model and related equipment. The gastrointestinal cancer pathological image segmentation method based on the ViT mechanism model comprises the following steps: obtaining a gastrointestinal cancer pathological image to be segmented, and preprocessing the gastrointestinal cancer pathological image to be segmented; adopting a preset cell nucleus segmentation model based on a ViT main body to perform cell nucleus segmentation on the preprocessed gastrointestinal cancer pathological image, and generating two segmentation predictions; and adding the two segmentation predictions to obtain a prediction result, which is taken as a cell nucleus segmentation result. The present application is aimed at the problems of uneven staining of pathological images, irregular, overlapping gastrointestinal cancer cell nuclei, and insufficient segmentation accuracy of previous methods. The gastrointestinal cancer pathological image segmentation method based on the ViT mechanism model can automatically, efficiently and accurately detect and segment the cell nucleus region in the gastrointestinal cancer pathological image.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of medical image processing, and particularly relates to a gastrointestinal cancer pathological image segmentation method based on a ViT mechanism model and related equipment. BACKGROUND

[0002] Gastrointestinal cancer is one of the most common malignant tumors in the world. Since most of the early symptoms of gastrointestinal tumors are not obvious, most patients have obvious symptoms when they are in the middle and late stages. Pathological images are one of the gold standards for diagnosing gastrointestinal cancer, but their manual analysis takes a long time.

[0003] Digital pathological image analysis technology can provide quantitative and objective basis for pathological diagnosis. It is an important aid for pathologists to make objective, accurate and reliable pathological diagnosis. Among them, the positioning of the cell nucleus in the pathological image is one of the basic tasks of cancer detection, tumor grading, and survival prediction. Therefore, quickly and accurately segmenting the cell nucleus in the pathological image has important clinical significance for gastrointestinal cancer diagnosis. However, the segmentation of the cell nucleus in the pathological image usually has the following difficulties: first, the cell image is complex, usually with a large number of cells, each cell is in a different growth stage, and the cell nucleus morphology is different; second, the pathological image is subject to uneven staining, inconsistent lighting conditions, and large image differences; third, the spatial distribution of tissue cells results in a high probability of cell nucleus overlap in the image.

[0004] By introducing artificial intelligence methods into pathological image analysis, fast and accurate segmentation of cell nuclei can reduce the difficulty and time of doctors analyzing pathological images, improve the efficiency and accuracy of diagnosis, and better serve patients.

[0005] Currently, researchers have also proposed some cell nucleus segmentation algorithms, which can be divided into the following categories:

[0006] First, traditional machine learning methods. Among them, random forests and SVM have received a lot of attention due to their good performance in cell and cell nucleus segmentation. Unlike traditional classification models, K-means uses a clustering method that can also effectively segment single cell nucleus regions. However, traditional machine learning techniques rely on handcrafted features, require complex design, and have limited representation capabilities, resulting in less than ideal segmentation results.

[0007] Second, the convolutional neural network-based cell nucleus segmentation method. Convolutional neural network opens up a new segmentation idea. Compared with other network models, the parameter sharing characteristics of convolution operation greatly reduce the number of parameters that need to be optimized, improve the training efficiency and scalability of the model. For example, Zhang Wei et al. used a deep learning model to complete the cell nucleus segmentation of pathological pictures. In addition, many CNN variant models, such as MDC-Net and GCN, have been successfully applied to cell nucleus segmentation and detection of pathological images. Some methods also introduce attention mechanisms (He Yongjun, Zhao Jing, a cell nucleus segmentation method based on attention learning, CN112446892A) to improve segmentation accuracy. Since 2015, when U-Net was proposed, improved models of U-net have achieved good results in cell nucleus segmentation tasks. For example, Micro-Net improved the original loss function of U-Net and achieved great success in cell nucleus segmentation. HoverNet extracts multi-scale features of cell nuclei to achieve segmentation.

[0008] Third, cell nucleus segmentation based on Transformer mechanism. Transformer is a model proposed in 2017 for Seq2Seq tasks, which has achieved certain improvement in tasks such as machine translation. In recent years, it has gradually migrated to the field of image processing. The core of Transformer is the introduction of self-attention mechanism and multi-head attention mechanism, which enables the model to capture long-range dependencies of features. For example, MedT introduces a gated attention mechanism, adding four gated heads based on axial attention, which solves the problem of difficult learning of position bias in small-scale data sets. Similarly, Zhang et al. proposed MBT-Net for cell nucleus segmentation, which also uses the ideas of transformer and multi-branch.

[0009] Deep learning-based segmentation algorithms have the advantages of fast segmentation speed and high precision, but the current methods still have the following problems: First, CNN-based methods, as the number of network layers increases, the model's features will lack long-range dependencies, resulting in insufficient segmentation accuracy, especially in cases where the number of cell nuclei is large, overlapping, and small in area, making it difficult to identify. Second, for encoder-decoder models, the contribution difference between different levels of features needs to be designed an effective mechanism to fuse these features. Third, the staining degree of different pathological sections differs, affecting the segmentation results. SUMMARY

[0010] This invention addresses the limitations of existing deep learning-based cell nucleus segmentation algorithms in gastrointestinal cancer pathological images, which suffer from limited representation capabilities and inefficient feature fusion. It provides a gastrointestinal cancer pathological image segmentation method and supporting equipment based on the ViT mechanism model, aiming to improve the accuracy and efficiency of cell nucleus segmentation in gastrointestinal cancer pathological images and apply it to the clinical auxiliary diagnosis of gastrointestinal cancer.

[0011] A method for segmenting gastrointestinal cancer pathological images based on the ViT mechanism model, comprising:

[0012] Obtain the pathological images of gastrointestinal cancer to be segmented, and preprocess the pathological images of gastrointestinal cancer to be segmented;

[0013] The preprocessed gastrointestinal cancer pathological image is segmented into nuclei using a pre-defined ViT-based nucleus segmentation model, generating two segmentation predictions. The sum of the two segmentation predictions is taken as the nucleus segmentation result.

[0014] As a preferred embodiment, before performing nucleus segmentation on the preprocessed gastrointestinal cancer pathological image using a preset ViT-based nucleus segmentation model, the method further includes training the nucleus segmentation model:

[0015] Obtain pathological data of gastrointestinal cancer, label cell nuclei in tissue images to obtain several labeled pathological images of gastrointestinal cancer, and establish a specialized database of gastrointestinal cancer pathology.

[0016] The labeled gastrointestinal cancer pathological images are preprocessed;

[0017] A cell nucleus segmentation model based on the ViT backbone is constructed. The cell nucleus segmentation model is trained using the preprocessed gastrointestinal cancer pathological images. During training, the two segmentation predictions generated by the cell nucleus segmentation model are used to calculate the loss function with a preset mask, and then the two segmentation predictions are summed as the final adjustment data and passed to the cell segmentation model.

[0018] As a preferred embodiment, the cell nucleus segmentation model includes an encoder for feature capture and a decoder for aggregating features at different levels;

[0019] The encoder employs a deformable self-attention mechanism. Through calculations performed by the encoder, four levels of features f from different depths are extracted from the gastrointestinal cancer pathological image. i , i = 1, 2, 3 or 4, where f1 is a low-order feature, and f2, f3 and f4 are high-order features;

[0020] The decoder aggregates the low-order features and the high-order features into two segmentation predictions.

[0021] As a preferred embodiment, the decoder includes:

[0022] An aggregation module stacks the higher-order features from each level together, and after convolution, obtains the result of higher-order feature aggregation, which is an aggregated feature T1, and generates a segmentation prediction P1 based on it.

[0023] A parallel pooling attention convolution module (ParNet Attention) generates three levels of feature maps based on the low-order feature f1 through pooling layers. Each level of feature map is convolved and then fused together. Finally, a low-order feature T2 is obtained by connecting an average pooling layer and a fully connected layer.

[0024] A similarity aggregation module (SAM) fuses the features T1 and T2 to generate another segmentation prediction P2.

[0025] As a preferred approach, when preprocessing the segmented gastrointestinal cancer pathological images or the labeled gastrointestinal cancer pathological images, size normalization and color normalization are employed.

[0026] As a preferred embodiment, the size normalization process involves cropping the gastrointestinal cancer pathological image to a preset size.

[0027] As a preferred embodiment, the color normalization processing method includes:

[0028] Let the pathological image of the gastrointestinal cancer be image s, and the preset template image be image t;

[0029] Using the following formulas (1) and (2), the optical density V of image s is... s Convert to W s H s The optical density V of image t t Convert to W t H t W s Let H be the color appearance matrix of image s. s Let W be the coloring density matrix of image s. t Let H be the color appearance matrix of image t. t Let be the coloring density matrix of image t;

[0030]

[0031] V=WH (2)

[0032] Where V represents the optical density of the image, I0 represents the incident light intensity, I represents the original RGB image, W represents the color appearance matrix of the image, and H represents the color density matrix of the image.

[0033] Using the following formula (3), the color density matrix of image s and the color density matrix of image t are registered to obtain the color density matrix of the registered image s.

[0034]

[0035] Among them, H s (j,:) is the staining density matrix of image s before registration, H t (j,:) is the staining density matrix before image t is registered, j is the index of r staining agents, and RM(·) is used to calculate the robust pseudo maximum at 99% for each row vector.

[0036] Using the following formula (4), the color density matrix of the registered image s is multiplied by the color appearance matrix of the unregistered image t to obtain the color-normalized image s;

[0037]

[0038] in, For the color-normalized image s, W t The color appearance matrix of image t before registration. The color density matrix of the registered image s.

[0039] A gastrointestinal cancer pathological image segmentation device based on the ViT mechanism model, comprising:

[0040] The preprocessing module is used to acquire the pathological images of gastrointestinal cancer to be segmented and to preprocess the pathological images of gastrointestinal cancer to be segmented.

[0041] The cell nucleus segmentation module is used to perform cell nucleus segmentation on the preprocessed gastrointestinal cancer pathological image using a preset cell nucleus segmentation model based on the ViT backbone, generate two segmentation predictions, and use the sum of the two segmentation predictions as the cell nucleus segmentation result.

[0042] A computer device includes a memory and a processor, wherein the memory stores computer-readable instructions, which, when executed by the processor, cause the processor to perform the steps of the above-described gastrointestinal cancer pathological image segmentation method based on the ViT mechanism model.

[0043] A storage medium storing computer-readable instructions, which, when executed by one or more processors, cause the one or more processors to perform the steps of the above-described gastrointestinal cancer pathological image segmentation method based on the ViT mechanism model.

[0044] The positive and progressive effects of this invention are as follows: This invention employs a gastrointestinal cancer pathological image segmentation method and related equipment based on the ViT mechanism model, which has the following advantages:

[0045] 1. To address issues such as uneven staining in pathological images, irregular and overlapping nuclear morphology of gastrointestinal cancer cells, and insufficient segmentation accuracy of previous methods, the gastrointestinal cancer pathological image segmentation method based on the ViT mechanism model proposed in this invention can be trained using a small training dataset. Furthermore, the trained model can automatically, efficiently, and accurately detect and segment the cell nucleus region in gastrointestinal cancer pathological images, providing valuable assistance for tasks such as computer-aided diagnosis and radiomics analysis.

[0046] 2. A visual Transformer backbone with deformable attention is used to encode pathological images of gastrointestinal cancer. Image feature information at different levels is extracted sequentially from shallow to deep, and the fine feature data at deep levels is aggregated and encoded. This can refine inaccurate and coarse estimates into more accurate edge prediction maps, thereby effectively improving the segmentation performance of the model.

[0047] 3. Introducing nonlocal operations in the graph convolution domain to achieve similarity aggregation can more fully combine high-order and low-order features.

[0048] 4. Using color uniformity technology, images with different color distributions and color depths are normalized to make the color information distribution of the images as consistent as possible. Attached Figure Description

[0049] Figure 1 This is a flowchart of the method of the present invention;

[0050] Figure 2 This is a model structure diagram of the cell nucleus segmentation model of the present invention;

[0051] Figure 3 This is a model structure diagram of the deformable self-attention module of the present invention. Detailed Implementation

[0052] To make the technical means, creative features, objectives and effects of this invention easier to understand, the invention will be further described below with reference to specific illustrations.

[0053] Reference Figure 1 A method for segmenting gastrointestinal cancer pathological images based on the ViT mechanism model includes the following steps:

[0054] S1, Preprocessing of gastrointestinal cancer pathological images: Obtain the gastrointestinal cancer pathological images to be segmented and perform preprocessing on the gastrointestinal cancer pathological images to be segmented.

[0055] The pathological images of gastrointestinal cancer in this invention are derived from pathological slides of gastrointestinal cancer to be segmented, and are digitized using a gastrointestinal cancer pathological slide scanner to finally obtain pathological images of gastrointestinal cancer to be segmented.

[0056] Since gastrointestinal cancer pathology images may come from different models of machines, different parameters will lead to differences in image size, and inconsistent staining conditions of pathology sections will lead to differences in image color. Therefore, gastrointestinal cancer pathology images need to be preprocessed before subsequent segmentation steps.

[0057] In some embodiments, when preprocessing the gastrointestinal cancer pathology images to be segmented, size normalization and color normalization are employed.

[0058] In some embodiments, the size normalization process involves cropping the gastrointestinal cancer pathology image to a preset size. For example, the gastrointestinal cancer pathology image to be segmented is cropped to a size of 512*512.

[0059] In some embodiments, the color normalization process can adopt the color normalization process in the prior art, or it can adopt the following steps:

[0060] Let image s be a pathological image of gastrointestinal cancer, and image t be a preset template image. Image t is a preset image, and all pathological images of gastrointestinal cancer to be segmented need to be color-normalized to the colors of image t.

[0061] First, the pathological image or template image of gastrointestinal cancer to be segmented, i.e. the original RGB image I, is converted into optical density V. Two dye matrices are obtained by non-negative matrix decomposition, specifically represented by the following formulas (1) and (2).

[0062]

[0063] V=WH (2)

[0064] Where V represents the optical density of the image, I0 represents the incident light intensity, I represents the original RGB image, W represents the color appearance matrix of the image, and H represents the color density matrix of the image.

[0065] The optical density V of image s is calculated using formulas (1) and (2). s Convert to W s H s The optical density V of image t t Convert to W t Ht W s Let H be the color appearance matrix of image s. s Let W be the coloring density matrix of image s. t Let H be the color appearance matrix of image t. t Let be the coloring density matrix of image t;

[0066] Secondly, image s can be registered with the color density matrix of image t using the structure-preserving color normalization (SPCN) algorithm, resulting in the registered color density matrix of image s. Registration is performed using the following formula (3);

[0067]

[0068] Among them, H s (j,:) is the staining density matrix of image s before registration, H t (j,:) is the staining density matrix before image t is registered, j is the index of r staining agents, and RM(·) is used to calculate the robust pseudo maximum at 99% for each row vector.

[0069] Finally, the color density matrix of the registered image s is multiplied by the color appearance matrix of the unregistered image t to obtain the color-normalized image s, as shown in the following formula (4);

[0070]

[0071] in, For the color-normalized image s, W t The color appearance matrix of image t before registration. The color density matrix of the registered image s.

[0072] This invention utilizes color uniformization technology to normalize the color (color) of gastrointestinal cancer pathological images with different color distributions and staining depths, so that the color information distribution of gastrointestinal cancer pathological images is kept as consistent as possible, which is beneficial to improving the accuracy of subsequent model segmentation.

[0073] S2, Gastrointestinal Cancer Pathological Image Segmentation: The pre-defined ViT-based nuclear segmentation model is used to segment the pre-processed gastrointestinal cancer pathological image into nuclear segments, generating two segmentation predictions. The sum of the two segmentation predictions is used as the nuclear segmentation result.

[0074] To address the issues of uneven staining, irregular and overlapping cell nuclei in gastrointestinal cancer pathological images, and insufficient segmentation accuracy of previous methods, this invention introduces a cell nucleus segmentation model based on the ViT backbone to segment preprocessed gastrointestinal cancer pathological images. This model can automatically, efficiently, and accurately detect and segment the cell nucleus region in gastrointestinal cancer pathological images, and the segmented cell nucleus region is the result of this segmentation.

[0075] The ViT-based kernel segmentation model can transform 2D images into 1D patch sequences, enabling deformable self-attention modules (Transformers) to process the images. Specifically, ViT can stack multiple Transformer modules together to process non-overlapping image patch sequences, resulting in a non-convolutional image classification module. Compared to CNN models, Transformer-based models have a larger receptive field, excel at modeling long-term dependencies, and achieve superior performance with large amounts of training data and model parameters.

[0076] In some embodiments, before performing nucleus segmentation on the preprocessed gastrointestinal cancer pathological images using the nucleus segmentation model, the present invention also trains the nucleus segmentation model:

[0077] S201. Obtain pathological data of gastrointestinal cancer, label cell nuclei in tissue images, obtain several labeled pathological images of gastrointestinal cancer, and establish a specialized database of gastrointestinal cancer pathology.

[0078] The gastrointestinal cancer pathology data of this invention comes from various devices in several clinical hospitals. Gastrointestinal cancer pathology slides are selected and digitized using a pathology slide scanner. The slides are then named and anonymized using existing ImageViewerG software, resulting in several gastrointestinal cancer pathology images. Subsequently, the cell nuclei in the gastrointestinal cancer pathology images are labeled using the existing Labelme toolkit for training and testing of a cell nucleus segmentation model. The labeled gastrointestinal cancer pathology images are stored together as a specialized gastrointestinal cancer pathology database.

[0079] S202, preprocessing the labeled gastrointestinal cancer pathological images.

[0080] Since gastrointestinal cancer pathological images may come from different models of machines, the different parameters will lead to differences in image size, and the inconsistent staining conditions of the pathological sections will lead to differences in image color. Therefore, this invention also preprocesses the gastrointestinal cancer pathological images in the gastrointestinal cancer pathology database. This preprocessing process is the same as the preprocessing process for the gastrointestinal cancer pathological images to be segmented in step S1, that is, it also adopts size normalization and color normalization processing methods, which will not be described in detail here.

[0081] S203. Construct a cell nucleus segmentation model based on the ViT backbone. Train the cell nucleus segmentation model using preprocessed gastrointestinal cancer pathological images. During training, calculate the loss function of the two segmentation predictions generated by the cell nucleus segmentation model with the preset mask, and then sum the two segmentation predictions as the final adjustment data and pass it to the cell segmentation model.

[0082] When training the cell nucleus segmentation model, this invention measures and predicts the quality of the model by calculating a loss function with a mask, so as to adjust and optimize the model.

[0083] In some embodiments, refer to Figure 2 The ViT-based kernel segmentation model in this invention includes an encoder for feature capture and a decoder for aggregating features at different levels. For example, Figure 2 In this context, A) DAT Encoder is the encoder, and B) Par Attention, C) Aggregation Module, and D) SAM are the decoders.

[0084] The encoder employs a deformable attention transformer (DAT) mechanism. Through the encoder's computation, it extracts four levels of features from different depths from gastrointestinal cancer pathological images. i Let i = 1, 2, 3, or 4, where f1 is a low-order feature and f2, f3, and f4 are high-order features. The decoder aggregates the low-order and high-order features into two segmentation predictions, namely segmentation prediction P1 and segmentation prediction P2.

[0085] Specifically, the input image with shape H×W×3, i.e., the gastrointestinal cancer pathology image of this invention, is first embedded using a 4×4 non-overlapping convolution, followed by a normalization layer to obtain an H / 4×W / 4×C patch embedding. To construct a hierarchical feature pyramid, the backbone network consists of four stages with progressively increasing strides. Between two consecutive stages, a non-overlapping 2×2 convolution with stride=2 downsamples the feature map, halving the spatial size and doubling the feature size.

[0086] In the first two stages of DAT, the large space size of keys and values ​​significantly increases the computational overhead of dot products and bilinear interpolation in Deformable Attention. Therefore, to achieve a trade-off between model capacity and computational burden, the feature maps are first processed through a window-based local attention mechanism to locally aggregate information. Then, a shift window attention mechanism is used for further processing to achieve better representation in the early stages.

[0087] In the third and fourth stages of DAT, the feature maps are first processed through a window-based local attention mechanism to aggregate information locally. Then, a deformable attention block is used to model the global relationships between the locally enhanced representations (tokens). This alternative design of attention blocks with both local and global receptive fields helps the model learn strong representations.

[0088] Reference Figure 3 For deformable self-attention modules, given an input feature map x∈R H×W×C A uniform mesh map p is generated as a reference. Specifically, the mesh size is downsampled by a coefficient from the input feature map size, and the values ​​of the reference points are linearly spaced 2D coordinates, which are then normalized to a range according to the mesh shape. To obtain the offset of each reference point, the feature map is linearly projected onto q and fed into a subnetwork θ that computes the offset. offset To obtain the final offset θ offset (q). Calculate the features query(q), deformed and deformed The specific formula is as follows:

[0089]

[0090] Multi-head attention is applied to q, k, and v, and positional offsets are taken into account. The features of each head are concatenated together, and the final output is obtained through projection.

[0091] Using the encoder technology described above, it is possible to... Figure 2 The A)DAT Encoder part extracts features from different depths at four levels: f1, f2, f3, and f4.

[0092] In some embodiments, refer to Figure 2The decoder consists of three sub-modules: the aggregation module, the parallel pooling attention convolution module (ParNet Attention), and the similarity aggregation module (SAM).

[0093] The aggregation module stacks the high-order features from each level together, performs convolution, and obtains the aggregated high-order feature result, which is an aggregated feature T1. A segmentation prediction P1 is then generated based on this result. For example... Figure 2 As shown in the diagram, in specific implementation, for high-order feature f2, it is sequentially aggregated through max pooling and aggregation module 1-1 to obtain the first aggregation result. For high-order feature f3, after max pooling, it is aggregated together with the max-pooled f2 through aggregation module 2-1, and then aggregated together with the first aggregation result through aggregation module 2-2 to obtain the second aggregation result. For high-order feature f3, after max pooling, it is aggregated together with the max-pooled f2 and max-pooled f3 through aggregation module 3-1, and then aggregated together with the second aggregation result through aggregation module 3-2 to obtain the third aggregation result. The third aggregation result is then convolved twice consecutively to obtain the aggregated feature T1 / segmentation prediction P1.

[0094] The parallel pooling attention convolution module generates three levels of feature maps based on the low-order feature f1 through pooling layers. Each level of feature map is convolved and then fused together to sequentially mine complementary regions and details. Finally, a low-order feature T2 is obtained by connecting an average pooling layer and a fully connected layer. Figure 2 As shown, in specific implementation, the low-order feature f1 is divided into three feature maps of different sizes using two max pooling operations: the first-level feature map, the second-level feature map, and the third-level feature map. For the first-level feature map, after passing through two consecutive parallel network modules (ParNetBlock), it undergoes another max pooling operation to obtain the first-level result. For the second-level feature map, after passing through three consecutive parallel network modules, it is fused with the first-level result to obtain the second-level result. For the third-level feature map, it undergoes another max pooling operation, then passes through three consecutive parallel network modules, and is fused with the second-level result to obtain the third-level result. The third-level result is then passed through an average pooling and fully connected layer (Avgpooling + FCN) to obtain the low-order feature T2.

[0095] The similarity aggregation module fuses features T1 and T2 to generate another segmentation prediction P2. This module is a non-local operation that can use global attention to inject detailed appearance features into high-level semantic features. This invention introduces a non-local operation in the graph convolution domain to achieve similarity aggregation, enabling a more comprehensive combination of high-order and low-order features.

[0096] Given a feature map T1 containing high-level semantic information and a feature map T2 with rich appearance details, they are fused through self-attention to obtain the final Q, K, and V. Specifically, this is achieved using the following formula:

[0097]

[0098]

[0099]

[0100] Z = T1 + W z (Y′)

[0101]

[0102] The above formula is existing technology, and its specific meaning will not be elaborated here.

[0103] For segmentation predictions P1 and P2, during model training, P1 and P2 are each calculated with a pre-set mask to calculate a loss function. The sum of P1 and P2 is then used as the final adjusted data and passed to the cell segmentation model. During normal segmentation after model training, the sum of P1 and P2 obtained from the decoder is used as the cell nucleus segmentation result.

[0104] The aforementioned visual Transformer backbone with deformable attention is used to encode gastrointestinal cancer pathological images. Image feature information at different levels is extracted sequentially from shallow to deep, and the fine feature data at deep levels is aggregated and decoded. This can refine inaccurate and coarse estimates into more accurate edge prediction maps, thereby effectively improving the segmentation performance of the model.

[0105] In some embodiments, the present invention may further include step S3, which involves displaying, saving, and evaluating the results of cell nucleus segmentation.

[0106] In some embodiments, the present invention uses mean absolute error (MAE), DICE coefficient, intersection-over-union ratio (IoU), sensitivity (sen), and specificity (spec) as objective evaluation indicators. The present invention can randomly select several gastrointestinal cancer pathological images from a gastrointestinal cancer pathology database as a test set, and verify the segmentation accuracy using steps S1 to S2 of the present invention.

[0107] The comparison examples are the U-net, R2Unet, and Att-Unet models in the prior art. The segmentation results are obtained by replacing step S2 of the present invention with the models in the prior art.

[0108] The present invention was compared with the above comparative examples using mean absolute error (MAE), DICE coefficient, intersection-over-union ratio (IoU), sensitivity (sen), and specificity (spec) as objective evaluation indicators, and the segmentation accuracy was obtained as shown in the table below:

[0109] Metic U-net R2Unet Att-Unet The invention MAE 0.1559 0.1708 0.1376 0.0765 DICE 0.6127 0.5842 0.6084 0.8080 IoU 0.4576 0.4245 0.4566 0.6806 Sen 0.7159 0.6941 0.6525 0.8438 Spec 0.8462 0.8513 0.8853 0.9398

[0110] The results show that the method of the present invention has the best performance with an MAE of 0.0765, a DICE coefficient of 0.808, an IoU of 0.6806, a Sen of 0.8438, and a Spec of 0.9398.

[0111] In some embodiments, the present invention proposes a gastrointestinal cancer pathological image segmentation device based on the ViT mechanism model, comprising:

[0112] The preprocessing module is used to acquire the pathological images of gastrointestinal cancer to be segmented and to preprocess the pathological images of gastrointestinal cancer to be segmented.

[0113] The cell nucleus segmentation module is used to segment the preprocessed gastrointestinal cancer pathological images using a preset cell nucleus segmentation model based on the ViT backbone, generating two segmentation predictions. The sum of the two segmentation predictions is used as the cell nucleus segmentation result.

[0114] In some embodiments, the present invention provides a computer device including a memory and a processor, wherein the memory stores computer-readable instructions, and when the computer-readable instructions are executed by the processor, the processor causes the processor to perform the steps in the gastrointestinal cancer pathological image segmentation method based on the ViT mechanism model in the above embodiments.

[0115] In some embodiments, the present invention provides a storage medium storing computer-readable instructions, which, when executed by one or more processors, cause the one or more processors to perform the steps in the gastrointestinal cancer pathological image segmentation method based on the ViT mechanism model in the above embodiments.

[0116] Those skilled in the art will understand that all or part of the steps in the various methods of the above embodiments can be implemented by a program instructing related hardware. This program can be stored in a computer-readable storage medium, which may include: read-only memory (ROM), random access memory (RAM), a magnetic disk, or an optical disk, etc. The storage medium can be a non-volatile storage medium.

[0117] The foregoing has shown and described the basic principles, main features, and advantages of the present invention. Those skilled in the art should understand that the present invention is not limited to the above embodiments. The embodiments and descriptions in the specification are merely illustrative of the principles of the invention. Various changes and modifications can be made to the invention without departing from its spirit and scope, and all such changes and modifications fall within the scope of the present invention as claimed. The scope of protection of this invention is defined by the appended claims and their equivalents.

Claims

1. A method for segmenting gastrointestinal cancer pathological images based on the ViT mechanism model, characterized in that, include: Obtain the pathological images of gastrointestinal cancer to be segmented, and preprocess the pathological images of gastrointestinal cancer to be segmented; A pre-defined ViT-based nuclear segmentation model is used to segment the preprocessed gastrointestinal cancer pathological image into nuclear segments, generating two segmentation predictions. The sum of these two predictions is taken as the final nuclear segmentation result. The nuclear segmentation model includes an encoder for feature capture and a decoder for aggregating features at different levels. The encoder employs a deformable self-attention mechanism. Through the encoder's calculations, four levels of features from different depths are extracted from the gastrointestinal cancer pathological image. i i = 1, 2, 3 or 4, where f1 is a low-order feature and f2, f3 and f4 are high-order features; the decoder aggregates the low-order features and the high-order features into two segmentation predictions.

2. The gastrointestinal cancer pathological image segmentation method based on the ViT mechanism model as described in claim 1, characterized in that, Before performing nucleus segmentation on the preprocessed gastrointestinal cancer pathological image using a preset ViT-based nucleus segmentation model, the process also includes training the nucleus segmentation model: Obtain pathological data of gastrointestinal cancer, label cell nuclei in tissue images to obtain several labeled pathological images of gastrointestinal cancer, and establish a specialized database of gastrointestinal cancer pathology. The labeled gastrointestinal cancer pathological images are preprocessed; A cell nucleus segmentation model based on the ViT backbone is constructed. The cell nucleus segmentation model is trained using the preprocessed gastrointestinal cancer pathological images. During training, the two segmentation predictions generated by the cell nucleus segmentation model are respectively calculated with a preset mask to calculate the loss function. Then, the two segmentation predictions are summed as the final adjustment data and passed to the cell nucleus segmentation model.

3. The gastrointestinal cancer pathological image segmentation method based on the ViT mechanism model as described in claim 1, characterized in that, The decoder includes: An aggregation module stacks the higher-order features at each level together, and after convolution, obtains the result of higher-order feature aggregation, which is an aggregated feature T1, and generates a segmentation prediction P1 based on it. A parallel pooling attention convolution module generates three levels of feature maps based on the low-order feature f1 through pooling layers. Each level of feature map is convolved and then fused together. Finally, a low-order feature T2 is obtained by connecting an average pooling layer and a fully connected layer. A similarity aggregation module fuses the features T1 and T2 to generate another segmentation prediction P2.

4. The gastrointestinal cancer pathological image segmentation method based on the ViT mechanism model as described in claim 1, characterized in that, When preprocessing the segmented gastrointestinal cancer pathological images, size normalization and color normalization are used.

5. The gastrointestinal cancer pathological image segmentation method based on the ViT mechanism model as described in claim 2, characterized in that, When preprocessing the labeled gastrointestinal cancer pathological images, size normalization and color normalization are used.

6. The gastrointestinal cancer pathological image segmentation method based on the ViT mechanism model as described in claim 4 or 5, characterized in that, The size normalization process involves cropping the gastrointestinal cancer pathology image to a preset size.

7. The gastrointestinal cancer pathological image segmentation method based on the ViT mechanism model as described in claim 4 or 5, characterized in that, The color normalization processing method includes: Let the pathological image of the gastrointestinal cancer be image s, and the preset template image be image t; Using the following formulas (1) and (2), the optical density of image s is... Convert to The optical density of image t Convert to ,in Let s be the color appearance matrix of image s. Let be the coloring density matrix of image s. Let t be the color appearance matrix of image t. Let be the coloring density matrix of image t; （1）； （2）； in, Represents the optical density of an image. Indicates the intensity of incident light. Represents the original RGB image. The color appearance matrix of the image. The color density matrix representing the image; Using the following formula (3), the color density matrix of image s and the color density matrix of image t are registered to obtain the color density matrix of the registered image s. ; （3）； in, The color density matrix before registration of image s. Let j be the staining density matrix before image t is registered, and j be the indices of r staining agents. Used to calculate the robust pseudo maximum at 99% for each row vector. Using the following formula (4), the color density matrix of the registered image s is multiplied by the color appearance matrix of the unregistered image t to obtain the color-normalized image s; （4）； in, For the color-normalized image s, The color appearance matrix of image t before registration. The color density matrix of the registered image s.

8. A gastrointestinal cancer pathological image segmentation device based on the ViT mechanism model, characterized in that, include: The preprocessing module is used to acquire the pathological images of gastrointestinal cancer to be segmented and to preprocess the pathological images of gastrointestinal cancer to be segmented. The cell nucleus segmentation module is used to segment the preprocessed gastrointestinal cancer pathological image into cell nuclei using a preset ViT-based cell nucleus segmentation model. It generates two segmentation predictions, and the sum of these two predictions is used as the final cell nucleus segmentation result. The ViT-based cell nucleus segmentation model includes an encoder for feature capture and a decoder for aggregating features at different levels. The encoder employs a deformable self-attention mechanism. Through the encoder's calculations, four levels of features from different depths are extracted from the gastrointestinal cancer pathological image. i i = 1, 2, 3 or 4, where f1 is a low-order feature and f2, f3 and f4 are high-order features; the decoder aggregates the low-order features and the high-order features into two segmentation predictions.

9. A computer device, characterized in that, The system includes a memory and a processor, wherein the memory stores computer-readable instructions that, when executed by the processor, cause the processor to perform the steps of the gastrointestinal cancer pathological image segmentation method based on the ViT mechanism model as described in any one of claims 1 to 7.

10. A storage medium storing computer-readable instructions, characterized in that, When the computer-readable instructions are executed by one or more processors, the one or more processors perform the steps of the gastrointestinal cancer pathological image segmentation method based on the ViT mechanism model as described in any one of claims 1 to 7.

Images