An automated medical image segmentation method based on graph contrastive learning

Abstract

The present invention proposes an automated medical image segmentation method based on graph contrast learning, comprising: step 1: scale division of medical images; step 2: constructing a ViG‑CNN Block based on graph contrast learning based on the scale-divided medical image blocks; step 3: constructing a multi-scale dual-branch hybrid encoder based on the ViG‑CNN Block based on graph contrast learning; step 4: extracting medical image features based on the multi-scale dual-branch hybrid encoder; step 5: designing a BottleNeck module containing a Transformer, and updating the state of medical image features based on the BottleNeck module; step 6: designing a feature decoder based on convolution, inputting the updated medical image features into the decoder, and completing the automatic segmentation of medical images. The present invention enhances the learning ability of the encoder by transmitting high-level semantic information and low-level detail information stage by stage, and adds BottleNeck to the encoder and decoder. The experimental results on data sets of different modalities of the heart, multiple organs and brain fully demonstrate the superior performance of GCV‑UNet.

Description

Technical Field

[0001] The invention relates to an automatic medical image segmentation method based on graph contrast learning, and belongs to the technical field of image processing. Background Art

[0002] At present, deep learning-based medical image segmentation methods for different organs and multimodalities have achieved remarkable results. Among them, the typical image segmentation structure with encoder-decoder as the framework has been widely welcomed in the field of medical image segmentation because of its excellent performance. UNet and its variants U-Net++ and M-Net, which use this framework, have become the mainstream models for image segmentation. Due to the local receptive field of CNN, the network can pay more attention to local detail features, which makes it show excellent performance in the field of medical image segmentation. However, since CNN focuses on extracting local features, it has a weak understanding of the global context, which limits the further improvement of the model. The attention mechanism introduced by Transformer enables the model to better capture long-range dependencies. Its powerful expression ability in the field of NLP has attracted researchers to apply it to the field of CV. With the successful application of ViT in image classification, it has stimulated people's interest in exploring various image tasks, and has made considerable progress in subdivided fields such as image classification, image segmentation, and object detection. Given the advantages of CNN in detail information and Transformer's ability to solve long-range dependent information, some studies have tried to use hybrid CNN-Transformer to improve the segmentation ability of the model. TransUNet is the first framework in the field of medical image segmentation to use hybrid CNN-Transformer as the backbone structure. This framework fully utilizes the advantages of CNN in extracting detailed high-resolution spatial information, while combining the ability of Transformers in encoding global context. The outstanding performance of TransUNet and its variant models has pushed medical image segmentation to a new level.

[0003] On the other hand, the success of GNN on graph data has attracted great attention. Since Gorietal. proposed the concept of graph neural network in 2005, researchers have gradually explored and improved the concept and implementation of graph neural network. Unlike CNN, which is good at processing grid data, and Transformer, which is good at processing sequence data, GNN's flexible processing method enables it to still have excellent performance on non-Euclidean data. In 2022, Han, Kai, et al. proposed ViG. On the basis of dividing the image into patches, each patch is regarded as a node. By aggregating and exchanging the feature information of its own nodes and neighboring nodes, ViG successfully learns the effective representation of graph nodes. ViG's outstanding performance in image tasks has stimulated people's interest in further exploring graph neural networks (GNN) in the image field.

[0004] With the remarkable success of image contrastive learning, graph contrastive learning has once again become the focus of widespread attention. The classic graph contrastive learning (GCL) method first constructs multiple enhanced graphs by randomly augmenting the input graph, and then learns more meaningful feature representations by comparing positive samples with negative samples. At present, graph contrastive learning has been successfully applied in recommender systems, molecules and other fields. The GraphCL framework proposed by You, Yuning, et al. adopts four different data augmentation methods. The core idea of ​​DGI is to capture the structural and semantic information of the graph by maximizing the mutual information of nodes in the graph. GMI proposed by Peng, Zhen, et al. is a model that learns to maximize the graph mutual information between the input and output of the graph neural encoder. Hassani, Kaveh, and Amir Hosein Khasahmadi. introduced MVGRL, a self-supervised method for learning node and graph level representations by contrasting the structural views of the graph. GCC explores local structures through a graph sampling method based on random walk networks, which enables GCC to be extended to large-scale graph learning tasks. SUBGCON exploits the strong correlation between the central node and its sampled subgraphs to capture regional structural information.

[0005] In summary, existing medical image segmentation methods are based on CNN and Transformer models, which are better at processing grid or sequence structures, but lack the ability to process irregular images. Graph contrast learning improves the generalization and robustness of image segmentation by learning the differences between similar nodes. Its success has inspired people to introduce it into more fields. Summary of the invention

[0006] The present invention aims to solve the problem that the existing medical image segmentation technology in the prior art regards images as grids or sequence structures, which prevents the network from capturing irregular objects, and further proposes an automated medical image segmentation method based on graph contrast learning.

[0007] The technical solution adopted by the present invention to solve the above problems is: the present invention proposes an automated medical image segmentation method based on graph contrast learning, comprising:

[0008] Step 1: Scale the medical image;

[0009] Step 2: Construct ViG-C Block based on graph contrast learning based on the scale-divided medical image blocks;

[0010] Step 3: Construct a multi-scale dual-branch hybrid encoder based on ViG-C Block of graph contrastive learning;

[0011] Step 4: Extract medical image features based on a multi-scale dual-branch hybrid encoder;

[0012] Step 5: Design a BottleNeck module containing Transformer and update the status of medical image features based on the BottleNeck module;

[0013] Step 6: Design a convolution-based feature decoder and input the updated medical image features into the decoder to complete the automatic segmentation of the medical image.

[0014] Optionally, the step of scaling the medical image in step 1 includes:

[0015] Let the medical image x∈R H×W×C Divide the scale to obtain Medical image blocks of size 4*4;

[0016] The scale-divided medical image blocks are transformed to obtain feature vectors, and each feature vector is used as a node.

[0017] Optionally, the steps of constructing the ViG-C Block based on graph contrastive learning in step 2 include:

[0018] Step 2.1: Perform graph convolution operation on the scaled medical image blocks based on the K nearest neighbor algorithm;

[0019] Step 2.2: Insert a nonlinear activation function after the graph convolution operation, and perform feature transformation on each node in the medical image block after the graph convolution operation using a feedforward network containing two fully connected layers;

[0020] Step 2.3: Obtain sub-graphs G1 and G2 of the scale-divided medical image blocks based on the graph contrast learning algorithm and ViG-C Block;

[0021] Step 2.4: Based on the similar subgraphs G1 and G2 of the scale-divided medical image blocks, the structural information of the medical image is captured and the graph contrast loss is calculated to complete the construction of the ViG-C Block for graph contrast learning;

[0022] The expression of the nonlinear activation function is:

[0023] Y=GeLU(GraphConv(XW1))W2+X(1)

[0024] In formula (1), W1 and W2 are the weights of the linear layer, GeLU is the activation function, and X is the medical image block after the graph convolution operation;

[0025] The expression for feature conversion of medical image blocks after scale division is:

[0026] Z=GeLU(YW1′)W2′+Y(2)

[0027] In formula (2), W1′ and W2′ are the weights of the fully connected layer, and Y is the nonlinear activation function.

[0028] Optionally, the step of performing a graph convolution operation on the scale-divided medical image block in step 2.1 includes:

[0029] Step 2.1.1: For each node v in the medical image block after scale division i is the central node, calculate the central node v i With adjacent nodes N(v i ) between the two sides;

[0030] Step 2.1.2: Select K nodes with the smallest distance to the current central node;

[0031] Step 2.1.3: Introduce a node from node v into the selected adjacent nodes. j , v j ∈N(v i ) points to the central node v i The edge ji , construct Graph G = (V, E);

[0032] Step 2.1.4: Repeat steps 2.1.1 to 2.1.3, taking all nodes as central nodes for feature aggregation and update operations in turn to complete the graph convolution operation;

[0033] The expression of the graph convolution operation is:

[0034] X′=GraphConv(X)(3)

[0035] In formula (3), Graph is the graph function, and X′ is the medical image after graph convolution operation;

[0036] The expression for node update is:

[0037] x i ′=h(x i ,g(x i ,N(x i ),W aggregate ),W update )(4)

[0038] g(.)=[x i ,max({x j -x i |j∈N(x i )})](5)

[0039] h(.)=x i ″W update +b(6)

[0040] In formulas (4), (5) and (6), W aggregate is the weight of the node aggregation operation, W update is the weight of the node update operation, g(.) is the information between the aggregated nodes, h(.) is the feature of the updated node, and b is the bias term.

[0041] Optionally, the step of obtaining the similar subgraph G1 and the similar subgraph G2 of the medical image block after the graph convolution operation in step 2.3 includes:

[0042] Step 2.3.1: Randomly delete a preset proportion of nodes from the medical image blocks after the graph convolution operation according to the uniform distribution rule, and delete the edges corresponding to the deleted nodes;

[0043] Step 2.3.2: Randomly delete a preset proportion of edges while the nodes of the medical image blocks after the graph convolution operation remain unchanged, and perform node deletion and edge masking operations on the medical image blocks after the graph convolution operation to generate similar subgraphs G1 and G2.

[0044] Optionally, the steps of calculating the graph contrast loss in step 2.4 include:

[0045] Step 2.4.1: Obtain the feature vectors x1 and x2 of the similar subgraph G1 and the similar subgraph G2, and input the feature vectors x1 and x2 into the ViG-C Block to obtain the feature vectors z1 and z2 respectively;

[0046] Step 2.4.2: Input the feature vector z1 into the prediction head of the MLP layer to obtain the prediction vector p1, and prevent the ViG-C Block from collapsing by gradient stopping;

[0047] Step 2.4.3: Calculate the contrast loss of similar subgraph G1 and similar subgraph G2 based on the prediction vector p1 and the feature vector z2;

[0048] Step 2.4.4: Calculate the encoder's global contrast loss based on the contrast loss of similar sub-graphs G1 and G2;

[0049] The expression of the contrast loss between similar subgraph G1 and similar subgraph G2 is:

[0050]

[0051] In formula (7), p1 is the prediction vector of feature vector z1;

[0052] The calculation expression of the encoder's global contrast loss is:

[0053]

[0054] In formula (8), L c is the global contrast loss of the encoder, and p2 is the prediction vector of the feature vector z2.

[0055] Optionally, the structure of the multi-scale dual-branch hybrid encoder in step 3 includes: a CNN branch and a ViG branch;

[0056] The CNN branches include Stage 0, Stage 1, Stage 2, and Stage 3;

[0057] Stage 0 layer is used to preprocess the input medical images;

[0058] The main body of the ViG branch is the ViG-C Block for graph contrast learning;

[0059] The ViG branch includes Stage 0, Stage 1, Stage 2, and Stage 3.

[0060] Optionally, the step of extracting medical image features in step 4 includes:

[0061] Step 4.1: Input the medical image into the CNN branch to obtain the multi-scale features of all layers of the CNN branch;

[0062] The multi-scale features of all layers of the CNN branch include:

[0063] Step 4.2: Input the medical image after the graph convolution operation into the ViG branch to obtain the multi-scale features of all layers of the ViG branch;

[0064] The multi-scale features of all layers of the ViG branch include

[0065] Step 4.3: Fuse the multi-scale features of all layers of the CNN branch and the multi-scale features of all layers of the ViG branch to obtain medical image features;

[0066] The expression of feature fusion is:

[0067]

[0068] In formula (9), For CNN branch Multi-scale features of the layer, Forked from ViG The multi-scale features of the layer, f i For CNN branch Multi-scale features of layers and ViG branches The multi-scale features of the layer are fused.

[0069] Optionally, the step of updating the status of the medical image feature in step 5 includes:

[0070] The medical image features are input into the BottleNeck module, and the global information is integrated through the attention mechanism in the BottleNeck module to complete the status update of the medical image features;

[0071] The expression of the attention mechanism is:

[0072]

[0073] In formula (10), Q, K, and V are all matrices, and d h is the dimension of matrices K and V, K T is the dimension of matrices K and Q, and S is the weight calculated after the softmsx function.

[0074] Optionally, the step of completing automatic segmentation of the medical image in step 6 includes:

[0075] The updated medical image features are input into the convolution-based feature decoder, upsampled and skip-connected, and then input into the Transformer Block to generate element-level segmentation predictions, thereby completing the automatic segmentation of the medical image.

[0076] The beneficial effects of the present invention are:

[0077] 1. Based on the traditional U-shaped network image segmentation, this paper proposes a hybrid encoder composed of enhanced ViG and CNN based on image comparison, which enhances the learning ability of the encoder by transmitting high-level semantic information and low-level detail information stage by stage, and adds BottleNeck to the encoder and decoder. The experimental results on data sets of different modalities of heart, multiple organs and brain fully demonstrate the superior performance of GCV-UNet. Not only the segmentation effect is excellent, but also the generalization performance is excellent.

[0078] 2. The present invention proposes an innovative GCV-UNet network, which combines graph contrast learning and a U-shaped network enhanced with ViG, specifically for medical image segmentation. GCV-UNet cleverly combines GNN's ability to perceive global information and CNN's advantage in extracting local information. By transferring high-level semantic information and low-level detail information stage by stage, the learning ability of the encoder is enhanced. In addition, through graph contrast learning, the present invention further improves the generalization ability of ViG. In particular, the Transformerblock in GCV-UNet can better capture the long-range dependencies between features. The experimental results on data sets of different modalities of heart, multiple organs and brain fully prove that GCV-UNet has achieved excellent segmentation effects on DSC and HD95 indicators on multiple data sets, especially on irregular brain tumor data sets. The performance is far higher than other models. Other models often only use pure CNN, or CNN-Transfomer hybrid models. The ViG branch in GCV-UNet is better at processing complex images, especially migrating graph contrast to image segmentation, and further improving the segmentation performance of the model by improving the generalization ability of ViG. BRIEF DESCRIPTION OF THE DRAWINGS

[0079] Figure 1 is a flow chart of an automated medical image segmentation method based on graph contrast learning provided by an embodiment of the present invention;

[0080] Figure 2 A structural diagram of the ViG-C Block, encoder, decoder, and BottleNeck module provided in an embodiment of the present invention;

[0081] Figure 3 A structural diagram of the ViG-C Block provided in an embodiment of the present invention;

[0082] Figure 4 A comparison chart of the segmentation results of various methods on the ACDC dataset provided by an embodiment of the present invention;

[0083] Figure 5The segmentation result diagram of various methods of the Synapse multi-organ CT data set provided by the embodiment of the present invention;

[0084] Figure 6 This is a visualization of the segmentation results of various methods on the BraTS dataset provided by an embodiment of the present invention.

[0085] In the figure, CNN is the CNN neural network, ViG is the ViG neural network, Transformer is the attention mechanism, C is the number of channels, H is the image height, W is the image width, and R is the RGB image. Example

[0086] Combination Figure 1-3 This embodiment is described as follows. Figure 1 and Figure 2 As shown, the steps of an automated medical image segmentation method based on graph contrast learning proposed by the present invention include:

[0087] Step S1: Construct ViG-C Block based on graph contrast learning;

[0088] The structure of ViG-C Block is as follows Figure 3 As shown;

[0089] Further, step S1 includes:

[0090] Step S101: For a given medical image x∈R H×W×C Divide into scales patches and convert them into feature vectors, treating them as different nodes;

[0091] Step S102: constructing a graph using a K-nearest neighbor algorithm;

[0092] Further, step S102 includes:

[0093] Step S10201: Each node v in the medical image block after scale division i is the central node, calculate the central node v i With adjacent nodes N(v i ) between the two sides;

[0094] Step S10202: Sort in ascending order of distance and select K (K=9) points with the smallest distance to the current point;

[0095] Step S10203: For the selected adjacent node, introduce a link from node v j , v j ∈N(v i ) points to the central node v i The edge ji, thus constructing G = (V, E), the construction process is expressed as G = G (x);

[0096] All nodes are regarded as central nodes in turn through feature aggregation and update operations to realize the graph convolution process. The process is described as:

[0097] X′=GraphConv(X)(1)

[0098] In formula (1), Graph is the graph function, and X′ is the medical image block after the graph convolution operation;

[0099] The updated node is represented as:

[0100] x i ′=h(x i ,g(x i ,N(x i ),W aggregate ),W update )(2)

[0101] g(.)=[x i ,max({x j -x i |j∈N(x i )})](3)

[0102] h(.)=x i ″W update +b(4)

[0103] In formulas (2), (3) and (4), W aggregate is the weight of the node aggregation operation, W update is the weight of the node update operation, g(.) is the information between the aggregated nodes, h(.) is the feature of the updated node, and b is the bias term.

[0104] Step S103: Preventing the collapse of the ViG layer and alleviating the over-smoothing phenomenon;

[0105] Further, step S103 includes:

[0106] Step S10301: In order to avoid the common layer collapse phenomenon in the GNN layer, a nonlinear activation function is inserted after the graph convolution, which can be expressed as:

[0107] Y=GeLU(GraphConv(XW1))W2+X(5)

[0108] In formula (5), W1 and W2 are the weights of the linear layer, GeLU is the activation function, and X is the medical image after the graph convolution operation;

[0109] Step S10302: Each node in the graph uses a feed-forward network with two fully connected layers to improve the feature conversion capability and further alleviate the over-smoothing phenomenon. It can be expressed as:

[0110] Z=GeLU(YW1′)W2′+Y(6)

[0111] In formula (6), W1′ and W2′ are the weights of the fully connected layer, and Y is the nonlinear activation function.

[0112] Step S104: Design graph contrast learning should be applied to ViG-CBlock to further improve the generalization ability of the model:

[0113] Further, step S104 includes:

[0114] Step S10401: given a medical image G, randomly delete a certain proportion of nodes according to a uniformly distributed probability, and delete the edges corresponding to the deleted nodes;

[0115] Step S10402: Randomly delete a preset proportion of edges on the premise that the nodes of the medical image after the graph convolution operation remain unchanged, that is, discard the corresponding relationship between the nodes, and the shielding of the edge attributes also obeys the uniform probability distribution;

[0116] Step S10403: Generate similar subgraphs G1 and G2 by deleting nodes and masking edge attributes on the original graph G. The two generated subgraphs can still be regarded as the same as the original graph, and then maximize the two subgraphs to better capture the structural information of the original graph.

[0117] Step S105: Calculate graph contrast loss;

[0118] Further, step S105 includes: the feature vectors x1 and x2 of the two enhanced images are respectively sent to the ViG Block (encoder f sharing the same parameters), and two feature vectors z1=f(x1) and z2=f(x2) are obtained. For z1, it is sent to a prediction head of an MLP layer to obtain a prediction vector p1. The gradient stop is used to prevent the neural network from collapsing.

[0119] Following SimSiam, the present invention uses cosine similarity to calculate this contrast loss:

[0120]

[0121] In formula (7), p1 is the prediction vector of feature vector z1;

[0122] The global contrast loss of the encoder is defined as:

[0123]

[0124] In formula (8), L c is the global contrast loss of the encoder, and p2 is the prediction vector of the feature vector z2.

[0125] Step S2: Design a multi-scale dual-branch hybrid encoder to extract image features, and use the ViG-C Block in step S1 as the main body of the ViG branch;

[0126] The structure of the multi-scale dual-branch hybrid encoder includes: a CNN branch and a ViG branch;

[0127] The CNN branches include Stage 0, Stage 1, Stage 2, and Stage 3;

[0128] Stage 0 layer is used to preprocess the input medical images;

[0129] The main body of the ViG branch is the ViG-C Block for graph contrast learning;

[0130] The ViG branch includes Stage 0, Stage 1, Stage 2, and Stage 3.

[0131] Further, step S2 includes:

[0132] Step S201: construct a CNN branch. The ResNet50 in the CNN branch consists of Stage 0 including data preprocessing and other 3 stages, so that rich low-level spatial information can be obtained through a shallower model.

[0133] The four multi-scale features generated by all layers of the CNN branch include:

[0134] Step S202: inputting the medical image after the graph convolution operation into the ViG branch to obtain multi-scale features of all layers of the ViG branch;

[0135] The four multi-scale features generated by all layers of the ViG branch include

[0136] Step S203: Fusing the multi-scale features of all layers of the CNN branch and the multi-scale features of all layers of the ViG branch to obtain medical image features;

[0137] The expression of feature fusion is:

[0138]

[0139] In formula (9), For CNN branch Multi-scale features of the layer, Forked from ViG The multi-scale features of the layer, f i For CNN branch Multi-scale features of layers and ViG branches The multi-scale features of the layer are fused.

[0140] Step S3: Design a BottleNeck module containing Transformer to learn the long-range dependencies between the features extracted in step S2;

[0141] The present invention designs a BottleNeck between the encoder and the decoder to introduce more long-range dependencies in feature representation and enhance the model's attention to feature regions. The core principle of Transformer Block is the Self-Attention mechanism. This mechanism updates the state of features by integrating global information in each layer:

[0142] The expression of the attention mechanism is:

[0143]

[0144] In formula (10), Q, K, and V are all matrices, and d h is the dimension of matrices K and V, K T is the dimension of matrices K and Q, and S is the weight calculated after the softmsx function.

[0145] Step S4: Design a convolution-based feature decoder to ultimately achieve automatic image segmentation;

[0146] Further, step S4 includes:

[0147] The decoder is divided into three stages. In addition to conventional upsampling and skip connections, the present invention also adds Transformer Block. Specifically, the fused features are passed through TransformerBlock to build long-range dependencies, which significantly improves the model's decoding ability and ability to perceive global and local information. Ultimately, high-level semantic information can be restored to the original image resolution, and pixel-level segmentation predictions are generated, achieving accurate recovery of image information.

[0148] Example 2

[0149] Combination Figure 4-6 To illustrate this implementation, the model of the present invention is compared with the following five models:

[0150] 1) Unet: The structure of the UNet model consists of a downsampling path and an upsampling path. Their combined shape is similar to the letter U. Downsampling captures the contextual information of the image and gradually reduces the spatial dimension through continuous convolution and maximum pooling layers; the upsampling operation gradually restores the spatial dimension of the image, while retaining the position information through jump links combined with the corresponding feature maps in the contraction path.

[0151] 2) TransUnet: TransUnet is the first architecture to apply Transformer to medical image segmentation. It uses Transformer as an encoder to extract global features and CNN as a decoder for accurate local feature reconstruction and upsampling.

[0152] 3) Swin-Unet: Swin-Unet adapts to different scales of images by using hierarchical Transformers and introduces a shift window mechanism to improve computational efficiency, capturing long-distance dependencies and rich contextual information.

[0153] 4) LeViT Unet: LeViT is used as an encoder to capture rich global features, while the UNet structure is responsible for fine local feature recovery and upsampling.

[0154] 5) DA-TransUNet: It integrates the dual attention mechanism and puts it before the Transformer layer of the encoder. This can enhance its feature extraction ability,

[0155] The above models are all based on the U-shaped structure and adopt the ideas of CNN and Transformer. On this basis, the present invention uses GNN to flexibly capture the complex relationships between nodes and process complex images.

[0156] Brain tumor segmentation dataset: In the 2020 Brain Tumor Segmentation Automatic Diagnosis Challenge, organizers provided brain scan datasets from different patients, which were manually segmented by one to four reviewers according to a unified annotation protocol. Each BraTS dataset consists of 155 slices, annotated with three types of tumor tissue: enhancing tumor (ET, indicated by label 4), peritumoral edema (ED, indicated by label 2), and necrotic / non-enhanced tumor core (NCR / NET, indicated by label 1). The data used to train the model contains 295 cases, totaling 45,725 axial slices, in addition to 37 cases for model validation and another 37 for final testing.

[0157] Multi-organ dataset: The public multi-organ segmentation dataset Synapse collected 30 abdominal CT scan cases in the 2015 MICCAI challenge. Each CT scan has 18 cases as training sets and 12 cases as test sets, where each case contains 85 to 198 slices. The present invention evaluates 8 organs including spleen, liver, aorta, gallbladder, left kidney, right kidney, stomach and pancreas on Synapse.

[0158] ACDC dataset for cardiac diagnosis: ACDC consists of 100 patient cases, including 70 training samples, 10 validation samples and 20 test samples. Each sample has left ventricle (LV), right ventricle (RV) and myocardium (MYO) and contains corresponding annotations.

[0159] GCV-UNet and other models are used on the above three datasets, and the evaluation indicators are DSC and HD95:

[0160] DSC: (Dice Similarity Coefficient) The Dice coefficient is a set similarity measurement function, usually used to calculate the similarity between two samples, and its value range is between [0,1].

[0161] True Positive (TP): TP indicates the number of positive class samples correctly predicted by the model as positive class;

[0162] False Positive (FP): FP indicates the number of negative class samples that the model incorrectly predicts as positive class;

[0163] True Negative (TN): TN indicates the number of negative class samples that the model correctly predicts as negative classes;

[0164] False Negative (FN): FN indicates the number of positive samples that the model mistakenly predicts as negative samples;

[0165] DSC (Dice Similarity Coefficient):

[0166]

[0167] HD (Hausdorff Distance): is a distance metric used to measure the similarity or difference between two point sets, and is widely used in fields such as image segmentation. HD95 is an extension of Hausdorff Distance, which refers to the Hausdorff Distance that minimizes the distance between 95% of the point pairs.

[0168] HD95=max k95％ [d(X,Y),d(Y,X)]

[0169] The comparison results of various models on different data sets are as follows:

[0170] 1) Comparison results on Synapse

[0171] Table 1 shows the comparison results of GCV-UNet and other methods on the Synapse multi-organ CT dataset. In order to provide a more comprehensive evaluation result, the present invention uses the DSC index and HD95 to evaluate the performance of the model respectively. The experimental results show that the method of the present invention achieves the best performance, with segmentation accuracy up to 82.22% (DSC↑) and 20.04 (HD95↓). The excellent performance of GCV-UNet on the HD index further confirms the powerful ability of GCV-UNet in edge prediction. Although it is not always ranked first in the evaluation indicators of certain specific organs, GCV-UNet performs well in the overall segmentation effect, proving that GCV-UNet has strong robustness and reliability in processing multi-organ segmentation tasks. GCV-Unet shows the most accurate segmentation effect on the ACDC MRI dataset in Table 2. Specifically, not only does it achieve an overall accuracy of 92% on the DSC index, but it also achieves the best performance on each individual category LV, RV and MYO. GCV-UNet with enhanced ViG is particularly good at processing irregular objects. The MRI-based BraTS dataset in Table 3 demonstrates its characteristics well. GCV-UNet is still 0.9% higher than the second place on the BraTS dataset, and also shows the best results in individual organ segmentation.

[0172] Experiments further prove that GCV-UNet with enhanced ViG improves the effect of the model by flexibly integrating local and global information through the message passing mechanism, and the excellent performance of GCV-UNet on three different modal datasets proves the generalization and versatility of GCV-UNet.

[0173] Table 1: Segmentation results on the Synapse dataset

[0174]

[0175]

[0176] 2) Comparison results on ACDC

[0177] Table 2: Segmentation results of each model on ACDC

[0178] Method DSC RV Myo LV UNet 87.60 84.62 84.52 93.68 AttnUnet 86.90 83.27 84.33 93.53 nnUnet 91.59 90.25 89.10 95.41 nnformer 91.78 90.22 89.53 95.59 TransUNet 89.71 86.67 87.27 95.18 Swin-UNet 88.07 85.77 84.42 94.03 MT-Unet 90.43 86.64 89.04 95.62 LeViT-UNet 90.32 89.55 87.64 93.76 CS-UNet 91.37 89.20 89.47 95.42 <![CDATA[C 2 Former]]> 91.42 91.13 88.41 94.70 DA-TransUNet 90.97 88.77 88.58 95.55 GCV-Unet 92.01 90.54 89.72 95.77

[0179] 3) Comparison results on BraTS

[0180] Table 3: Segmentation results of each model on the BraTS dataset

[0181] Method DSC HD95 WT TC ET Unet 80.17 2.90 84.40 80.93 75.19 AttnUnet 80.23 2.87 84.52 80.04 76.14 TransUnet 80.39 2.83 84.02 80.43 76.71 Swin-Unet 79.56 2.95 83.89 79.90 74.88 MSC-UNet 81.65 2.56 85.47 81.85 77.64 DA-TransUnet 82.49 2.79 87.25 81.77 78.44 GCV-Unet 83.39 2.79 88.14 82.29 79.73

[0182] To ensure the reliability of the experimental results, this paper conducts a detailed analysis and comparison of the three data sets and explores the segmentation effects of four different segmentation methods. Figure 3 As shown in the figure, for the easy-to-segment part LV, these methods can accurately segment, but for the more difficult Myo and RV, other methods cannot accurately segment. More specifically, the DA-TransUNet method in the first row and the TransUNet and SwinUNet in the third row have obvious over-segmentation phenomenon on RV, and GCV-UNet shows accurate segmentation ability in all three parts. Figure 4 From the second row, it can be clearly seen that other methods have significant deficiencies in identifying pancreas, while the method of the present invention can successfully predict this part. From the third and fourth rows, it can be clearly seen that other methods are relatively rough in predicting the stomach boundary, while the method of the present invention shows a more accurate prediction ability. Figure 5 The second row also reveals the problem of over-segmentation of other methods when processing ED in the BraTS dataset. In contrast, the error of GCV-UNet is significantly smaller, which further confirms the superiority of GCV-UNet in noise processing. In summary, GCV-UNet has shown excellent segmentation effects on all three datasets, fully confirming the effectiveness of the hybrid encoder based on enhanced ViG and multi-scale feature fusion in the model.

[0183] The present invention conducts effective ablation experiments on each proposed module to verify its necessity and effectiveness. Taking the traditional Unet model as the benchmark, a series of experiments were conducted on the ACDC dataset, and DSC was used as the evaluation index. The experimental results show that after adding the innovation of the present invention on the basis of Unet, the performance is significantly improved. Specific data can be found in Table 4. First, the present invention applies ViG to the U-shaped segmentation network, and DSC is improved by 2.87% compared with the benchmark model. This preliminarily proves the effectiveness of ViG in the medical image segmentation task. In order to further improve the performance, the present invention improves ViG and forms an enhanced ViG by introducing Graph Contrastive Learning. Compared with the basic ViG, DSC is improved by 0.57%. This result confirms the importance of introducing enhanced graphs and learning the differences between enhanced graphs to improve model performance. Next, the present invention designs a dual-branch hybrid encoder, which consists of a CNN branch and a ViG branch. The experimental results show that this design significantly improves the performance of the model. This shows that the detailed information provided by CNN can assist ViG in building a more effective feature representation. Finally, the present invention adopts the BottleNeck structure constructed by Transformer. This structure can better capture long-range dependencies and incorporate them into feature representation, thereby further improving the segmentation performance of the model.

[0184] Table 4: The role of each module in the GCV-UNet method

[0185] Method DSC RV Myo LV Unet 87.60 84.62 84.52 93.68 ViG-Unet 90.47 88.67 87.80 94.94 ViG-UNet+CL 91.04 88.88 88.87 95.38 ViG-Unet+CL+multi-scale fusion 91.78 90.13 89.49 95.71 ViG-UNet+CL+multi-scale fusion+BottleNeck 92.01 90.54 89.72 95.77

[0186] In the hybrid encoder, the present invention deeply explores the optimal fusion strategy of the branch and verifies it through a series of ablation experiments. The present invention uses the pure VIG branch as the encoder model as a benchmark and tries a variety of fusion methods. As shown in Table 5, first, only the single-scale features obtained by CNN processing in the first stage are fed into the VIG branch. Although the result is slightly improved, the improvement is very limited, only 0.1%. Subsequently, the present invention fuses the features of the CNN branch and the ViG branch in the first two stages. The results show that the segmentation effect reaches 91.76%, and the DSC is improved by 0.18% compared with not using scale fusion. Furthermore, by adding feature fusion of 3 stages, the segmentation effect can reach 91.89%. When the present invention performs feature fusion in all 4 stages, the best segmentation effect is obtained. These experimental results strongly prove that in the hybrid encoder, the multi-scale feature fusion of the CNN branch and the ViG branch helps the network to pay more attention to the details and overall structure of the image, thereby significantly improving the segmentation effect.

[0187] Table 5: Ablation study of the impact of multi-scale fusion

[0188] Method DSC RV Myo LV GCV-UNet (noCNN Branch) 91.58 89.87 89.31 95.56 GCV-UNet+1-stage 91.62 90.06 89.23 95.57 GCV-UNet+2-stage 91.76 90.25 89.46 95.57 GCV-UNet+3-stage 91.89 90.54 89.52 95.60 GCV-UNet+multi-stage 92.01 90.54 89.72 95.77

[0189] The above is only a preferred embodiment of the present invention and does not limit the present invention in any form. Although the present invention has been disclosed as a preferred embodiment as above, it is not used to limit the present invention. Any technician familiar with this profession can make some changes or modify the technical contents disclosed above into equivalent embodiments without departing from the scope of the technical solution of the present invention. However, any simple modification, equivalent replacement and improvement made to the above embodiments without departing from the content of the technical solution of the present invention, based on the technical essence of the present invention, within the spirit and principles of the present invention, still fall within the protection scope of the technical solution of the present invention.

Claims

1. An automated medical image segmentation method based on graph contrast learning, characterized in that: The steps of the automated medical image segmentation method based on graph contrast learning include: Step 1: Scale the medical image; Step 2: constructing a ViG-C Block based on graph contrast learning based on the medical image blocks after the scale division; The steps to construct the ViG-C Block based on graph contrastive learning in step 2 include: Step 2.1: Perform graph convolution operation on the scaled medical image blocks based on the K nearest neighbor algorithm; Step 2.2: inserting a nonlinear activation function after the graph convolution operation, and performing feature conversion on each node in the medical image block after the graph convolution operation using a feedforward network containing two fully connected layers; Step 2.3: Obtain sub-graphs G1 and G2 of the scale-divided medical image blocks based on the graph contrast learning algorithm and the ViG-C Block; Step 2.4: Based on the similar subgraphs G1 and G2 of the medical image blocks after the scale division, the structural information of the medical image is captured and the graph contrast loss is calculated to complete the construction of the ViG-C Block for graph contrast learning; The expression of the nonlinear activation function is: Y=GeLU(GraphConv(XW1))W2+X(1) In formula (1), W1 and W2 are the weights of the linear layer, GeLU is the activation function, and X is the medical image block after the graph convolution operation; The expression for feature conversion of medical image blocks after scale division is: Z=GeLU(YW1′)W2′+Y(2) In formula (2), W1′ and W2′ are the weights of the fully connected layer, and Y is the nonlinear activation function; The steps to calculate the graph contrast loss in step 2.4 include: Step 2.4.1: Obtain the feature vectors x1 and x2 of the similar subgraph G1 and the similar subgraph G2, and input the feature vectors x1 and x2 into the ViG-C Block to obtain feature vectors z1 and z2 respectively; Step 2.4.2: Input the feature vector z1 into the prediction head of the MLP layer to obtain the prediction vector p1, and prevent the ViG-C Block from collapsing by gradient stopping; Step 2.4.3: Calculate the contrast loss of similar subgraph G1 and similar subgraph G2 based on the prediction vector p1 and the feature vector z2; Step 2.4.4: Calculate the global contrast loss of the encoder based on the contrast loss of the similar sub-graph G1 and the similar sub-graph G2; The expression of the contrast loss between similar subgraph G1 and similar subgraph G2 is: In formula (7), p1 is the prediction vector of feature vector z1; The calculation expression of the encoder's global contrast loss is: In formula (8), L c is the global contrast loss of the encoder, p2 is the prediction vector of the feature vector z2; Step 3: Construct a multi-scale dual-branch hybrid encoder based on the ViG-C Block of the graph contrast learning; Step 4: extracting medical image features based on the multi-scale dual-branch hybrid encoder; Step 5: Design a BottleNeck module containing a Transformer, and update the state of the medical image feature based on the BottleNeck module; Step 6: Design a convolution-based feature decoder, input the updated medical image features into the decoder, and complete the automatic segmentation of the medical image.

2. The method for automated medical image segmentation based on graph contrast learning according to claim 1, characterized in that: The steps of scaling the medical image in step 1 include: Let the medical image x∈R H×W×C Divide the scale to obtain Medical image blocks of size 4*4; The scaled medical image blocks are transformed to obtain feature vectors, and each feature vector is used as a node.

3. The method for automated medical image segmentation based on graph contrast learning according to claim 1, characterized in that: The steps of performing graph convolution operation on the scaled medical image blocks in step 2.1 include: Step 2.1.1: For each node v in the medical image block after scale division i is the central node, calculate the central node v i With adjacent nodes N(v i ) between the two sides; Step 2.1.2: Select K nodes with the smallest distance to the current central node; Step 2.1.3: Introduce a node from node v into the selected adjacent nodes. j , v j ∈N(v i ) points to the central node v i The edge ji , construct Graph G = (V, E); Step 2.1.4: Repeat steps 2.1.1 to 2.1.3, taking all nodes as central nodes for feature aggregation and update operations in turn to complete the graph convolution operation; The expression of the graph convolution operation is: X′=GraphConv(X) (3) In formula (3), Graph is the graph function, and X′ is the medical image after graph convolution operation; The expression for node update is: x i ′=h(x i ,g(x i ,N(x i ),W aggregate ),W update )(4) g(.)=[x i ,max({x j -x i |j∈N(x i )})] (5) h(.)=x″ i W update +b (6) In formulas (4), (5) and (6), W aggregate is the weight of the node aggregation operation, W update is the weight of the node update operation, g(.) is the information between the aggregated nodes, h(.) is the feature of the updated node, and b is the bias term.

4. The method for automated medical image segmentation based on graph contrast learning according to claim 1, characterized in that: The steps of obtaining similar subgraphs G1 and G2 of the medical image block after the graph convolution operation in step 2.3 include: Step 2.3.1: Randomly delete a preset proportion of nodes from the medical image blocks after the graph convolution operation according to the uniform distribution rule, and delete the edges corresponding to the deleted nodes; Step 2.3.2: Randomly delete a preset proportion of edges while the nodes of the medical image blocks after the graph convolution operation remain unchanged, and perform node deletion and edge masking operations on the medical image blocks after the graph convolution operation to generate similar subgraphs G1 and G2.

5. The method for automated medical image segmentation based on graph contrast learning according to claim 1, characterized in that: The structure of the multi-scale dual-branch hybrid encoder in step 3 includes: a CNN branch and a ViG branch; The CNN branch includes Stage 0, Stage 1, Stage 2 and Stage 3; The Stage 0 layer is used to preprocess the input medical image; The main body of the ViG branch is the ViG-C Block for graph contrast learning; The ViG branch includes Stage 0, Stage 1, Stage 2 and Stage 3.

6. The method for automated medical image segmentation based on graph contrast learning according to claim 1, characterized in that: The steps of extracting medical image features in step 4 include: Step 4.1: Input the medical image into the CNN branch to obtain multi-scale features of all layers of the CNN branch; The multi-scale features of all layers of the CNN branch include: Step 4.2: Input the medical image block after the graph convolution operation into the ViG branch to obtain multi-scale features of all layers of the ViG branch; The multi-scale features of all layers of the ViG branch include Step 4.3: Fusing the multi-scale features of all layers of the CNN branch and the multi-scale features of all layers of the ViG branch to obtain medical image features; The expression of feature fusion is: In formula (9), is the multi-scale feature of the CNN branch i-1 layer, is the multi-scale feature of the ViG branch i-1 layer, f i It is the multi-scale feature obtained by fusing the multi-scale features of the i-1 layer of the CNN branch and the multi-scale features of the i-1 layer of the ViG branch.

7. The method for automated medical image segmentation based on graph contrast learning according to claim 1, characterized in that: The step of updating the status of the medical image feature in step 5 includes: Inputting the medical image features into the BottleNeck module, integrating global information through the attention mechanism in the BottleNeck module, and completing the update of the status of the medical image features; The expression of the attention mechanism is: In formula (10), Q, K, and V are all matrices, and d h is the dimension of matrices K and V, K T is the dimension of matrices K and Q, and S is the weight calculated after the softmsx function.

8. The method for automated medical image segmentation based on graph contrast learning according to claim 1, characterized in that: The steps of completing the automatic segmentation of the medical image in step 6 include: The updated medical image features are input into the convolution-based feature decoder, upsampled and skip-connected, and input into TransformerBlock to generate element-level segmentation prediction, thereby completing the automatic segmentation of the medical image.

Images