CNN-Transform double-branch fusion-based pathological image segmentation method

Abstract

The invention discloses a pathological image segmentation method based on CNN-Transform double-branch fusion. The method comprises the following steps: 1, acquiring a pathological image data set; 2, preprocessing the data set, and adopting data enhancement methods such as horizontal overturning, vertical overturning and rotation; 3, a segmentation network model PFTransCNN based on CNN-Transform double-branch fusion is constructed, the model fuses the CNN and the Transform double branches through an interactive fusion module (Fusion), and upsampling is gradually realized through an upsampling modulation module (UFM); 4, network training is carried out, a composite loss function composed of boundary loss and Dice is used, an RAdam optimizer is used for optimizing model parameters, and the data set processed in the step 2 is input into the double-branch fusion network for end-to-end training; and 5, obtaining a segmentation result, inputting the pathological image to be segmented into the network by loading the weight of the model trained in the step 3, and finally obtaining the segmented image.

Description

Technical Field

[0001] The present invention relates to the technical field of image processing, and in particular to a pathological image segmentation method based on CNN-Transformer dual-branch fusion. Background Art

[0002] In the clinical diagnosis process, accurate segmentation and evaluation of lesion areas in pathological images are crucial for pathological analysis. This not only helps to deeply understand the disease and make accurate diagnoses, but also to formulate effective treatment plans. However, the traditional manual segmentation method is not only time-consuming and labor-intensive, but its accuracy depends largely on the personal experience and skills of the pathologist. Therefore, it is particularly important to develop an efficient and automated pathological image segmentation technology, which can not only significantly reduce the work pressure of pathologists, but also greatly reduce the risk of misdiagnosis or missed diagnosis caused by differences in personal subjective judgment.

[0003] In recent years, deep learning, especially convolutional neural networks (CNNs), has been widely used in medical image segmentation. CNNs have shown excellent performance in tasks such as segmentation, classification, and disease detection, and have become the core technology in the field of medical image processing. FCN (Fully Convolutional Network) solves the problem of inconsistent input image sizes by using convolutional layers instead of fully connected layers, allowing the network to accept image inputs of any size and perform pixel-level predictions. The U-Net network further proposed an encoder-decoder structure on this basis, and fused multi-level features through skip connections, effectively improving the accuracy of image segmentation, especially in medical image tasks with limited sample size. The introduction of deep supervision strategies and skip connections has enabled U-Net to not only improve segmentation accuracy, but also enhance the stability and robustness of the model. In addition to the traditional U-Net architecture, a variety of innovative technologies based on CNN are also constantly promoting the technical progress of medical image segmentation. With the continuous optimization of the network architecture, these technologies have significantly improved the performance of the model and improved the accuracy and efficiency when processing complex medical image tasks. Although CNN has significant advantages in local feature extraction, it still has limitations in modeling long-range dependencies, especially in capturing long-range feature associations. To solve this problem, the Transformer architecture has been gradually introduced into the field of computer vision, successfully making up for the shortcomings of CNN in global modeling. TransUNet combines the low-level feature extraction capabilities of CNN with the global modeling capabilities of Transformer, and further improves the performance and accuracy of the model through jump connections. Although the combination of Transformer and CNN has made some progress, how to better integrate the channel and position correlation between the two is still a key issue in current research. Therefore, researchers are exploring how to further improve the accuracy, robustness and computational efficiency of medical image segmentation technology by optimizing the combination of the two, so as to achieve more accurate clinical diagnosis and treatment. Summary of the invention

[0004] The purpose of this invention is to overcome the shortcomings of the prior art and propose a pathological image segmentation method based on CNN-Transformer dual-branch fusion, aiming to improve the accuracy of pathological image segmentation. This method can achieve higher segmentation accuracy while using less medical image data. The specific technical solution is as follows:

[0005] A pathological image segmentation method based on CNN-Transformer dual-branch fusion, comprising the steps of:

[0006] Step 1: obtain a pathological image dataset, which includes pathological images and corresponding label images, and divide the dataset into a training set and a test set;

[0007] Step 2: Preprocess the data set and apply data enhancement techniques, including horizontal flipping, vertical flipping, and random rotation;

[0008] Step 3: Build a segmentation network model based on CNN-Transformer dual-branch fusion, input the same-size features extracted by the two branches into the designed Fusion module for fusion, expand the receptive field through the CNN branch to capture local features, and use the Transformer branch to perform global self-attention processing to capture long-range dependencies, and then use the multi-level feature fusion graph to upsample and fuse them step by step using the designed upsample feature modulator (UFM);

[0009] Step 4: In the network training phase, a combination of boundary loss and Dice is used as the loss function, and the RAdam optimizer is used to optimize the model. The training set and test set preprocessed in step 2 are input into the network framework for end-to-end training.

[0010] Step 5: Generate the segmentation result, load the weight parameters trained in step 3, input the pre-trained pathological image into the network model for processing, and finally generate the segmented image.

[0011] As a preference: in the step 3, the network model of the dual-branch fusion of CNN-Transformer is composed of a CNN branch and a fusion of a Transformer branch, and ResNet-34 is used as the backbone network of the CNN branch. This structure converts a high-resolution image into a low-resolution image by four downsamplings, and the specific size is reduced from C×H×W to These output feature maps of different scales include and It is then passed into the interactive fusion module and fused with the output from the Transformer branch;

[0012] The Transformer branch uses ViT-B16 as the backbone to finely capture the details of global dependencies, and then uses the decoder to use the progressive upsampling method SETR to obtain feature maps, which will be combined with feature maps of the same size as the output of the CNN branch in the interactive fusion module;

[0013] The two branches are fused using the Fusion module. This module combines the global feature attention mechanism (GFA) and the local feature attention mechanism (LFA). The global feature attention mechanism obtains C×1×1 global features by performing global average pooling (GAP) on the Transformer branch features, and then evaluates the importance of all channels through the fully connected layer (FC); the local feature attention mechanism enhances the channel and spatial attention of the features from the CNN branch. First, the input features are processed by two parallel horizontal and vertical average pooling (W AvgPool and H AvgPool) to generate C×H×1 and C×1×W features respectively. After these features are spliced ​​in dimension, they are compressed through the convolution layer with a convolution kernel size of 1, and batch normalization is used to accelerate network convergence and stabilize the training process. Then the split operation is used to generate the corresponding height and width attention weights through convolution with a convolution kernel size of 11. The specific process can be expressed as follows:

[0014]

[0015] c=Conv(WAvgPool(t i )+HAvgPool(t i ));

[0016]

[0017] where g i and t i Represent the input of Transformer branch and CNN branch respectively, Sigmoid(·) represents the activation function, FC(·) represents the fully connected layer, Conv(·) represents the convolution operation, WAvgPool(·) and HAvgPool(·) represent the average pooling of the input features in the horizontal and vertical dimensions respectively, BN(·) represents batch normalization, Residual(·) represents the residual connection, represents the broadcast product, ⊙ represents the element-wise product, Represents the Transformer branch features g respectively i and CNN branch feature t i The feature map of and is the weight matrix of these two branches, and the interaction features are generated by combining the Hadamard product The local feature attention mechanism focuses on strengthening the location information of the feature, which is crucial to improving the ability of the CNN branch to capture local detail features and accurately locate the feature. Finally, the features from the two branches are and Interaction features with them are merged and processed through residual connections to form the final feature f i ,This design not only retains rich spatial information ,from global to local locations, but also effectively avoids the loss of ,boundary feature information.

[0018] As a preferred embodiment: in the step 3, a pathological image segmentation method based on CNN-Transformer dual-branch fusion is characterized in that: in the step 3, the designed upsampling feature modulator module (UFM) is used to upsample and fuse step by step, specifically, the output result obtained from the interactive fusion module is the same as the feature of the previous level at the same scale Figure 1 As the input of UFM. Then, these input features are convolved and feature concatenation is performed in the channel dimension. In order to stabilize network training and accelerate convergence, the input features are then normalized by LayerNorm. After normalization, two parallel processing paths are used: one path integrates and nonlinearly transforms the feature map through a combination of a convolution layer with a convolution kernel size of 1 and a GELU activation function to enhance the model's ability to capture complex features, and then deeply mines the large-scale edge features in the image through a convolution layer with a convolution kernel size of 11, which is crucial for achieving accurate segmentation of medical images; the other path extracts small-scale details such as image edges and textures through a convolution layer with a convolution kernel size of 1. The results on both sides are then element-wise multiplied to effectively combine large-scale structural information and small-scale detail information. Finally, feature projection is performed through a convolution layer with a convolution kernel size of 1 to output the final upsampled feature map.

[0019] As a preference: in the step 4, the network training model uses a combination of boundary loss and Dice as a loss function, and the final loss function can be described in formula language as follows:

[0020] L = α·BOU(y,y ′ )+β·Dice(y,y ′ )

[0021] Where L is the final loss function combination, y ′ represents the image segmentation prediction value, y represents the image segmentation label value, BOU represents the boundary loss function, Dice represents the Dice loss function, α and β represent the coefficients of the loss function, α is set to 0.5, and β is set to 0.5.

[0022] The beneficial effects of the present invention are as follows: This paper proposes a pathological image segmentation network based on the parallel dual-branch fusion of CNN and Transformer, named PFTransCNN, which integrates the Transformer branch on the basis of the classic U-Net architecture. The network uses ViT-B16 as the backbone of the Transformer to finely capture the details of the global dependency, while the CNN encoder is responsible for providing spatial information and low-resolution semantic features. This combination effectively promotes the deep learning of the model and improves the segmentation performance. Maintaining the complete path during the downsampling process is crucial to capturing deep detail features, which provides a solid foundation for subsequent feature fusion. To this end, this paper designs an interactive fusion module that can effectively fuse the information provided by CNN and Transformer at a fixed size, thereby constructing a complex dependency relationship between the global context and spatial information of the target feature. Then, by using the UFM module for hierarchical upsampling, the low-level spatial features and high-level semantic context of the image can be accurately captured. Finally, through the deconvolution operation, a fine and accurate predicted segmentation image is generated, thereby significantly improving the accuracy and detail performance of the segmentation result. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] Figure 1 It is the model structure of the pathological image segmentation network (PFTransCNN) based on the dual-branch fusion of CNN-Transformer;

[0024] Figure 2 It is an interactive fusion module (Fusion) that efficiently fuses the global and local encoding feature information from CNN and Transformer;

[0025] Figure 3 It is the upper feature modulation module (UFM) used in the upsampling process of the segmentation network;

[0026] Figure 4 This is the segmentation visualization result diagram of the PFTransCNN network model; DETAILED DESCRIPTION

[0027] The technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all the embodiments. The components of the embodiments of the present invention generally described and shown in the drawings here can be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of the present invention provided in the drawings is not intended to limit the scope of the claimed invention, but merely represents the selected embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without making creative work belong to the scope of protection of the present invention.

[0028] It should be noted that similar numbers and letters represent similar items in the following figures, and once an item is defined in one figure, it does not need to be further defined and explained in the subsequent figures. At the same time, in the description of the present invention, the terms "first", "second", etc. are only used to distinguish the description, and cannot be understood as indicating or implying relative importance, or implying any such actual relationship or order between these entities or operations. In addition, the terms "connected", "connected", etc. can be directly connected between elements, or indirectly connected via other elements.

[0029] Example:

[0030] In this embodiment, GlaS, MoNuSeg and SEED data sets were used to evaluate the segmentation network architecture. GlaS is a gland segmentation data set derived from colon tissue images, containing 85 images for training and 80 images for testing. The SEED data set comes from the SEED Big Data Development and Application Competition (Hualu Cup) Gastric Cancer Pathology Slice Recognition AI Challenge, which contains 65 images for training and 50 images for testing. The MoNuSeg data set contains tissue slice images captured at 40 times magnification and stained with hematoxylin and eosin (H&E), covering pathological images of multiple organs. The data set contains 30 images, with approximately 20,000 nuclear boundary annotations for training, and 14 images with a total of more than 5,000 nuclear boundary annotations for testing.

[0031] A pathological image segmentation method based on CNN and Transformer parallel dual-branch fusion, the specific steps are:

[0032] S1. Obtain a pathological image dataset, which includes pathological images and corresponding label images, and divide the dataset into a training set and a test set.

[0033] S2. Preprocess the dataset and apply data augmentation techniques, including horizontal flipping, vertical flipping, and random rotation.

[0034] S3. Construct a segmentation network model based on CNN-Transformer dual-branch fusion, input the same-size features extracted by the two branches into the designed Fusion module for fusion, expand the receptive field through the CNN branch to capture local features, and use the Transformer branch to perform global self-attention processing to capture long-range dependencies, and then use the multi-level feature fusion graph and the designed upsampled feature modulator module (UFM) to upsample and fuse step by step.

[0035] S4, in the network training phase, a combination of boundary loss and Dice is used as the loss function, and the RAdam optimizer is used to optimize the model. The training set and test set preprocessed in step 2 are input into the network framework for end-to-end training.

[0036] S5. Generate the segmentation result, load the weight parameters trained in step 3, input the pre-trained pathological image into the network model for processing, and finally generate the segmented image.

[0037] In step S3: the network model of CNN-Transformer dual-branch fusion is composed of CNN branch and Transformer branch fusion, such as Figure 1 As shown in the figure, ResNet-34 is used as the backbone network of the CNN branch. This structure converts the high-resolution image into a low-resolution image by four downsamplings. The specific size is reduced from C×H×W to These output feature maps of different scales include and It is then passed to the interactive fusion module, such as Figure 2As shown, it is fused with the output from the Transformer branch. The Transformer branch uses ViT-B16 as the backbone to finely capture the details of global dependencies, and then uses the decoder to obtain feature maps using the progressive upsampling method SETR. These feature maps will be combined with feature maps of the size matching the output of the CNN branch in the interactive fusion module. The two branches are fused using the Fusion module. This module combines the global feature attention mechanism (Global Feature Attention, GFA) and the local feature attention mechanism (Local Feature Attention, LFA). The global feature attention mechanism obtains C×1×1 global features by performing global average pooling (GlobalAverage Pooling, GAP) on the Transformer branch features, and then evaluates the importance of all channels through the fully connected layer (Fully Connected, FC); the local feature attention mechanism enhances the channel and spatial attention of the features from the CNN branch. First, the input features are processed by two parallel horizontal and vertical average pooling (W AvgPool and HAvgPool) to generate C×H×1 and C×1×W features respectively. After these features are concatenated in dimension, they are compressed by convolution layer with kernel size of 1, and batch normalization is used to accelerate network convergence and stabilize the training process. Then the split operation is used to generate attention weights corresponding to height and width respectively by convolution with kernel size of 11. The specific process can be expressed as follows:

[0038]

[0039] c=Conv(WAvgPool(t i )+HAvgPool(t i ));

[0040]

[0041] where g i and t i Represent the input of Transformer branch and CNN branch respectively, Sigmoid(·) represents the activation function, FC(·) represents the fully connected layer, Conv(·) represents the convolution operation, WAvgPool(·) and HAvgPool(·) represent the average pooling of the input features in the horizontal and vertical dimensions respectively, BN(·) represents batch normalization, Residual(·) represents the residual connection, represents the broadcast product, ⊙ represents the element-wise product, Represents the Transformer branch features g respectively iand CNN branch feature t i The feature map of and is the weight matrix of these two branches, and the interaction features are generated by combining the Hadamard product The local feature attention mechanism focuses on strengthening the location information of the feature, which is crucial to improving the ability of the CNN branch to capture local detail features and accurately locate the feature. Finally, the features from the two branches are and Interaction features with them are merged and processed through residual connections to form the final feature f i ,This design not only retains rich spatial information ,from global to local locations, but also effectively avoids the loss of ,boundary feature information.

[0042] In the pathological image segmentation method based on CNN-Transformer dual-branch fusion in step S3, the designed upsampling feature modulator module (UFM) is used to upsample and fuse step by step, such as Figure 3 Specifically, the output result obtained from the interactive fusion module is the same as the feature of the previous level at the same scale. Figure 1 As the input of UFM. Then, these input features are convolved and feature concatenation is performed in the channel dimension. In order to stabilize network training and accelerate convergence, the input features are then normalized by LayerNorm. After normalization, two parallel processing paths are used: one path integrates and nonlinearly transforms the feature map through a combination of a convolution layer with a convolution kernel size of 1 and a GELU activation function to enhance the model's ability to capture complex features, and then deeply mines the large-scale edge features in the image through a convolution layer with a convolution kernel size of 11, which is crucial for achieving accurate segmentation of medical images; the other path extracts small-scale details such as image edges and textures through a convolution layer with a convolution kernel size of 1. The results on both sides are then element-wise multiplied to effectively combine large-scale structural information and small-scale detail information. Finally, feature projection is performed through a convolution layer with a convolution kernel size of 1 to output the final upsampled feature map.

[0043] In step S4, the network training model uses a combination of boundary loss and Dice as a loss function. The final loss function can be described in formula language as follows:

[0044] L = α·BOU(y,y ′ )+β·Dice(y,y ′ )

[0045] Where L is the final loss function combination, y ′represents the image segmentation prediction value, y represents the image segmentation label value, BOU represents the boundary loss function, Dice represents the Dice loss function, α and β represent the coefficients of the loss function, α is set to 0.5, and β is set to 0.5.

[0046] This embodiment adopts the PyTorch deep learning framework. In the model training stage, the mini-batch stochastic gradient descent (SGD) algorithm is used to optimize the network parameters. At the same time, data enhancement techniques such as vertical flipping, random rotation, and horizontal flipping are performed on the data set. Specifically, each experiment contains 200 rounds of iterations, each batch contains 4 samples, the learning rate is set to 7e-5, and the momentum parameter is set to 0.9, which helps to accelerate convergence and reduce fluctuations during training. The experimental platform used in this paper is PC (Intel (R) Core (TM) i9-10940X @ 3.30GHz), the graphics card is NVIDIA GeForce RTX1080Ti, the memory capacity is 64G, the Windows 10 operating system, and the development tool is JetBrains PyCharm 2021.2.3 Community Edition, and the Python language is used to implement the relevant algorithms in the experiment.

[0047] In order to comprehensively evaluate the medical image segmentation method proposed in this paper, this paper selected three standard performance indicators: Dice Similarity Coefficient (DSC), Mean Intersection over Union (mIoU) and Accuracy (Acc). These indicators can comprehensively reflect the effect of the model in the image segmentation task.

[0048] The Dice similarity coefficient is a widely used indicator in the field of medical image segmentation. It is mainly used to evaluate the overlapping area between the predicted value and the true label, thereby calculating the similarity between the two samples. It measures the similarity by comparing the consistency between the segmentation result and the ground truth, and its value ranges from 0 to 1. The closer the Dice similarity coefficient is to 1, the higher the overlap between the model's segmentation result and the true label, and the better the segmentation performance.

[0049] The calculation formula of Dice similarity coefficient is as follows:

[0050]

[0051] Among them, X and Y represent the segmentation result set and the true label set of the model respectively, |X∩Y| is the size of the intersection of sets X and Y, indicating the number of pixels shared by the segmentation result and the truth, and X and Y are the total number of pixels in the segmentation result and the true label respectively.

[0052] Mean Intersection over Union (mIoU) is a commonly used metric for evaluating the performance of image segmentation algorithms, especially in segmentation tasks. It is obtained by calculating the IoU of each category and then taking the average. The IoU measures the similarity between the predicted segmentation border and the true border. Ideally, the predicted border completely overlaps the true border, and the IoU value is 1. The calculation formula of mIoU can be rearranged and described as follows:

[0053]

[0054] Where N represents the total number of categories, P ii P represents the total number of pixels whose true category is i and whose predicted category is also i, that is, the number of correctly predicted pixels of the i-th category, ij It represents the total number of pixels whose true category is i but is incorrectly predicted to be class j.

[0055] Accuracy (Acc) is one of the basic indicators for evaluating the performance of segmentation tasks. It measures the proportion of correctly classified samples to the total number of samples. The higher the accuracy, the better the classification performance of the model. The calculation formula for accuracy is usually defined as:

[0056]

[0057] Among them, TP, TN, FP and FN represent the number of image pixels that belong to true positive examples, true negative examples, false positive examples and false negative examples in the features respectively.

[0058] In order to comprehensively evaluate the performance of the PFTransCNN model, six models, including U-Net, U-Net++, Swin-Unet, SwinT, TransUNet, and UCTransNet, were also selected for comparative experiments.

[0059] Table 1 compares the performance of these models on the GlaS dataset, showing the advantages of PFTransCNN over other methods in the colorectal tumor segmentation task. Experimental results show that PFTransCNN achieves 91.61%, 82.96% and 91.92% in Dice similarity coefficient (DSC), mean intersection over union (mIoU) and accuracy (Acc), respectively, which are the highest values ​​in the test. Figure 4A visual comparison of GlaS pathological image segmentation was presented. The results showed that using CNN or Transformer networks alone resulted in under-segmentation, while the CNN-Transformer fusion network significantly improved the performance in image segmentation tasks. However, there is still a problem of adhesion when segmenting edge features, especially for adjacent lesion areas. PFTransCNN performed well in preserving edge details, surpassing other methods, proving that it successfully optimized detail features in pathological image edge segmentation and significantly improved segmentation accuracy and quality.

[0060] The indicator data in Table 2 show that PFTransCNN performs best in all three evaluation indicators, namely Dice similarity coefficient (DSC) 90.32%, mean intersection over union (mIoU) 81.32%, and accuracy (Acc) 91.07%. In order to intuitively compare the prediction results of different methods, Figure 4 The segmentation results of each model are shown. The visualization results show that all algorithms have different degrees of over-segmentation, which is mainly manifested in the difficulty in accurately separating glands and cell tissues, affecting the determination of clinical gland boundaries. In contrast, PFTransCNN introduces global and local attention mechanisms, pays more attention to the edge processing of pathological image segmentation during feature fusion, and thus achieves the best segmentation effect.

[0061] Table 3 shows the performance comparison of PFTransCNN with other models on the MoNuSeg cell nucleus segmentation dataset. The experimental results show that PFTransCNN performs well in three key performance indicators: Dice similarity coefficient (DSC), mean intersection over union (mIoU), and accuracy (Acc), which are 81.3%, 68.68%, and 92.33% respectively. Figure 4 As shown in the figure, this paper provides a visual comparison of the MoNuSeg nucleus segmentation results. The visualization results show that all algorithms have different degrees of over-segmentation, especially at the boundaries of adjacent nuclei, where adhesion problems frequently occur, which may affect clinical cell counts. Compared with other comparison algorithms, PFTransCNN achieves the best segmentation effect by optimizing feature fusion and modulating the upsampling process. This feature enables PFTransCNN to better distinguish adjacent cells and reduce the risk of miscounting when dealing with complex nucleus segmentation tasks, verifying its practical application value in medical image processing.

[0062] Table 1

[0063]

[0064] Table 2

[0065]

[0066] Table 3

[0067]

Claims

1. A pathological image segmentation method based on CNN-Transformer dual-branch fusion, characterized by: The following steps are involved: Step 1: obtain a pathological image dataset, which includes pathological images and corresponding label images, and divide the dataset into a training set and a test set; Step 2: Preprocess the data set and apply data enhancement techniques, including horizontal flipping, vertical flipping, and random rotation; Step 3: Build a segmentation network model based on CNN-Transformer dual-branch fusion, input the same-size features extracted by the two branches into the designed Fusion module for fusion, use the CNN branch to expand the receptive field to capture local features, and use the Transformer branch to perform global self-attention processing to capture long-range dependencies, and then use the designed upsampled feature modulator (UFM) module (UFM) to upsample and fuse step by step through the multi-level feature fusion graph; Step 4: Perform network training, use a loss function that combines boundary loss and Dice coefficient, use RAdam optimizer to optimize the model, and input the training set and test set preprocessed in step 2 into the network framework for end-to-end training; Step 5: Generate the segmentation result, load the weight parameters trained in step 3, input the pre-trained pathological image into the network model for processing, and finally generate the segmented image.

2. A pathological image segmentation method based on CNN-Transformer dual-branch fusion, characterized by: In step 3, the CNN-Transformer dual-branch fusion network model consists of a CNN branch and a Transformer branch. ResNet-34 is used as the backbone network of the CNN branch. This structure converts a high-resolution image into a low-resolution image by four downsamplings. The specific size is reduced from C×H×W to These output feature maps of different scales include and It is then passed into the interactive fusion module and fused with the output from the Transformer branch; the Transformer branch uses ViT-B16 as the backbone to finely capture the details of the global dependency, and then uses the decoder to generate feature maps using the progressive upsampling method SETR. These feature maps will be combined with the feature maps of the size matching the output of the CNN branch in the interactive fusion module; the two branches are fused through the Fusion module, which combines the global feature attention mechanism (Global Feature Attention, GFA) and the local feature attention mechanism (Local Feature Attention, LFA). The global feature attention mechanism obtains C×1×1 global features by performing global average pooling (Global Average Pooling, GAP) on the Transformer branch features, and evaluates the importance of all channels through the fully connected layer (Fully Connected, FC); the local feature attention mechanism enhances the channel and spatial attention of the features from the CNN branch. First, the input features are averaged by two parallel horizontal and vertical average pooling (W AvgPool and HAvgPool) are used to generate C×H×1 and C×1×W features respectively. After these features are concatenated in dimension, they are compressed through a convolution layer with a convolution kernel size of 1, and batch normalization is used to accelerate network convergence. Then, the segmentation operation is performed and the attention weights corresponding to the height and width are generated through a convolution layer with a convolution kernel size of 11. The specific process can be expressed by the following formula: c=Conv(WAvgPool(t i )+HAvgPool(t i )); where g i and t i Represent the input of Transformer branch and CNN branch respectively, Sigmoid(·) represents the activation function, FC(·) represents the fully connected layer, Conv(·) represents the convolution operation, WAvgPool(·) and HAvgPool(·) represent the average pooling of the input features in the horizontal and vertical dimensions respectively, BN(·) represents batch normalization, Residual(·) represents the residual connection, represents the broadcast product, represents element-wise product, Represents the Transformer branch features g respectively i and CNN branch feature t i The feature map of and is the weight matrix of these two branches, and the interaction features are generated by combining the Hadamard product The local feature attention mechanism focuses on strengthening the location information of the feature, which is crucial to improving the ability of the CNN branch to capture local detail features and accurately locate the feature. Finally, the features from the two branches are and Interaction features with them are merged and processed through residual connections to form the final feature f i ,This design not only retains rich spatial information ,from global to local locations, but also effectively avoids the loss of ,boundary feature information.

3. A pathological image segmentation method based on CNN-Transformer dual-branch fusion, characterized by: In the step three, the designed upsampling feature modulator module (UFM) is used for step-by-step upsampling and fusion, specifically: the output result obtained from the interactive fusion module is taken together with the feature map of the same scale of the previous level as the input of the UFM; then, the input features are convolved and feature splicing is performed in the channel dimension; in order to stabilize the network training and accelerate convergence, the input features are normalized by LayerNorm; after normalization, two parallel processing paths are adopted: one path integrates and nonlinearly transforms the feature map through a combination of a convolution layer with a convolution kernel size of 1 and a GELU activation function to enhance the model's ability to capture complex features, and then deeply mines the large-scale edge features in the image through a convolution layer with a convolution kernel size of 11, which is crucial for achieving accurate segmentation of medical images; the other path extracts small-scale details such as image edges and textures through a convolution layer with a convolution kernel size of 1; then the results of the two paths are element-by-element multiplied to effectively combine large-scale structural information and small-scale detail information; finally, feature projection is performed through a convolution layer with a convolution kernel size of 1 to output the final upsampled feature map.

4. A pathological image segmentation method based on CNN-Transformer dual-branch fusion, characterized by: In step 4, the network training model uses a combination of boundary loss and Dice as the loss function. The final loss function can be described by the formula: L=α·BOU(y,y ′ )+β·Dice(y,y ′ ) Where L is the final loss function combination, y ′ represents the image segmentation prediction value, y represents the image segmentation label value, BOU represents the boundary loss function, Dice represents the Dice loss function, α and β represent the coefficients of the loss function, α is set to 0.5, and β is set to 0.5.

Images