A double-end hierarchical feature fusion method for pathological image segmentation

Abstract

The application discloses a double-end hierarchical feature fusion method for pathological image segmentation, comprising the following steps: S1, the entire whole-view digital slice is divided into slice small blocks, and the blocks are put into a global encoder and a local encoder in parallel; S2, in the global encoder, a block embedding layer divides the slice into non-overlapping patches, and a feature dimension is mapped to a dimension-converted feature, and the feature is generated at each stage through a continuous Swin Transformer block and a block merging layer to generate a hierarchical global feature representation; S3, in the local encoder, a Stem layer generates patch features, and a corresponding hierarchical local feature representation is generated at each stage through a ConvNeXt block and a down-sampling layer; S4, in the decoder, global coarse-grained information and local fine-grained information of the same level of the double branches are obtained through a global-local fusion module GLFusion and a skip connection; S5, pixel-level segmentation of the image is obtained, and all slice block segmentation results are combined to restore the final WSI tumor distribution map. The application has high practical application value.

Description

Technical Field

[0001] This invention relates to the fields of pathological image segmentation and computer application technology, and more specifically, to a dual-level hierarchical feature fusion method for pathological image segmentation. Background Technology

[0002] Cancer is one of the leading causes of death worldwide, with lung cancer, liver cancer, stomach cancer, and breast cancer being the top four cancers causing cancer-related deaths. In 2020, China accounted for 24% of new confirmed cancer cases globally and 30% of all cancer-related deaths worldwide. The World Health Organization reports that the global cancer burden is projected to increase by 29.4 million new cases by 2040. Currently, most cancer patients are diagnosed at middle or late stages, which significantly reduces the probability of a cure. Therefore, early cancer screening and timely diagnosis are crucial for improving patient prognosis and quality of life.

[0003] Histopathological images are the gold standard for cancer diagnosis. Accurate segmentation of full-field digital sections (WSI) stained with hematoxylin and eosin (H&E) has become an indispensable reference for surgical evaluation, significantly increasing the survival probability for early cancer diagnosis. Unfortunately, traditional manual pathological analysis relies on visual identification through a microscope, a difficult, time-consuming, and inefficient process. Even professional pathologists can make errors due to subjective diagnosis, leading to misdiagnosis and prognostic uncertainty. Therefore, there is an urgent need for a fully automated and reliable WSI cancer detection method that can significantly reduce the burden on pathologists.

[0004] In recent years, deep learning technology has been widely applied to histopathological image tasks, demonstrating powerful learning capabilities. However, the choice of pathological image segmentation model still depends on the disease type, density, and target size of the dataset. Different models may be adapted to different scenarios, which limits the segmentation effect. Excellent segmentation performance and generalization ability are the goals pursued by most experiments. Inspired by recent research, convolutional neural networks (CNNs) are effective in learning discriminative features and extracting generalizable priors. Due to the characteristics of convolutional operations, they are good at acquiring local information. However, as the network depth increases and the spatial resolution gradually decreases, the local information that is crucial for dense pixel segmentation tasks is easily discarded. At the same time, deep networks with small medical image datasets are often unstable and prone to overfitting. Transformers are good at learning explicit global and long-range semantic information interactions and have high interpretability, but they have limitations in capturing fine-grained details in medical images.

[0005] To fully leverage the advantages of CNNs and Transformers, it is necessary to develop a dual-end hierarchical feature fusion method for pathological image segmentation, which can facilitate the automated detection of cancerous lesion areas for subsequent early screening and timely diagnosis. Summary of the Invention

[0006] The purpose of this invention is to provide a dual-level hierarchical feature fusion method for pathological image segmentation, so as to overcome the defects of the existing technology.

[0007] To achieve the above objectives, the technical solution adopted by the present invention is as follows:

[0008] A dual-end hierarchical feature fusion method for pathological image segmentation includes the following steps:

[0009] S1. Determine the entire full-view digital slice X∈R with length H and width W. H×W×3 Divide the image into small slices x∈R with length h and width w. h×w×3 Then, it is fed into the global encoder and the local encoder in parallel, where R is the spatial dimension;

[0010] S2. In the global encoder, the block embedding layer segments small image slices into non-overlapping patches, maps the feature dimension to the number C, and the feature size is (w / 4, h / 4, C). The mapped and transformed features are processed through successive SwinTransformer blocks and block merging layers at various stages to generate hierarchical global features.

[0011] S3. In the local encoder, the Stem layer generates patch features of size (w / 4, h / 4, C), and generates corresponding local features at each stage through the ConvNeXt block and the downsampling layer.

[0012] S4. In the decoder, by using the global-local fusion module GLFusion, global coarse-grained feature information and local fine-grained feature information are obtained through skip connections during the upsampling process, so as to perform accurate segmentation in the future.

[0013] S5. The linear mapping layer is applied to the output of the last GLFusion module in the upsampling process to obtain pixel-level segmentation of the image, and all slice block segmentation results are merged to restore the final distribution map.

[0014] Furthermore, the block embedding layer, Swing Transformer block, and block merging layer of the global encoder in step S2 are implemented in the following way:

[0015] S21, The block embedding layer slices the input small image x∈R h×w×3The patch is converted into non-overlapping patches of size 4×4, with each patch treated as a token. The features after the block embedding layer are denoted as follows: (This is achieved by mapping the 4×4×3 patch feature dimensions to an arbitrary dimension C.) The calculation formula is as follows:

[0016]

[0017] S22, Features after block embedding layer The token count is maintained by facilitating communication with nearby windows through a series of Swing Transformer blocks. These Swing Transformer blocks are constructed based on sliding windows and appear in pairs. The first Swing Transformer block consists of a LayerNorm layer, a window-based multi-head self-attention module (W-MSA), residual connections, and a multilayer perceptron (MLP). The next Swing Transformer block replaces the W-MSA with a shift-window-based multi-head self-attention module (SW-MSA). The calculation formula for the Swing Transformer block is as follows:

[0018]

[0019]

[0020]

[0021]

[0022] in, This represents the output of the sum of residuals in stage i. This represents the global feature calculated in stage i;

[0023] S23. The calculation process of the Swin Transformer block in stage i can be summarized by formulas (2), (3), (4) and (5):

[0024]

[0025] S24. Using formulas (1) and (6), the global feature calculation process of stage 1 is expressed as follows:

[0026]

[0027] S25. In the process of network deepening, the block merging layer connects the features of each group of 2×2 adjacent patches and uses a mapping layer to downsample by a factor of 2 and double the channel dimension. The global feature calculation process of stages 2, 3, and 4 is represented as follows:

[0028]

[0029] Furthermore, the Stem layer, ConvNeXt block, and downsampling layer of the local encoder in step S3 are implemented through the following steps:

[0030] S31, the Stem layer uses a convolution operation with an output channel of C, a kernel size of 4, and a stride of 4, and passes it through an LN layer. The computation result of the Stem layer can be expressed as:

[0031]

[0032] S32, ConvNeXt blocks facilitate local feature fusion. Local features are passed through a 7×7 large-kernel channel-wise convolution, followed by an LN layer, and then through a first pointwise convolution to adjust the hidden dimensions to four times the input dimensions, forming an inverted bottleneck residual design. A second pointwise convolution adjusts the hidden dimensions to the input dimensions. The output of a block is formed by residual connections with the input features. The computation process of a ConvNeXt block is represented as follows:

[0033]

[0034] In the formula, This represents the characteristics calculated in stage i. The characteristics at the bottleneck in the middle are represented. The structure of each ConvNeXt block is the same. According to Equation (10), the process of cascading the output of the three consecutive ConvNeXt blocks in stage i is summarized as follows:

[0035]

[0036] S33. Through equations (9) and (11), the local feature calculation process of stage 1 is expressed as follows:

[0037]

[0038] S34. The downsampling layer achieves downsampling by a factor of 2 and double the channel dimension through a convolution operation with a kernel size and stride of 2 between each stage. Simultaneously, it passes through an LN layer. Therefore, the local feature calculation process for stages 2, 3, and 4 can be represented as follows:

[0039]

[0040] Furthermore, the GLFusion module in the decoder in step S4 is implemented through the following operations:

[0041] S41. A double-skip connection is used to fuse high-level global context information and low-level precise details from both branches of CNN and Transformer. The GLFusion module uses addition to fuse local and global features at the same level. The result is then skipped with the previous fusion module. Convolution, batch normalization, and ReLU activation functions are used to make the output size consistent with the input channel dimension. This process is described as follows:

[0042]

[0043] In the formula, These are the features in stage i of the local encoder and the global encoder, respectively, f i For GLFusioni i The module's output;

[0044] S42, x sc Through a module that functions oppositely to the downsampling layer and block merging layer, a cross-scale expansion layer enables the module to achieve upsampling with double the resolution and halving the channel dimension, or upsampling with four times the resolution and preserving the channel dimension. The operation process of the cross-scale expansion layer is represented as follows:

[0045]

[0046] In the formula, T represents the number of cross-scale convolution operations used. Since GLFusion4 is the bottom module of the model and has no output from the previous module, the GLFusion4 calculation process can be expressed as:

[0047]

[0048] S43, GLFusion 2,3 The calculation process is expressed as follows:

[0049]

[0050] The calculation process for S44 and GLFusion1 is as follows:

[0051]

[0052] Furthermore, step S42 is specifically implemented by using multiple transposed convolutions with the same amplitude but different kernel sizes. Each patch has the same center but different scales, and different receptive fields are obtained through different convolution kernels. All generated features will be concatenated in the channel dimension.

[0053] Compared with the prior art, the advantages of the present invention are as follows: The dual-end hierarchical feature fusion method for pathological image segmentation provided by the present invention exhibits excellent segmentation performance on liver cancer Liver, lung cancer WSSS4LUAD, and breast cancer BCSS datasets with different disease types, densities, and target sizes. It can be extended to the tumor microenvironment analysis of other types of diseases and has high practical application value. Attached Figure Description

[0054] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0055] Figure 1 This is a structural framework diagram of the DHUnet model in this invention.

[0056] Figure 2 This is a structural framework diagram of the GLFusion module in this invention.

[0057] Figure 3 This is a qualitative comparison diagram of different methods used on liver cancer in this invention.

[0058] Figure 4 This is a qualitative comparison chart of different methods on lung cancer WSSS4LUAD in this invention. Detailed Implementation

[0059] The preferred embodiments of the present invention will now be described in detail with reference to the accompanying drawings, so that the advantages and features of the present invention can be more easily understood by those skilled in the art, thereby providing a clearer and more explicit definition of the scope of protection of the present invention.

[0060] See Figure 1 As shown, this embodiment discloses a dual-level hierarchical feature fusion method for pathological image segmentation, including the following steps:

[0061] Step S1: Divide the entire field of view digital slice X∈R into sections of length H and width W. H×W×3 Divide the image into small slices x∈R with length h and width w. h×w×3 Then, it is fed into the global encoder and the local encoder in parallel, where R is the spatial dimension;

[0062] Step S2: In the global encoder, the block embedding layer segments small image slices into non-overlapping patches, maps the feature dimension to the number C, and the feature size is (w / 4, h / 4, C). The mapped and transformed features are processed through successive SwingTransformer blocks and block merging layers at various stages to generate hierarchical global features.

[0063] Step S3: In the local encoder, the Stem layer generates patch features of size (w / 4, h / 4, C), and generates corresponding local features at each stage through the ConvNeXt block and the downsampling layer.

[0064] Step S4: In the decoder, by using the global-local fusion module GLFusion, global coarse-grained feature information and local fine-grained feature information are obtained through skip connections during the upsampling process, so as to perform accurate segmentation in the future.

[0065] Step S5: The linear mapping layer is applied to the output of the last GLFusion module in the upsampling process to obtain pixel-level segmentation of the image, and all slice block segmentation results are merged to restore the final distribution map.

[0066] Specifically, the block embedding layer, Swing Transformer block, and block merging layer of the global encoder in step S2 are implemented in the following ways:

[0067] Step S21: The Swin Transformer is used as a major component of the global encoder in DHUnet. First, the block embedding layer slices small images x∈R from the input. h×w×3 The patch is converted into non-overlapping patches of size 4×4, with each patch treated as a token. The features after the block embedding layer are denoted as follows: (The original text appears to be incomplete and contains several typographical errors. A more accurate translation would require the full context.) The calculation formula is as follows:

[0068]

[0069] Step S22: Features after block embedding layer This is achieved through a series of Swing Transformer blocks to facilitate communication with nearby windows and maintain the token count. Unlike traditional multi-head self-attention (MSA) modules, Swing Transformer blocks are constructed based on sliding windows. Swing Transformer blocks appear in pairs. The first Swing Transformer block consists of a LayerNorm layer, a window-based multi-head self-attention module (W-MSA), residual connections, and a multilayer perceptron (MLP). The next Swing Transformer block replaces the W-MSA with a shift-window-based multi-head self-attention module (SW-MSA). This reduces the algorithm's complexity, allowing the algorithm to grow linearly with the input size. Furthermore, the non-overlapping design of the different sliding windows is more hardware-friendly, resulting in faster actual running speeds. The calculation formula for the Swing Transformer block is as follows:

[0070]

[0071]

[0072]

[0073]

[0074] in, This represents the output of the sum of residuals in stage i. This represents the global feature calculated in stage i. In this embodiment, the number of Swing Transformer blocks in each stage is adjusted from (2,2,6,2) to (2,2,2,2).

[0075] Step S23: The calculation process of the Swin Transformer block (STB) in stage i is summarized by formulas (2), (3), (4) and (5):

[0076]

[0077] Step S24: Using formulas (1) and (6), the global feature calculation process of stage 1 is expressed as follows:

[0078]

[0079] Step S25, Block Merging Layer (PM): During network deepening, the features of each group of 2×2 adjacent patches are connected and combined with a mapping layer, resulting in a downsampling of 2 times the resolution and double the channel dimension. The global feature calculation process for stages 2, 3, and 4 is represented as follows:

[0080]

[0081] In this embodiment, the Stem layer, ConvNeXt block, and downsampling layer of the local encoder in step S3 are implemented through the following steps:

[0082] Step S31: ConvNeXt serves as the main component of the local encoder. Similar to the process of the global encoder, the same image x passes through the Stem layer, specifically using a convolution operation with an output channel of C, a kernel size of 4, and a stride of 4, followed by an LN layer. The computation result of the Stem layer can be expressed as:

[0083]

[0084] Step S32 To facilitate local feature fusion, ConvNeXt blocks are used. The local features are passed through a channel-wise convolution (DWConv) with a large 7×7 kernel, followed by an LN layer, and then through a first pointwise convolution (PWConv) to adjust the hidden dimensions to four times the input dimensions, forming an inverted bottleneck residual design. A GELU activation layer is chosen for competitive performance. Simultaneously, a second pointwise convolution adjusts the hidden dimensions to the input dimensions, forming the block's output through residual connections with the input features. The computation process of a ConvNeXt block is represented as follows:

[0085]

[0086] In the formula, This represents the characteristics calculated in stage i. The characteristics at the bottleneck of the inversion are represented. The structure of each ConvNeXt block is the same. In this embodiment, the number of ConvNeXt blocks in each stage is adjusted from (3,3,9,3) to (3,3,3,3).

[0087] According to equation (10), the process of cascading output of the three consecutive ConvNeXt blocks in stage i can be summarized as follows:

[0088]

[0089] Step S33, using equations (9) and (11), the local feature calculation process of stage 1 is expressed as follows:

[0090]

[0091] Step S34: The downsampling layer (DS) achieves downsampling by a factor of 2 and double the channel dimension through a convolution operation with a kernel size and stride of 2 between each stage. Simultaneously, it passes through an LN layer. Therefore, the local feature calculation process for stages 2, 3, and 4 can be represented as follows:

[0092]

[0093] In this embodiment, the GLFusion module in the decoder in step S4 is implemented through the following operations:

[0094] Step S41: A double-skip connection is used to fuse high-level global context information and low-level precise details from both the CNN and Transformer branches, reducing spatial information loss during downsampling. The GLFusion module uses addition to fuse local and global features at the same level (offering fast computation speed and performance). The result is then skipped (SC) with the previous fusion module, using convolution, batch normalization, and ReLU activation functions to ensure the output size matches the input channel dimension. This process is described as follows:

[0095]

[0096] In the formula, These are the features in stage i of the local encoder and the global encoder, respectively, f i For GLFusioni i The module's output;

[0097] Step S42, x sc Through a module that functions oppositely to the downsampling layer and block merging layer, a cross-scale extension layer enables the module to achieve upsampling with double the resolution and halved channel dimensions, or upsampling with four times the resolution and maintaining the channel dimensions.

[0098] The specific operation is achieved by using multiple transposed convolutions with the same amplitude but different kernel sizes. Each patch has the same center but different scales, and different receptive fields are obtained through different convolution kernels. All generated features are concatenated in the channel dimension. Specific parameters are shown in Table 1. The cross-scale extension layer in GLFusion1 differs from others in that the final resolution of the input features is extended to 4×(h). i ×w i ×dim i →4h i ×4w i ×dim i ), while in other GLFusion implementations (h i ×w i ×dim i →2h i ×2w i ×dim i / 2).

[0099] Table 1 Configuration parameters for the cross-scale extension layer

[0100]

[0101] Therefore, the operation process of the cross-scale extension layer can be represented as follows:

[0102]

[0103] In the formula, T represents the number of cross-scale convolution operations used. In this embodiment, T in GLFusion1 is set to 4. 2,3,4 The value is 2. Therefore, since GLFusion4 is the bottommost module of the model and has no output from the previous module, the calculation process of GLFusion4 can be represented as:

[0104]

[0105] Step S43, GLFusion 2,3 The calculation process is expressed as follows:

[0106]

[0107] Step S44, the GLFusion1 calculation process is represented as follows:

[0108]

[0109] Experiments revealed that CNNs and Transformers models performed inconsistently on different datasets. CNNs excelled at segmenting large, sparsely distributed digital slices of liver cancer, while Transformers tended to segment smaller, densely distributed lung cancer data. This aligns with previous research, as CNNs, due to the nature of convolutional operations, lacked global information crucial for densely distributed lung tissue, while Transformers had limitations in capturing fine-grained liver tissue details in medical images. However, this invention, DHUnet, fully leverages the advantages of both CNNs and Transformers, achieving superior segmentation performance on liver cancer (Liver), lung cancer (WSSS4LUAD), and breast cancer (BCSS) datasets with varying disease types, density, and target sizes. Therefore, it can be extended to tumor microenvironment analysis for other disease types, demonstrating significant practical application value.

[0110] Table 2 below shows that, compared with these existing state-of-the-art methods, the proposed DHUnet achieves the best performance on the Dice, Jaccard, and Accuracy evaluation metrics, reaching 93.07%, 87.04%, and 97.52%, respectively, indicating that DHUnet has better internal and boundary prediction capabilities. This is because DHUnet fully utilizes the advantages of CNNs, and with the help of a dual-end hierarchical global-local feature fusion module and skip connections, it comprehensively acquires both global coarse-grained information and local fine-grained information from both branches, resulting in its remarkable segmentation performance. Therefore, DHUnet achieves the best results on the liver cancer dataset segmentation task. Compared with the best Transformer-based model TransUnet, it improves performance by 1.14%, 1.97%, and 0.46%; and compared with the best CNN-based model ConvNeXt, it improves performance by 0.18%, 0.29%, and 0.05%.

[0111] Table 2. Segmentation performance of different methods for Liver cancer.

[0112]

[0113] Figure 3 The visualization results of different methods on liver cancer are presented. From the magnified images within the black boxes, it can be seen that Transformer-based models exhibit unclear local segmentation, uneven corners, and segmentation errors, especially MedicalTransformer and SwinUnet, which lack local details. CNN models, particularly ConvNeXt and U-Net, perform relatively better. DHUnet also shows better intra-region prediction and less over-segmentation compared to other models, achieving higher segmentation accuracy, which is closely related to the design of modules such as skip connections and GLFusion.

[0114] Table 3 below shows that, compared with these existing state-of-the-art methods, the proposed DHUnet still achieves the best segmentation results, with its three segmentation metrics reaching 86.54%, 77.87%, and 91.79%, respectively. This is because DHUnet fully utilizes the advantages of Transformer, capturing global and long-term semantic information. Simultaneously, through the Cross-scale Expand Layer, it captures the multi-scale context of the two branches with the same center but different scales during the upsampling process, resulting in its remarkable segmentation performance. Therefore, DHUnet achieves the best results on the lung cancer dataset segmentation task. Compared with the best Transformer-based model, SwinUnet, it improves performance by 0.21%, 0.21%, and 0.56%; and compared with the best CNN-based model, DeepLabV3, it improves performance by 2.36%, 3.05%, and 1.07%.

[0115] Table 3. Segmentation performance of different methods on WSSS4LUAD for lung cancer

[0116]

[0117] Appendix Figure 4 The visualization results of different methods on lung cancer are presented. It can be seen that while these models all demonstrate high segmentation accuracy, they also exhibit oversegmentation and segmentation errors. DHUnet, however, maintains a high degree of similarity, overlap, and accuracy with the ground truth labels.

[0118] Table 4 shows that DHUnet also achieves competitive results on breast cancer datasets with densities between liver and lung cancer, reaching 78.88%, 68.01%, and 92.51% respectively. This is inseparable from the excellent feature extraction capabilities of the ConvNeXt and SwinTransformer blocks and the hierarchical structured design. Although TransFuse also combines CNN and Transformer, its global branch is not a pure Transformer structure and its local branch is a weak ResNet backbone feature extraction network, which may lead to insufficient global and local feature extraction capabilities. Meanwhile, deeper hierarchical structures can perform better on WSI datasets, such as SwinUnet, which shows strong segmentation capabilities closer to DHUnet on BCSS and Liver datasets. DHUnet can extract breast cancer features in both global and local dimensions and obtain better segmentation results. This also demonstrates the generalization ability and robustness of our proposed method, enabling it to exhibit excellent segmentation capabilities on WSI datasets with different disease types, densities, and target sizes.

[0119] Table 4. Segmentation performance of different methods for breast cancer BCSS

[0120]

[0121]

[0122] Although embodiments of the present invention have been described in conjunction with the accompanying drawings, the patent owner may make various modifications or alterations within the scope of the appended claims, as long as they do not exceed the protection scope described in the claims of the present invention, they shall be within the protection scope of the present invention.

Claims

1. A dual-end hierarchical feature fusion method for pathological image segmentation, characterized in that, Includes the following steps: S1, divide the whole field-of-view digital slice with length H and width W into small image slices with length h and width w S2, put the small image slices into the global encoder and the local encoder in parallel, R is the spatial dimension S3, output the global feature and the local feature S2. In the global encoder, the block embedding layer segments small image slices into non-overlapping patches, maps the feature dimension to the number C, and the feature size is (w / 4, h / 4, C). The mapped and transformed features are processed through successive Swing Transformer blocks and block merging layers at various stages to generate hierarchical global features. S3. In the local encoder, the Stem layer generates patch features of size (w / 4, h / 4, C), and generates corresponding local features at each stage through the ConvNeXt block and the downsampling layer. S4. In the decoder, by using the global-local fusion module GLFusion, global coarse-grained feature information and local fine-grained feature information are obtained through skip connections during the upsampling process, so as to perform accurate segmentation in the future. S5. The linear mapping layer is applied to the output of the last GLFusion module in the upsampling process to obtain pixel-level segmentation of the image, and all slice block segmentation results are merged to restore the final distribution map. The GLFusion module in the decoder in step S4 is implemented through the following operations: S41. A double-skip connection is used to fuse high-level global context information and low-level precise details from both branches of CNN and Transformer. The GLFusion module uses addition to fuse local and global features at the same level. The result is then skipped with the previous fusion module. Convolution, batch normalization, and ReLU activation functions are used to make the output size consistent with the input channel dimension. This process is described as follows: wherein are the features in the local and global encoder stages i, respectively, is the output of the module; S42、 The cross-scale expansion layer is implemented by a module opposite to the downsampling layer and the block merging layer. The module implements upsampling double resolution and halving channel dimension or upsampling four times resolution and keeping channel dimension. The operation process of the cross-scale expansion layer is represented as: In the formula, T represents the number of cross-scale convolution operations used. Since GLFusion4 is the bottom module of the model and has no output from the previous module, the GLFusion4 calculation process can be expressed as: S43, G LFusion 2,3 The computational procedure is represented as: The calculation process for S44 and GLFusion1 is as follows: 。 2. The dual-end hierarchical feature fusion method for pathological image segmentation according to claim 1, characterized in that, The block embedding layer, Swin Transformer block, and block merging layer of the global encoder in step S2 are implemented in the following ways: S21, the block embedding layer converts the input small image slice into non-overlapping patches of patch size 4x4, each patch is regarded as a token, the feature dimension of the 4x4x3 patch is mapped to an arbitrary dimension C, and the feature after the block embedding layer is recorded as: The calculation formula is: （1）； S22, features after block embedding layer The information communication with the nearby windows will be promoted by a series of Swin Transformer blocks, the number of tokens is maintained, the Swin Transformer blocks are based on sliding window construction, the Swin Transformer blocks appear in pairs, the first Swin Transformer block is composed of a LayerNorm layer, a window-based multi-head self-attention module W-MSA, a residual connection and a multi-layer perceptron MLP, the next Swin Transformer block replaces the W-MSA with a shift window-based multi-head self-attention module SW-MSA, and the formula for the Swin Transformer block calculation process is: （2） （3） （4） （5） wherein, represents the output of the residual addition in stage i, represents the global feature computed by stage i; S23. The calculation process of the Swin Transformer block in stage i can be summarized by formulas (2), (3), (4) and (5): S24. Using formulas (1) and (6), the global feature calculation process of stage 1 is expressed as follows: S25. In the process of network deepening, the block merging layer connects the features of each group of 2×2 adjacent patches and uses a mapping layer to downsample by a factor of 2 and double the channel dimension. The global feature calculation process of stages 2, 3, and 4 is represented as follows: 。 3. The dual-end hierarchical feature fusion method for pathological image segmentation according to claim 2, characterized in that, The Stem layer, ConvNeXt block, and downsampling layer of the local encoder in step S3 are implemented through the following steps: S31, the Stem layer, uses a convolution operation with an output channel of C, a kernel size of 4, and a stride of 4, and passes it through an LN layer. The computation result of the Stem layer can be expressed as: S32、 The local features are fused by a ConvNeXt block. The local features are processed by a 7x7 large kernel channel-wise convolution, followed by a LN layer, and then a first point-wise convolution to adjust the hidden dimension to four times the input dimension to form an inverted bottle-neck residual design. A second point-wise convolution adjusts the hidden dimension to the input dimension, and a residual connection with the input features forms the output of the block. The calculation process of a ConvNeXt block is represented as: wherein denotes the features computed in stage i, denotes the features at the bottleneck of the middle stage, the structure of each ConvNeXt block is the same, and the process of concatenating the outputs of the 3 consecutive ConvNeXt blocks in stage i according to equation (10) is summarized as: S33. Through equations (9) and (11), the local feature calculation process of stage 1 is expressed as follows: S34, the down-sampling layer realizes the resolution reduction by 2 and the channel dimension doubling between each stage through a convolution operation with a kernel size and a step size of 2, and through a LN layer, and the local feature calculation process of stages 2, 3 and 4 is represented as: 。 4. The dual-end hierarchical feature fusion method for pathology image segmentation according to claim 1, characterized in that, The step S42 is specifically realized by using multiple transpose convolutions with the same amplitude but different kernel sizes, each patch has the same center but different scales, different receptive fields are obtained through different convolution kernels, and all generated features will be concatenated in the channel dimension.

Images