Cellulose lung lesion segmentation method and network based on vit and context feature fusion

Abstract

This invention provides a method and network for segmenting cellular lung lesions based on ViT and contextual feature fusion, belonging to the field of image processing technology. The network of this invention can achieve higher precision segmentation of cellular lung lesions; it utilizes channel-mixing convolutional blocks to increase the interaction of information between different channels, fully extracting feature information of the lesion site; it employs a Transformer architecture as a feature connector between the encoder and decoder, enhancing the feature representation of global information and expanding the receptive field of the network; it uses a CFB module to fuse features from multiple stages, reducing the semantic gap between high-level and low-level features, and improving the segmentation accuracy of lesion sites and target edges; this invention is applied to cellular lung lesion segmentation.

Description

Technical Field

[0001] This invention provides a method and network for segmenting cellular lung lesions based on ViT and contextual feature fusion, belonging to the field of image processing technology. Background Technology

[0002] Interstitial lung disease (ILD) is a highly fatal, insidious, and destructive diffuse lung disease. It is also known as "honeycomb lung" because it appears as honeycomb-like, high-density shadows in parts of the lung interstitium on CT images. Currently, the diagnosis of ILD in clinical practice largely relies on specialist radiologists who analyze CT images of the patient's lungs to determine if honeycomb lung is present. With the massive increase in clinical imaging data, my country faces a severe shortage of radiologists relative to its large population. During diagnosis, specialists rely on their existing knowledge and experience to manually interpret CT images, but the heavy mental workload and long hours can easily lead to visual fatigue and errors, resulting in highly subjective diagnostic results and even misdiagnosis or missed diagnosis, thus increasing the difficulty of later treatment. Therefore, using image segmentation methods to automatically segment honeycomb lung can assist doctors in accurately diagnosing the severity of the patient's condition, improving the accuracy and efficiency of image interpretation, and providing appropriate plans for guiding clinical decisions and prognosis, which has significant clinical value.

[0003] In recent years, thanks to the powerful feature representation capabilities and complex task modeling abilities of deep learning, CNN-based methods have been widely used in the field of medical image processing, especially U-shaped convolutional neural networks composed of skip connections and encoder-decoder architectures, which have achieved significant performance in medical image segmentation. Ronneberger et al., employing the concept of "fully convolutional" architecture, first proposed an encoder-decoder network model for medical image segmentation—UNet. This model uses skip connections to fuse high-level features from the upsampling stage with low-level features from the downsampling stage, achieving good segmentation results on three medical datasets. To address the issue of different scales in medical images, IBTEHAZ et al. proposed the MultiResUNet network for skin disease segmentation. Employing residual learning, they constructed multi-residual convolutional modules and introduced residual paths into the UNet network to improve training performance. Sharp U-Net uses sharpened convolutional kernels to generate intermediate feature maps instead of skip connections in the U-shaped network, solving the oversegmentation problem caused by semantic gaps and demonstrating better segmentation performance. To obtain more accurate edge information, ALOM et al. proposed the R2U-Net model based on a recurrent neural network, using recurrent residual convolutional layers for feature accumulation. This model achieves better performance in retinal segmentation tasks while maintaining the same number of network parameters. The HDA-ResUNet network replaces the bottom convolutional layers in UNet with dilated convolutional layers incorporating channel attention mechanisms, fusing information from receptive fields of different sizes to address the problem of missing multi-scale information in the network. Building upon UNet, UNet++ employs improved dense skip connections to fuse feature information from different levels of depth, reducing the semantic gap between encoder and decoder feature maps. Chen et al. proposed the DeepLabv3+ model with an Encoder-Decoder structure, increasing the image receptive field and extracting richer contextual information by introducing dilated convolutions and spatial pyramid pooling. To address the issue of spatial feature information loss during convolution, GU et al. proposed the CE-Net model for medical image segmentation, utilizing context connectors that fuse dense convolutional blocks to generate more semantic feature maps. Similarly, CA-Net, based on an encoder-decoder structure, proposes a joint spatial attention module and a scale attention module, using attention mechanisms to recalibrate channel feature responses and enhance the expression of relevant feature channels. The CNN-based methods described above are all variants of the UNet network, and the excellent segmentation performance on numerous medical datasets has fully demonstrated the applicability of the UNet network in the field of medical segmentation. Therefore, this invention uses the UNet network as the basic model for segmenting cellular lungs.

[0004] The aforementioned work primarily addresses the issues of insufficient feature extraction and missing multi-scale information during network feature extraction. While CNN methods based on encoder-decoder structures offer significant advantages in extracting local image features, the traditional convolutional stacking operations in fully convolutional networks lead to feature redundancy, affecting segmentation performance. Furthermore, most encoder-decoder networks only recalibrate the encoder feature map, neglecting the importance of deep semantic information contained in the decoder feature map for edge information extraction, resulting in the loss of detailed information. Simultaneously, due to the limited receptive field of a single convolutional kernel, the network focuses only on a specific sub-region of the image, making it difficult to model the contextual relationships within the image. These limitations of convolutional operations pose a challenge to learning global information in images, especially crucial for pixel-level tasks such as semantic segmentation.

[0005] Recently, the emergence of Transformers has challenged the dominance of CNNs in computer vision tasks. ViT, one of the many variants of Transformers, has demonstrated outstanding performance in medical image segmentation. Chen et al. proposed the TransUNet network for multi-organ segmentation, using ViT to replace ordinary convolutional blocks as the basic encoder module of the segmentation network. Leveraging the excellent global information modeling capabilities of Transformers, they achieved precise localization of lesions. To further segment target edges accurately, the Medical Transformer uses a gated axis Transformer module to obtain more precise positional information. Therefore, medical segmentation networks based on the Transformer architecture can fully extract global features by establishing global relationships in the image, thus achieving high-quality segmentation of medical targets. Summary of the Invention

[0006] In order to overcome the shortcomings of the prior art, the technical problem to be solved by the present invention is to provide an improved method for segmenting cellular lung lesions based on ViT and contextual feature fusion.

[0007] To solve the above-mentioned technical problems, the technical solution adopted by the present invention is: a cellular lung lesion segmentation method based on ViT and contextual feature fusion, comprising the following steps:

[0008] S1: Acquire cellular lung CT image data, preprocess the raw images, and divide the preprocessed dataset into training and testing sets;

[0009] S2: Construct the basic UNet network, including a downsampled encoder, an upsampled decoder, skip connections, and a bottleneck layer;

[0010] S3: Improve the basic UNet network by changing the convolution operations in the sampling and downsampling layers of the network and using channel-mixed convolutional blocks to replace the traditional convolution structure;

[0011] The channel mixing convolution block separates the feature map into two branches, S1 and S2. Branch S1 keeps the feature map and number of channels unchanged, and performs convolution on branch S2. Then, the feature map after convolution is fused with the feature map of branch S1. Finally, channel mixing is performed to exchange information between the channels of the feature map. The resulting feature map is the same size as the upper-layer feature map. The obtained feature map is then subjected to channel separation, branch convolution, channel concatenation and channel mixing operations again.

[0012] S4: In the bottleneck layer of the network, the global relationship between each pixel in the image is modeled using a ViT-based context connector, and ViT is used to replace the high-channel convolutional module.

[0013] S5: Improve the feature fusion scheme between low-level and high-level features in the skip connection part by using a context-aware fusion module, and relearn the features;

[0014] S6: Define the hyperparameters of the segmentation model, train it using the dataset processed in S1, and obtain the loss value of the loss function and the segmentation result;

[0015] S7: Adjust network parameters based on the results, generate and save the trained lesion segmentation model, input the test set data into the trained lesion segmentation model, segment the lesions in the cell lung CT image, and output the segmentation results.

[0016] The ViT-based context connector uses six Transformer coding blocks to extract global information from the image. ViT comprises four parts:

[0017] (1) Slice embedding: By converting the original two-dimensional image into one-dimensional sequence data, the size of the slice is set according to the output image of the encoder, the image is converted into multiple slices, and the dimension of the slice is calculated;

[0018] (2) Position encoding: Mark the corresponding position information of each slice in order to recover the dimension of the image. The position information is consistent with the output of the slice embedding. Save the correct position information of each slice and calculate its dimension;

[0019] (3) Multi-head attention mechanism in Transformer encoding block: After the image slicing and location encoding information establishment stage, the image slice information is input into the Transformer encoding block to learn the relationship between slices and between each pixel in each slice, and to perform context modeling of global information; the multi-head attention layer in this part receives slice and location encoding information, learns more relevant information in different subspaces by using multiple attention heads, initializes Q and K in each attention head using different weight matrices, performs Dropout operation on the global features learned by the multi-head attention layer, and uses a normalization layer to modify the feature dimension to adapt to the subsequent output, thereby accelerating the convergence speed of the model;

[0020] (4) Multilayer perceptron layer in Transformer coding block: This part uses MLP block to learn the nonlinear relationship between features; residual connection and layer normalization operation are used in each sub-layer.

[0021] The context-aware fusion module receives low-level features from the encoder and high-level features from the decoder. First, it uses GAP (Global Average Pooling) to generate feature maps with global spatial information. Then, it uses a multilayer perceptron with shared weights to model the context information in the low-level and high-level feature maps, generating vectors h and l, where h and l represent the weight vectors in the high-level and low-level feature maps, respectively. Second, it uses residual learning to multiply the weight vectors with the two feature maps to generate a redistributed feature map. This redistributed feature map is then concatenated along the channel dimension to generate a feature map containing global context information from both local and high-level stages. Third, it uses two 3x3 convolutions to achieve weighted feature fusion and uses residual connections to receive information from the high-level features.

[0022] The encoder contains four-channel hybrid convolutional blocks for extracting high-level and low-level features of the image. Each channel hybrid convolutional block contains two convolutional layers, a batch normalization layer, and a corrected linear unit (ReLU). The feature map extracted by each channel hybrid convolutional block contains two paths. The first path connects to a max pooling layer to downsample the feature map and pass it to the next convolutional block; the other path enters a skip connection path.

[0023] The decoder contains four channel-mixed convolutional blocks identical to those in the encoder. Each channel-mixed convolutional block contains two layers of convolutional computation, batch normalization, and ReLU activation function. After each channel-mixed convolutional block, the feature map is upsampled using deconvolution. The feature map after the transposed convolution operation is expanded to twice its size.

[0024] The cellular lung lesion segmentation network based on ViT and contextual feature fusion includes an encoder, a decoder, a ViT-based context connector located in the network bottleneck layer, and four context-aware fusion modules located in the skip connection stage. The encoder includes four downsampling modules, and the decoder includes four upsampling modules. Both downsampling and upsampling modules include channel-mixed convolutional blocks. Each channel-mixed convolutional block contains two convolutional layers, a batch normalization layer, and a corrected linear unit (ReLU). The feature map extracted by each channel-mixed convolutional block in the downsampling module contains two paths. The first path connects to a max pooling layer to downsample the feature map and pass it to the next convolutional block; the other path enters the skip connection path. After each channel-mixed convolutional block in the upsampling module, the feature map is upsampled using deconvolution. The feature map after the transposed convolution operation is expanded to twice its size.

[0025] At the bottleneck layer of the network, ViT is used to slice the feature map and calculate the global relationships in the feature map;

[0026] During the skip connection stage, the feature maps obtained from the four-channel convolutional mixing blocks in the encoder and the upsampled feature maps are respectively input into the path containing the context-aware fusion module for feature enhancement.

[0027] The channel convolutional mixing block includes a channel separation module, a channel splicing module, and a channel mixing module. The channel separation module includes two branches, S1 and S2. Branch S1 keeps the feature map and the number of channels unchanged, and performs convolution on branch S2. The feature maps output by the two branches are processed by the channel splicing module and the channel mixing module. Each channel convolutional mixing block performs two channel separation, channel splicing, and channel mixing operations to output a feature map.

[0028] The ViT-based context connector includes a slice embedding module, a position encoding module, and six Transformer encoding blocks. Each Transformer encoding block contains a multi-head attention mechanism module and a multilayer perceptron module. The slice embedding module and the position encoding module are located at the beginning of the ViT module. The slice embedding module converts the original two-dimensional image into a one-dimensional sequence of slices. The position encoding module assigns corresponding position information to each slice. The multi-head attention mechanism module is located in the first half of the Transformer encoding block and receives slice and position encoding information, including multiple attention heads, Dropout operation, and a normalization layer. The multilayer perceptron module is located in the second half of the Transformer encoding block and includes a multilayer perceptron layer and a normalization layer.

[0029] The context-aware fusion module includes two parallel global average pooling layers, a multilayer perceptron layer, and a convolutional block. The two parallel global average pooling layers receive low-level features from the encoder and high-level features from the decoder, respectively, to generate feature maps with global spatial information. The multilayer perceptron uses shared weights to model the context information in the low-level and high-level feature maps to generate weight vectors. The weight vectors are multiplied by the two feature maps to generate redistributed feature maps, which are then concatenated along the channel dimension to generate feature maps containing global context information from both local and high-level stages. Weighted feature fusion is achieved using two 3*3 convolutions in the convolutional block.

[0030] The advantages of this invention compared to existing technologies are as follows: Based on deep learning technology, this invention proposes a U-shaped image segmentation network (CSC-UTNet) that combines channel-mixed convolutional blocks, ViT, and a context-aware fusion module for cellular lung segmentation. This effectively improves the problem of lost edge information at lesion sites and reduces over-segmentation. The model utilizes channel-mixed convolutional blocks to increase the interaction of information between different channels, fully extracting feature information from cellular lung lesions. ViT is used as a feature connector between the encoder and decoder to enhance the feature representation of global information and expand the receptive field of the network. The context-aware fusion module fuses features from multiple stages, reducing the semantic gap between high-level and low-level features and improving the segmentation accuracy of lesion edges. Through ablation experiments on a cellular lung dataset, CSC-UTNet achieves superior performance in evaluation metrics such as similarity intersection-union ratio, Dice coefficient, mIoU, and mDice, demonstrating stronger generalization and higher segmentation accuracy. Attached Figure Description

[0031] The present invention will be further described below with reference to the accompanying drawings:

[0032] Figure 1 This is a schematic diagram of the process of the present invention;

[0033] Figure 2 This is a schematic diagram of the network structure of the present invention;

[0034] Figure 3 This is a schematic diagram of the channel hybrid convolution block of the present invention during the upsampling stage;

[0035] Figure 4 This is a schematic diagram of the ViT-based context connector of the present invention;

[0036] Figure 5 This is a schematic diagram illustrating the calculation process of the multi-head attention mechanism in the Transformer coding block of this invention;

[0037] Figure 6 This is a schematic diagram of the context-aware fusion module of the present invention. Detailed Implementation

[0038] like Figures 1 to 6 As shown, the main improvement of the cellular lung lesion segmentation method based on ViT and contextual feature fusion in this invention is as follows:

[0039] (1) A lightweight channel hybrid convolution block is proposed to reduce the convolution calculation of some feature channels and prevent excessive redundant features from reducing the segmentation accuracy;

[0040] (2) Introduce ViT as a feature connector between the encoder and decoder in the network bottleneck layer to enhance the feature representation of global information and improve the segmentation accuracy of lesion areas;

[0041] (3) The context-aware fusion module is used to reconstruct the feature distribution in the encoder, reduce the semantic gap caused by the mismatch of the receptive domain, enhance the semantic correlation between the encoder and the decoder, and realize the context feature fusion of semantic matching.

[0042] This invention employs the UNet network as the baseline segmentation network for cellular lung segmentation. This network consists of an encoder and a decoder, with the bottleneck layer and skip connection module connecting the encoder and decoder using high-channel convolution and feature addition, respectively. The proposed medical image segmentation network CSC-UTNet structure is as follows: Figure 2 As shown. Specifically, the encoder contains four-channel hybrid convolutional blocks for extracting high-level and low-level feature information from the image. Each convolutional block contains two convolutional layers, a batch normalization layer, and a Rectified Linear Unit (ReLU). The feature map extracted by each convolutional block has two paths: the first path connects to a max-pooling layer to downsample the feature map before passing it to the next convolutional block; the other path enters the skip connection path. Similar to the encoder, the decoder has four identical convolutional blocks, each containing two convolutional layers, batch normalization, and a ReLU activation function. After each convolutional block, the feature map is upsampled using deconvolution, and the feature map after transposed convolution is expanded to twice its size. At the bottleneck layer of the network, ViT is used to slice the feature map, improving the interaction of global information in the sliced ​​features. In the skip connection stage, the feature maps obtained from the four convolutional blocks in the encoder are combined with the upsampled feature maps. Figure 1 For each corresponding input path containing the context-aware fusion module, feature enhancement operations are performed. By improving the feature fusion scheme for low-level and high-level features, semantic gaps between features are bridged. During feature fusion, concatenation operations are used to retain more beneficial features, avoiding the loss of original features caused by direct addition operations, which would affect the segmentation results.

[0043] The improved network portion of this invention will be described in the following sections.

[0044] Channel Hybrid Convolutional Block: In current mainstream segmentation models, many networks use high-level channel convolutions to extract deep abstract features of images to obtain better segmentation results. However, stacking a large number of convolutional layers increases the computational cost of the model, especially increasing the number of channels, which leads to an excessive number of parameters. Therefore, this invention proposes a novel channel hybrid convolutional block, the structure of which is as follows: Figure 3 As shown, this module splits the feature map into two branches, S1 and S2, after separation. Branch S1 maintains the same feature map and number of channels, while branch S2 is convolved. The computational cost of convolution in branch S2 is reduced by 1 / 1-r times compared to ordinary convolution in the original network. Then, the convolved feature map is fused with the feature map from branch S1. Finally, a channel mixing operation is performed to exchange information between the channels of the feature map, enhancing the interaction between different groups of feature maps. At this point, the obtained feature map is the same size as the previous layer's feature map. Channel separation, branch convolution, channel concatenation, and channel mixing operations are then performed on the obtained feature map again. Finally, an average pooling layer is used to downsample the feature map M, and the output feature map is used as the input to the next channel mixing convolution block. Channel mixing and channel split operations not only effectively reduce the computational cost during convolution but also increase the interaction of information between different channels, effectively extracting local feature information of the lesion site and preventing feature redundancy from wasting computational resources.

[0045] ViT-based Context Connector: Currently, many existing segmentation models use convolutional operations for feature extraction. While these methods offer strong local feature extraction capabilities, their fixed kernel size affects the receptive field of the image, and multiple convolutional operations lead to the loss of global information and redundancy of local features. Furthermore, due to the irregular and blurred edges of honeycomb lung lesions, CNNs often fail to extract sufficient features, easily losing edge information. To address these issues and construct a feature extractor that focuses more on global information and has a larger receptive field, this invention introduces ViT into the bottleneck layer of a U-shaped network. By enriching the encoded representation information in different subspaces of the network, the global receptive field of the image is expanded. The acquired global information is then fused with the local information from the encoder to extract more effective feature information.

[0046] To improve model performance without excessively increasing computational cost, the ViT-based context connector of this invention employs six Transformer coding blocks to extract global information from the image, providing more favorable features for the learning of the U-shaped network decoder. The model structure of the ViT-based context connector is as follows: Figure 4 and Figure 5 As shown, it mainly consists of four parts:

[0047] (1) Slice embedding. By converting the original two-dimensional image into one-dimensional sequence data, the visual task is transformed into an NLP problem. Since ViT is located in the bottleneck layer of the U-shaped network, the size of the encoder's output image is X∈R. C×H×W Set the size of the slice to patch∈R p×p The image is transformed into (H / p × W / p) slices, where the slice size is x. p i ∈R N×(p×p×C) (N here refers to the number of slices).

[0048] (2) Position Encoding. This part assigns the corresponding position information to each slice to recover the image's dimensions, ensuring the image structure remains intact. Similarly, the position information is consistent with the output of the slice embedding, preserving the correct position information for each slice, with a dimension of E. pos ∈R (p2∙C)×D) (Here, D represents the dimension of the linear mapping layer).

[0049] (3) Multi-head attention mechanism in Transformer encoding block. After the image slicing and location encoding information establishment stage, the image slice information is input into the Transformer encoding block to learn the relationship between slices and between each pixel in each slice, and to perform contextual modeling of global information. The multi-head attention layer in this part receives slice and location encoding information. This invention sets up 4 attention heads, and learns more relevant information in different subspaces by using multiple attention heads. Q and K in each attention head are initialized with different weight matrices to express more different features. In order to prevent overfitting, Dropout operation is applied to the global features learned by the multi-head attention layer, and normalization layer is used to modify the feature dimension to adapt to the subsequent output, thereby accelerating the convergence speed of the model.

[0050] (4) Multilayer perceptron layer in the Transformer encoding block. This part uses MLP blocks to learn the nonlinear relationship between features; similarly, to prevent gradient vanishing or gradient explosion and to speed up model convergence, residual connections and layer normalization operations are used in each sub-layer. Therefore, the calculation process of the Transformer encoding block is shown in equations (1) to (3).

[0051] (1)

[0052] (2)

[0053] (3)

[0054] In the above formula: E and E represent the slice size and the output layer of linear projection, respectively. pos This indicates the position information of each slice. and These represent the outputs of the multi-head attention layer and the MLP layer, respectively.

[0055] Context-Aware Fusion Module: In traditional U-shaped networks, low-level semantic information and high-level abstract features are fused through direct connections between encoder and decoder features, ignoring the contextual relationships between features and weakening the expression of important features. This invention introduces a context-aware fusion module to reconstruct the skip connection structure. This module utilizes residual learning to fuse features from multiple stages, enhancing the expression of important features in the global context and suppressing irrelevant background noise from low-order features. The structure of the context-aware fusion module is as follows: Figure 6 As shown, this module receives low-level features from the encoder and high-level features Fh and Fl from the decoder. First, it uses GAP (Global Average Pooling) to generate feature maps with global spatial information. Then, it uses a multilayer perceptron with shared weights to model the contextual information in the low-level and high-level feature maps, generating vectors h and l, where h and l represent the weight vectors in the high-level and low-level feature maps, respectively. Second, using residual learning, the weight vectors are multiplied by the two feature maps to generate redistributed feature maps Fl and Fh. These are then concatenated along the channel dimension to generate feature maps containing global contextual information from both local and high-level stages. Third, it uses two 3x3 convolutions to achieve weighted feature fusion and uses residual connections to receive information from the high-level features, thereby capturing more effective features.

[0056] Regarding the specific structure of this invention, it should be noted that the connection relationships between the various component modules used in this invention are definite and achievable. Except as specifically described in the embodiments, their specific connection relationships can bring about corresponding technical effects and solve the technical problems proposed by this invention without relying on the execution of corresponding software programs. The models of the components, modules, and specific components appearing in this invention, the connection methods between them, and the conventional usage methods and expected technical effects brought about by the above technical features, unless specifically described, are all publicly disclosed content in patents, journal articles, technical manuals, technical dictionaries, and textbooks that can be obtained by those skilled in the art before the application date, or belong to conventional technology, common knowledge, and other existing technologies in this field. There is no need to elaborate, which makes the technical solution provided in this case clear, complete, and achievable, and can reproduce or obtain corresponding physical products based on this technical means.

[0057] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A cellular lung lesion segmentation method based on ViT and contextual feature fusion, characterized in that: Includes the following steps: S1: Acquire cellular lung CT image data, preprocess the raw images, and divide the preprocessed dataset into training and testing sets; S2: Construct the basic UNet network, including a downsampled encoder, an upsampled decoder, skip connections, and a bottleneck layer; S3: Improve the basic UNet network by changing the convolutional operations in the sampling and downsampling layers. Replace the traditional convolutional structure with channel-mixed convolutional blocks. The resulting network includes an encoder, a decoder, a ViT-based context connector in the network bottleneck layer, and four context-aware fusion modules in the skip connection stage. The encoder includes four downsampling modules, and the decoder includes four upsampling modules. Both downsampling and upsampling modules include channel-mixed convolutional blocks. Each channel-mixed convolutional block contains two convolutional layers, a batch normalization layer, and a ReLU (Rectified Linear Unit). The feature map extracted by each channel-mixed convolutional block in the downsampling module has two paths. The first path connects to a max pooling layer to downsample the feature map and pass it to the next convolutional block; the other path enters the skip connection path. Each channel-mixed convolutional block in the upsampling module then uses deconvolution to upsample the feature map. The feature map after the transposed convolution operation is expanded to twice its size. The channel mixing convolution block separates the feature map into two branches, S1 and S2. Branch S1 keeps the feature map and number of channels unchanged, and performs convolution on branch S2. Then, the feature map after convolution is fused with the feature map of branch S1. Finally, channel mixing is performed to exchange information between the channels of the feature map. The resulting feature map is the same size as the upper-layer feature map. The obtained feature map is then subjected to channel separation, branch convolution, channel concatenation and channel mixing operations again. S4: In the bottleneck layer of the network, the global relationship between each pixel in the image is modeled using a ViT-based context connector, and ViT is used to replace the high-channel convolutional module. At the bottleneck layer of the network, ViT is used to slice the feature map and calculate the global relationships in the feature map; S5: Improve the feature fusion scheme between low-level and high-level features in the skip connection part by using a context-aware fusion module, and relearn the features; During the skip connection stage, the feature maps obtained from the four-channel convolutional mixing blocks in the encoder and the upsampled feature maps are respectively input into the path containing the context-aware fusion module for feature enhancement. S6: Define the hyperparameters of the segmentation model, train it using the dataset processed in S1, and obtain the loss value of the loss function and the segmentation result; S7: Adjust network parameters based on the results, generate and save the trained lesion segmentation model, input the test set data into the trained lesion segmentation model, segment the lesions in the cell lung CT image, and output the segmentation results.

2. The cellular lung lesion segmentation method based on ViT and contextual feature fusion according to claim 1, characterized in that: The ViT-based context connector uses six Transformer coding blocks to extract global information from the image, where ViT comprises four parts: (1) Slice embedding: By converting the original two-dimensional image into one-dimensional sequence data, the size of the slice is set according to the output image of the encoder, the image is converted into multiple slices, and the dimension of the slice is calculated; (2) Position encoding: Mark the corresponding position information of each slice in order to recover the dimension of the image. The position information is consistent with the output of the slice embedding. Save the correct position information of each slice and calculate its dimension; (3) Multi-head attention mechanism in Transformer encoding block: After the image slicing and location encoding information establishment stage, the image slice information is input into the Transformer encoding block to learn the relationship between slices and between each pixel in each slice, and to perform context modeling of global information; the multi-head attention layer in this part receives slice and location encoding information, learns more relevant information in different subspaces by using multiple attention heads, initializes Q and K in each attention head using different weight matrices, performs Dropout operation on the global features learned by the multi-head attention layer, and uses a normalization layer to modify the feature dimension to adapt to the subsequent output, thereby accelerating the convergence speed of the model; (4) Multilayer perceptron layer in Transformer coding block: This part uses MLP block to learn the nonlinear relationship between features; residual connection and layer normalization operation are used in each sub-layer.

3. The cellular lung lesion segmentation method based on ViT and contextual feature fusion according to claim 1, characterized in that: The context-aware fusion module receives low-level features from the encoder and high-level features from the decoder. First, it uses GAP (Global Average Pooling) to generate feature maps with global spatial information. Then, it uses a multilayer perceptron with shared weights to model the context information in the low-level and high-level feature maps, generating vectors h and l, where h and l represent the weight vectors in the high-level and low-level feature maps, respectively. Second, using residual learning, it multiplies the weight vectors with the two feature maps to generate a redistributed feature map. This redistributed feature map is then concatenated along the channel dimension to generate a feature map containing global context information from both local and high-level stages. Third, it uses two 3x3 convolutions to achieve weighted feature fusion and uses residual connections to receive information from the high-level features.

4. The cellular lung lesion segmentation method based on ViT and contextual feature fusion according to claim 1, characterized in that: The encoder contains four-channel hybrid convolutional blocks for extracting high-level and low-level features of the image. Each channel hybrid convolutional block contains two convolutional layers, a batch normalization layer, and a corrected linear unit (ReLU). The feature map extracted by each channel hybrid convolutional block contains two paths. The first path connects to a max pooling layer to downsample the feature map and pass it to the next convolutional block; the other path enters a skip connection path.

5. The cellular lung lesion segmentation method based on ViT and contextual feature fusion according to claim 1, characterized in that: The decoder contains four channel-mixed convolutional blocks identical to those in the encoder. Each channel-mixed convolutional block contains two layers of convolutional computation, batch normalization, and ReLU activation function. After each channel-mixed convolutional block, the feature map is upsampled using deconvolution. The feature map after the transposed convolution operation is expanded to twice its size.

6. The cellular lung lesion segmentation method based on ViT and contextual feature fusion according to claim 1, characterized in that: The channel convolutional mixing block includes a channel separation module, a channel splicing module, and a channel mixing module. The channel separation module includes two branches, S1 and S2. Branch S1 keeps the feature map and the number of channels unchanged, and performs convolution on branch S2. The feature maps output by the two branches are processed by the channel splicing module and the channel mixing module. Each channel convolutional mixing block performs two channel separation, channel splicing, and channel mixing operations to output a feature map.

7. The cellular lung lesion segmentation method based on ViT and contextual feature fusion according to claim 1, characterized in that: The ViT-based context connector includes a slice embedding module, a position encoding module, and six Transformer encoding blocks. Each Transformer encoding block contains a multi-head attention mechanism module and a multilayer perceptron module. The slice embedding module and the position encoding module are located at the beginning of the ViT module. The slice embedding module converts the original two-dimensional image into a one-dimensional sequence of slices. The position encoding module assigns corresponding position information to each slice. The multi-head attention mechanism module is located in the first half of the Transformer encoding block and receives slice and position encoding information, including multiple attention heads, Dropout operation, and a normalization layer. The multilayer perceptron module is located in the second half of the Transformer encoding block and includes a multilayer perceptron layer and a normalization layer.

8. The cellular lung lesion segmentation method based on ViT and contextual feature fusion according to claim 1, characterized in that: The context-aware fusion module includes two parallel global average pooling layers, a multilayer perceptron layer, and a convolutional block. The two parallel global average pooling layers receive low-level features from the encoder and high-level features from the decoder, respectively, to generate feature maps with global spatial information. The multilayer perceptron uses shared weights to model the context information in the low-level and high-level feature maps to generate weight vectors. The weight vectors are multiplied by the two feature maps to generate redistributed feature maps, which are then concatenated along the channel dimension to generate feature maps containing global context information from both local and high-level stages. Weighted feature fusion is achieved using two 3*3 convolutions in the convolutional block.

Images