Liver tumor segmentation method based on SCAF-TransUNet

Abstract

The invention discloses a liver tumor segmentation method based on SCAF-TransUNet, and the method comprises the steps: firstly, segmenting a liver image into semantically related superpixel regions through SLIC superpixel preprocessing, enabling a model to be more accurately focused on a tumor and other key regions, reducing the calculation redundancy, and improving the efficiency and precision of feature extraction; secondly, a CECA module introduced in jump connection replaces a traditional full connection layer through one-dimensional convolution, the attention mechanism between channels is simplified, capture of the channel dependency relation is enhanced, and meanwhile the feature extraction precision is kept; finally, the FPN is added in the decoder, and the fusion capability of the multi-scale features is further enhanced. According to the FPN, feature maps with different resolutions are fused layer by layer, so that the model can effectively combine global semantic information and local detail features before up-sampling, and the processing capacity for complex liver boundaries and small tumors is improved. Finally, the improvements significantly improve the accuracy and robustness of the liver tumor segmentation task.

Description

Technical Field

[0001] The present invention relates to the technical field of liver tumor segmentation, and in particular to a liver tumor segmentation method based on SCAF-TransUNet. Background Art

[0002] With the development of the times, the algorithms for liver tumor segmentation are constantly being updated and improved. From the initial traditional segmentation method to the current segmentation method based on deep learning, the efficiency and accuracy are getting higher and higher. The traditional image segmentation method is mainly based on the similarity between pixels for classification, that is, dividing the pixels in the image into different areas to achieve the separation and extraction of the target area. For example, Soler et al. Fully automatic anatomical,pathological, andfunctional segmentation from CT scans for hepatic surgery The threshold method proposed in "Segmentation of medical images using adaptive region growing" is to select appropriate thresholds to divide the pixels in the image into different regions or categories according to the characteristics and requirements of the image. Pohle et al. proposed a region growing algorithm in "Segmentation of medical images using adaptive region growing". This method first selects seed pixels or sub-regions as the target position. If adjacent pixels or regions meet the similarity conditions, they are merged so that the region can grow gradually. The graph cut method proposed by Boykov et al. in "Interactive graph cuts for optimal boundary segmentation of objects in ND images" is to convert the image into an undirected graph, in which each pixel is regarded as a node and the edges between adjacent pixels are regarded as edges. Then, the graph cut method uses the maximum flow minimum cut algorithm to process the graph to achieve image segmentation. Compared with traditional algorithms, feature extraction methods based on deep learning have better performance because they can automatically learn features from images, do not need to manually select feature types, and can better extract deep features of feature segmentation images.

[0003] However, among the existing image segmentation technologies, although deep learning-based methods have made significant progress, there are still some limitations, resulting in unsatisfactory accuracy of segmentation results, which is mainly reflected in the fact that the edge segmentation of low-contrast tumor areas is not close enough; the existence of multi-scale tumors makes it difficult for the algorithm to fully capture the different sizes and morphologies of tumors during feature extraction, resulting in missed detections and false detections in the segmentation results, seriously affecting the overall segmentation performance.

[0004] Therefore, the prior art still needs to be improved and enhanced. Summary of the invention

[0005] In view of the above-mentioned deficiencies in the prior art, the object of the present invention is to provide a liver tumor segmentation method based on SCAF-TransUNet, aiming to solve the problem that the algorithms in the prior art are difficult to fully capture the situation when extracting features, resulting in missed detections and false detections in the segmentation results.

[0006] In order to achieve the above object, the present invention adopts the following technical solutions:

[0007] In a first aspect, a liver tumor segmentation method based on SCAF-TransUNet is characterized by comprising:

[0008] Step 1: Input liver tumor image and first use Simple Linear Iterative Clustering superpixel preprocessing algorithm, which divides the image into multiple superpixel regions with similar features, reduces redundant information and retains local details of the image to form a superpixel image;

[0009] Step 2: The super-pixel image obtained after super-pixel preprocessing is input to the encoder part, and the multi-scale features of the input image are extracted through the convolutional neural network CNN. The Transformer Layer of SCAF-TransUNet is used to divide the CNN feature map into small patches and use the self-attention mechanism to capture the long-distance dependencies in the image.

[0010] Step 3: In the decoder stage of each layer, the upsampled feature map of the decoder is combined with the underlying encoder feature map transmitted through the skip connection. The encoder feature map is processed by the channel-efficient convolutional attention module during the transmission process, and finally the feature map weighted by channel attention and spatial attention is output;

[0011] Step 4: In the last layer of the decoder, the feature map after upsampling and feature fusion is compressed to the target category number through 1x1 convolution. Then, through Softmax activation, the category probability distribution is generated for each pixel, and the final output is the segmentation result of each pixel to achieve liver tumor image segmentation.

[0012] Furthermore, the superpixel preprocessing of the acquired liver tumor CT image to be segmented includes:

[0013] Step 1.1, initialize the seed:

[0014] Suppose the image has N pixels and we want to divide the image into K superpixel regions, then the area of ​​each superpixel is N / K, and the distance between seed points is:

[0015]

[0016] Step 1.2, gradient adjustment seed point:

[0017] In the n×n neighborhood of each seed point, usually n=3, the gradient value of the pixel is calculated and the seed is moved to the pixel position with the smallest gradient;

[0018] Step 1.3, calculate the color distance d c :

[0019]

[0020] Among them, L, a, and b are the three channels of the Lab color space, p represents the current pixel, and c represents the superpixel seed point;

[0021] Step 1.4: Calculate the spatial distance d s :

[0022]

[0023] Among them, x and y represent pixel coordinates respectively;

[0024] Step 1.5, final distance metric D′:

[0025] Combining color distance and spatial distance, a weighted metric is used to measure the distance from each pixel to the seed. In order to balance the influence of color distance and spatial distance, a weight parameter m is required. This value is used to control the relative importance of color and space, and is usually set to 10:

[0026]

[0027] in, is the step size between seed points, used to normalize spatial distances;

[0028] Step 1.6, iterative optimization:

[0029] The seed point and superpixel distribution are optimized by iterating the previous step multiple times. In each iteration, the seed point is gradually adjusted to the center of the superpixel area it affects, and finally stabilizes to form a superpixel image.

[0030] Furthermore, in step one, the connectivity between superpixel regions needs to be enhanced. Since discontinuous superpixel regions may be generated during segmentation, the connectivity enhancement step is finally performed to ensure the coherence of the superpixel regions, remove isolated pixels, and optimize the discontinuous superpixel regions.

[0031] Furthermore, the inputting of the super-pixel image obtained through super-pixel preprocessing into the encoder part, extracting multi-scale features of the input image through a convolutional neural network CNN includes:

[0032] In the encoder part, a convolutional neural network (CNN) is used to extract feature maps of different resolutions. For TransUNet, through convolution and pooling operations, the resolution of the image is gradually reduced to generate feature maps of different levels. Assuming that the size of the input image is H×W, the resolution of the feature map is gradually reduced. During the convolution operation, the input image will be gradually reduced to different feature maps. The feature map reduction process is as follows:

[0033] C1: input image size is H×W;

[0034] C2: After a pooling operation, the image resolution is reduced to Forming a larger low-level feature map;

[0035] C3; pooling again, the image resolution is further reduced to

[0036] C4: Finally, the image resolution is reduced to At this time, the feature map reflects more local information;

[0037] The calculation formula of the convolutional layer operation is expressed as:

[0038] F i =σ(W i *I i +b i )

[0039] Among them, F i Represents the feature map generated after the convolution operation; W i Represents the weight matrix of the convolution kernel; I i Represents the input image or the feature map of the previous layer; b i represents the bias term; * represents the convolution operation; σ is the nonlinear activation function.

[0040] Furthermore, the Transformer Layer of SCAF-TransUNet divides the CNN feature map into small patches and uses a self-attention mechanism to capture long-distance dependencies in the image, including:

[0041] Step 2.1: Visualization and embedding

[0042] First, the feature map is divided into P×P small blocks and expanded into a one-dimensional vector; each small block is linearly mapped to a high-dimensional space and the position information is added:

[0043]

[0044] Where E is the linear embedding matrix; E pos Expression position coding;

[0045] Step 2.2: Multi-head self-attention mechanism

[0046] Through the self-attention mechanism, the similarity between each small block is calculated, and the formula is:

[0047]

[0048] Among them, Q, K, V are query, key and value vectors respectively. The similarity between each position is calculated and the final output is obtained by weighting;

[0049] Step 2.3: Multilayer Perceptron MLP and output update

[0050] After the self-attention mechanism, a multi-layer perceptron MLP is used for nonlinear changes, and finally the output of the Transformer layer is obtained:

[0051] Z l =MLP(LN(Z′ l ))+Z′ l

[0052] Among them, Z l Describe the final output of the current layer; +Z′ l represents the intermediate output after the multi-head self-attention mechanism; LN(Z′ l ) indicates that the intermediate output layer is normalized to make the data more stable; MLP indicates that nonlinear transformation is performed through multi-layer perceptron to improve the feature representation capability; +Z′ l Represents a residual connection, which is used to preserve input information and enhance gradient flow.

[0053] Furthermore, the step three includes:

[0054] The feature map passed by the encoder is processed by the Channel-Efficient Convolutional Attention module, and the output is the feature map after channel attention weighting:

[0055] Step 3.1: Input feature map F

[0056] Assume that the input feature map is F, H, W, and C are the height, width, and number of channels of the feature map respectively;

[0057] Step 3.2, global average pooling:

[0058] Perform global average pooling on the input feature map F to generate a global representation of each channel; obtain a one-dimensional vector v∈R C , where each element represents the global information of a channel; the formula is as follows:

[0059]

[0060] Step 3.3, one-dimensional convolution instead of fully connected layer:

[0061] Use 1D convolution instead of the fully connected layer to generate dependency weights between channels; the size of the convolution kernel determines the range of cross-channel dependency:

[0062] W = Conv1D(v);

[0063] Step 3.4, activation of channel weights:

[0064] Use the Sigmoid function to map the weight W to the range of [0,1]:

[0065] W′=σ(W)

[0066]

[0067] Step 3.5, feature map weighting:

[0068] Use the channel weight W′ to weight each channel of the input feature map to generate a weighted channel feature map:

[0069] F′=F c ·W′, c = 1, 2..., C.

[0070] Furthermore, the step three includes:

[0071] The channel-efficient convolutional attention module is used to process the feature map weighted by channel attention, and the output is the feature map weighted by spatial attention:

[0072] Step 3.6, input feature map F′: After channel attention weighting, feature map F′∈R H×W×C Passed into the spatial attention module;

[0073] Step 3.7, pooling operation: In the spatial attention module, global average pooling and global maximum pooling are applied to compress along the channel dimension to generate two spatial feature maps F avg ∈R H×W×C and F max ∈R H×W×C :

[0074]

[0075] F max =max(F′(i,j,c)),c=1,2,...,C;

[0076] Step 3.8, feature map concatenation: Concatenate the two pooled feature maps along the channel dimension to form a 2D feature map F concat ∈R H×W×2 ;

[0077] Step 3.9, convolution operation: perform a 3x3 convolution on the concatenated feature map to generate a spatial attention weight map M s ∈R H×W :

[0078] M s =σ(Conv3x3(F concat ))

[0079] Where σ is the Sigmoid activation function, which limits the attention weight to between [0,1];

[0080] Step 3.10, spatial attention weighting:

[0081] Use the spatial attention weight M s Spatial weighting of the previously weighted feature map F′:

[0082] F″(i,j,c)=F′(i,j,c)·M s (i, j), i=1, 2...., H, j=1, 2...., W;

[0083] Step 3.11, final output: the final output feature map F″(i, j, c)∈R H×W×C It is the feature map after channel attention and spatial attention weighting.

[0084] Furthermore, the step three also includes:

[0085] At each layer of the decoder, the upsampled feature map is fused with the feature map processed by the Channel-Efficient Convolutional Attention module passed through the jump connection. The fused feature map F fusion It is passed as input to the PFN module for further processing to improve the expression and optimization of features:

[0086] Feature extraction: In the encoder part, a convolutional neural network is used to extract feature maps of different resolutions. For SCAF-TransUNet, the resolution of the image is gradually reduced through convolution and pooling operations to generate feature maps of different levels. Assuming that the size of the input image is H×W, the resolution of the feature map is gradually reduced, and these multi-scale feature maps will be used as the input of the subsequent FPN. Each feature map C i They all contain spatial and semantic information at different levels;

[0087] Horizontal connection: At each resolution level, the feature map is compressed by 1x1 convolution to obtain the reduced-dimensional feature map P i ; The role of 1x1 convolution is to reduce the number of channels of the feature map, reduce the computational complexity, and retain the most important key information:

[0088] P 5 =Conv1x1(C 5 ),P 4 ×Conv1x1(C 4 ),P 3 =Conv1x1(C 3 ),P 2 =Conv1x1(C 2 );

[0089] Top-down feature fusion: FPN adopts a top-down feature fusion strategy; the highest layer feature map P 5 It has the smallest spatial resolution and the richest semantic information, while the feature maps of the lower layers have higher resolution and more spatial details. When fusion is performed, the low-resolution feature map is first upsampled to a higher resolution and then added element by element with the higher-resolution feature map:

[0090] P 4 =P 4 +Upsample(P 5 )

[0091] P 3 =P 3 +Upsample(P 4 )

[0092] P 2 =P 2 +Upsample(P 3 )

[0093] Through this top-down approach, the global semantic information of high-level feature maps is propagated to low-level feature maps step by step, while retaining the low-level detail information;

[0094] Upsampling operation: The upsampling operation enlarges the low-resolution feature map to a size that matches the high-resolution feature map through bilinear interpolation or deconvolution;

[0095] Convolution after fusion: After each upsampling and feature fusion, FPN further processes the fused feature map by applying a 3x3 convolution operation; this convolution operation helps smooth the feature map and reduce aliasing effects while maintaining the combination of global semantic information and local details:

[0096] Pi =Conv3x3(C i );

[0097] Combination of FPN and skip connection: In the improved SCAF-TransUNet, the skip connection part has introduced the Channel-Efficient Convolutional Attention module; the feature map generated by FPN will be combined with the skip connection feature map from the encoder:

[0098] P i =P i + SkipConnection (CECA(C i )).

[0099] Furthermore, in the SCAF-TransUNet decoder part in step 3, upsampling is performed by using the feature map fused by FPN. After each upsampling, the resolution of the feature map is gradually restored until the original resolution of the input image is reached.

[0100]

[0101] Among them, P i Represents the input feature map of the current layer, the upsampled feature map The chapter will be passed to the next level;

[0102] Finally, the resolution of the output image is restored to the size of the original image, and the segmentation result is:

[0103] Output= Upsample (P 2 ).

[0104] The technical solution adopted by the present invention has the following beneficial effects:

[0105] The network of the present invention is based on the improved scheme of TransUNet network, which combines SLIC superpixel preprocessing, CECA (Channel-Efficient Convolutional Attention) hybrid attention module and multi-scale feature fusion module FPN (Feature Pyramid Networks), aiming to improve the performance of liver tumor segmentation. First, SLIC superpixel preprocessing enables the model to focus on key areas such as tumors more accurately by segmenting liver images into semantically related superpixel areas, reducing computational redundancy and improving the efficiency and accuracy of feature extraction. Secondly, the CECA module introduced in the jump connection replaces the traditional fully connected layer with a one-dimensional convolution, simplifies the attention mechanism between channels, enhances the capture of channel dependencies, reduces computational overhead, and maintains the accuracy of feature extraction. This makes CECA show higher real-time performance and computational efficiency in liver tumor segmentation. Finally, FPN is added to the decoder to further enhance the fusion capability of multi-scale features. FPN fuses feature maps of different resolutions layer by layer, so that the model can effectively combine global semantic information and local detail features before upsampling, and improves the processing capability of complex liver boundaries and small tumors. Ultimately, these improvements significantly improve the accuracy and robustness of the liver tumor segmentation task. BRIEF DESCRIPTION OF THE DRAWINGS

[0106] Figure 1 A SCAF-TransUNet network structure diagram of an embodiment of the present invention;

[0107] Figure 2 It is a FPN (Feature Pyramid Network) module of an embodiment of the present invention;

[0108] Figure 3 A CECA (Channel-Efficient Convolutional Attention) module according to an embodiment of the present invention;

[0109] Figure 4 It is an ECA (Efficient Channel Attention) module of an embodiment of the present invention. DETAILED DESCRIPTION

[0110] In order to make the purpose, technical solution and effect of the present invention clearer and more specific, the present invention is further described in detail with reference to the accompanying drawings and examples. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention.

[0111] In the embodiment of the present invention, please refer to Figure 1, a liver tumor segmentation method based on SCAF-TransUNet includes the following steps:

[0112] Step 1: Input liver tumor image and first use Simple Linear Iterative Clustering superpixel preprocessing algorithm, which divides the image into multiple superpixel regions with similar features, reduces redundant information and retains local details of the image to form a superpixel image;

[0113] Step 2: The super-pixel image obtained after super-pixel preprocessing is input to the encoder part, and the multi-scale features of the input image are extracted through the convolutional neural network CNN. The Transformer Layer of SCAF-TransUNet is used to divide the CNN feature map into small patches and use the self-attention mechanism to capture the long-distance dependencies in the image.

[0114] Step 3: In the decoder stage of each layer, the upsampled feature map of the decoder is combined with the underlying encoder feature map transmitted through the skip connection. The encoder feature map is processed by the channel-efficient convolutional attention module during the transmission process, and finally the feature map weighted by channel attention and spatial attention is output;

[0115] Step 4: In the last layer of the decoder, the feature map after upsampling and feature fusion is compressed to the target category number through 1x1 convolution. Then, through Softmax activation, the category probability distribution is generated for each pixel, and the final output is the segmentation result of each pixel to achieve liver tumor image segmentation.

[0116] Step 1: Preprocessing

[0117] Innovation in the pre-input stage based on the SLIC superpixel algorithm:

[0118] The liver tumor image is input and the SLIC (Simple Linear Iterative Clustering) superpixel preprocessing algorithm is first used. This algorithm divides the image into multiple superpixel regions with similar features. By reducing redundant information and retaining local details of the image, subsequent feature acquisition is more efficient and accurate. The specific steps include:

[0119] Initialize the seed:

[0120] Assume that the image has N pixels and we want to segment the image into K superpixel regions, then the area of ​​each superpixel is N / K. The distance S (step length) between seed points is:

[0121]

[0122] Gradient adjustment seed point:

[0123] In the n×n neighborhood of each seed point (usually n=3), the gradient value of the pixel is calculated, and the seed is moved to the pixel position with the smallest gradient. This helps avoid the seed point being located at the boundary and improves the clustering effect.

[0124] Calculate the color distance d c :

[0125]

[0126] Among them, L, a, and b are the three channels of the Lab color space, p represents the current pixel, and c represents the superpixel seed point.

[0127] Calculate the spatial distance d s

[0128]

[0129] Among them, x and y represent pixel coordinates respectively.

[0130] The final distance metric D′ is:

[0131] Combining color distance and spatial distance, a weighted metric is used to measure the distance from each pixel to the seed. In order to balance the influence of color distance and spatial distance, a weight parameter m is required. This value is used to control the relative importance of color and space, and is usually set to 10:

[0132]

[0133] Among them, d c and d s Represent color distance and spatial distance respectively. is the step size between seed points, used to normalize spatial distances.

[0134] Iterative Optimization:

[0135] The distribution of seed points and superpixels is optimized by iterating the process of the previous step multiple times. Usually 10 iterations are sufficient. In each iteration, the seed point is gradually adjusted to the center of the superpixel area it affects, and finally stabilizes.

[0136] Enhanced Connectivity:

[0137] Since discontinuous superpixel regions may be generated during segmentation, a connectivity enhancement step is finally performed to ensure the coherence of the superpixel regions, remove isolated pixels, and optimize the discontinuous superpixel regions.

[0138] Output processed superpixel image:

[0139] The output image will be segmented into multiple superpixel regions. These superpixel regions can be used as input data for subsequent SCAF-TransUNet model processing and further used for detailed segmentation tasks.

[0140] The main purpose of SLIC (Simple Linear Iterative Clustering) superpixel preprocessing is to segment the input image into multiple compact regions (superpixels) with semantic associations to improve the processing efficiency and accuracy of the model. Specifically, SLIC clusters adjacent similar pixels into superpixel regions, retains local features in the image, and enables the model to better focus on important image regions. This method is particularly effective in dealing with medical image segmentation tasks because it can significantly reduce the redundant information of the image, avoid the model from wasting computing resources on irrelevant regions, and thus improve segmentation accuracy. In the SLIC workflow, superpixel seed points are first initialized through a regular grid, and then the surrounding pixels are clustered according to the spatial distance and color similarity of the pixels, gradually forming semantically related superpixel regions. Next, the SLIC algorithm continuously adjusts the boundaries of each superpixel region through iterative optimization to ensure that the color and spatial differences between adjacent pixels are minimized, generating compact superpixel segmentation results. At the same time, SLIC also pays special attention to the retention of boundary information, so that the segmented superpixels can not only reflect the real boundary features in the image, but also have good regional compactness. Through this preprocessing, the model simplifies the image representation at the input stage, making subsequent feature extraction more efficient. This feature of SLIC is also particularly helpful in enhancing the model's ability to handle boundaries and local details, avoiding boundary blur or over-smoothing problems. Ultimately, SLIC superpixel preprocessing provides the model with a more streamlined and semantically clear input image, greatly improving the efficiency and accuracy of the segmentation task.

[0141] Step 2: Encoder Stage

[0142] CNN convolution feature extraction:

[0143] For the image input obtained after SLIC superpixel preprocessing, the encoder part first uses a convolutional neural network (CNN) to extract the multi-scale features of the input image. The specific operations are as follows:

[0144] Convolutional layer and pooling operation:

[0145] The image first passes through the convolution layer and the pooling layer to extract low-level to high-level multi-scale feature maps. The pooling layer is responsible for reducing the spatial dimension of the image, while the convolution layer extracts local features through filters. Typically, the pooling layer uses maximum pooling, and the convolution layer applies a 3×3 convolution kernel.

[0146] Multi-scale feature maps:

[0147] During the convolution operation, the input image is gradually reduced to different feature maps. The feature map reduction process is as follows:

[0148] C1: The input image size is H×W.

[0149] C2: After a pooling operation, the image resolution is reduced to Form a larger low-level feature map.

[0150] C3; pooling again, the image resolution is further reduced to

[0151] C4: Finally, the image resolution is reduced to At this time, the feature map reflects more local information.

[0152] The calculation formula of the convolutional layer operation is expressed as:

[0153] F i =σ(W i *I i +b i )

[0154] Among them, F i Represents the feature map generated after the convolution operation. i Represents the weight matrix of the convolution kernel. i Represents the input image or the feature map of the previous layer. b i Represents the bias term. * represents the convolution operation. σ is the nonlinear activation function.

[0155] The Transformer Layer of SCAF-TransUNet divides the CNN feature map into small patches and uses the self-attention mechanism to capture long-distance dependencies in the image. The calculation steps of this part are as follows:

[0156] Patch Enbedding

[0157] First, the feature map is divided into P×P blocks and expanded into a one-dimensional vector. Each block is linearly mapped to a high-dimensional space and its position information is added:

[0158]

[0159] Where E is the linear embedding matrix. pos Describes the positional encoding.

[0160] Multi-Head Self-Attention (MSA)

[0161] Through the self-attention mechanism, the similarity between each small block is calculated, and the formula is:

[0162]

[0163] Among them, Q, K, and V are query, key, and value vectors respectively. The similarity between each position is calculated and weighted to obtain the final output.

[0164] Multilayer Perceptron (MLP) and output update:

[0165] After the self-attention mechanism, a multi-layer perceptron (MLP) is used for nonlinear changes, and finally the output of the Transformer layer is obtained:

[0166] Z l =MLP(LN(Z′ l ))+Z′ l

[0167] Among them, Z l Describes the final output of the current layer. Z′ l Represents the intermediate output after the multi-head self-attention mechanism. LN(Z′ l ) indicates that the intermediate output layer is normalized to make the data more stable. MLP indicates that nonlinear transformation is performed through multi-layer perceptron to improve the feature representation capability. +Z′ l Represents a residual connection, which is used to preserve input information and enhance gradient flow.

[0168] Step 3: Decoder stage

[0169] In the embodiment of the present invention, please refer to Figure 3 ,CECA module is introduced through skip connection to enhance feature expression;

[0170] In the decoder stage of each layer, the upsampled feature map of the decoder is combined with the underlying encoder feature map passed through the skip connection. The encoder feature map is processed by the CECA module (Channel Efficient Convolutional Attention Module) during the transmission process to enhance the important information in the channel and spatial dimensions.

[0171] F fusion =Concat(CECA(F encoder ), F decoder ) Among them, F encoder It is the feature map transmitted from the encoder part, which becomes CECA (F encoder ). decoder is the upsampled feature map of the current layer of the decoder. fusionIt represents the result of concatenating the feature map processed by the CECA module and the upsampled feature map of the current layer of the decoder.

[0172] CECA (Channel-Efficient Convolutional Attention) module:

[0173] Input feature map F:

[0174] Assume that the input feature map is F, H, W, and C are the height, width, and number of channels of the feature map, respectively.

[0175] Global Average Pooling:

[0176] Perform global average pooling on the input feature map F to generate a global representation of each channel. Get a one-dimensional vector v∈R C , where each element represents the global information of a channel. The formula is as follows:

[0177]

[0178] One-dimensional convolution instead of fully connected layer:

[0179] Use 1D convolution instead of the fully connected layer to generate dependency weights between channels. The size of the convolution kernel determines the range of cross-channel dependency:

[0180] W = Conv1D(v).

[0181] Activation Function of channel weight:

[0182] Use the Sigmoid function to map the weight W to the range of [0,1]:

[0183] W′=σ(W);

[0184]

[0185] Feature map weighting (Channel Reweighting):

[0186] Use the channel weight W′ to weight each channel of the input feature map to generate a weighted channel feature map:

[0187] F′=F c ·W′,c=1,2,...,C.

[0188] Spatial Attention Module:

[0189] 1) Input feature map F′: After channel attention weighting, feature map F′∈R H×W×C is passed into the spatial attention module.

[0190] 2) Pooling Operations: In the spatial attention module, global average pooling and global maximum pooling are applied to compress along the channel dimension to generate two spatial feature maps F avg ∈R H×W×C and F max ∈R H×W×C :

[0191]

[0192] F max =max(F′(i,j,c)),c=1,2....,C.

[0193] 3) Feature map concatenation: Concatenate the two pooled feature maps along the channel dimension to form a 2D feature map F concat ∈R H×W×2 .

[0194] 4) Convolution operation: Perform a 3x3 convolution on the concatenated feature map to generate a spatial attention weight map M s ∈R H ×W :

[0195] M s =σ(Conv3x3(F concat ))

[0196] Where σ is the Sigmoid activation function, which limits the attention weights to between [0, 1].

[0197] Spatial Reweighting:

[0198] Use the spatial attention weight M s Spatial weighting of the previously weighted feature map F′:

[0199] F″(i,j,c)=F′(i,j,c)·M s (i, j), i = 1, 2...., H, j = 1, 2...., W

[0200] Final output:

[0201] The final output feature map F″(i, j, c)∈R H×W×C It is the feature map after channel attention and spatial attention weighting.

[0202] Among them, the Channel-Efficient Convolutional Attention module is an improvement on the channel attention mechanism in CBAM (Convolutional Block Attention Module), replacing the channel attention part with a lighter ECA (Efficient Channel Attention) module. The traditional CBAM channel attention module extracts channel features through global average pooling and global maximum pooling, and generates attention weights for each channel through two fully connected layers. Although this method can accurately capture the dependencies between channels, its computational complexity is high, especially when processing high-resolution images, the computational overhead increases significantly.

[0203] In this embodiment, if Figure 4 As shown in the figure, the ECA module replaces the fully connected layer with a one-dimensional convolution, simplifying the design of the channel attention mechanism. It no longer requires channel compression and expansion operations, but directly captures cross-channel dependencies through 1D convolution. This improvement greatly reduces the computational cost, especially in high-resolution scenarios. ECA retains the precise feature extraction capability of the CBAM module through efficient convolution operations, while achieving the same or even better performance with fewer computing resources. This lightweight design is particularly suitable for tasks that require high real-time performance and computational efficiency. In medical image segmentation, CECA not only improves the ability to capture important features, but also speeds up the response speed of the model, ensuring the efficiency and accuracy of the segmentation task. By incorporating the CECA module into the jump connection, the model further enhances the information interaction between channels, optimizes the transmission between feature layers, and ultimately significantly improves the accuracy and robustness of medical image segmentation.

[0204] In this example, see Figure 2 , feature fusion and information transmission optimization are achieved by introducing the FPN module:

[0205] At each layer of the decoder, the upsampled feature map is fused with the encoder feature map (processed by the CECA module) passed through the jump connection. The fused feature map F fusion It is passed as input to the PFN module for further processing to improve the expression and optimization of features. The following is the detailed workflow of the PFN module.

[0206] Feature extraction (Encoder part)

[0207] In the encoder part, a convolutional neural network (CNN) is used to extract feature maps of different resolutions. For SCAF-TransUNet, through convolution and pooling operations, the resolution of the image is gradually reduced to generate feature maps of different levels. Assuming that the size of the input image is H×W, the resolution of the feature map is gradually reduced, for example, C 2 , C 3 , C 4 , C 5 , which represent different resolutions:

[0208] C 2 =(H / 2, W / 2), C 3 =(H / 4, W / 4), C 4 =(H / 8, W / 8), C 5 =(H / 16, W / 16);

[0209] These multi-scale feature maps will be used as the input of the subsequent FPN. Each feature map C i Both contain different levels of spatial and semantic information:

[0210] Lateral Connections:

[0211] At each resolution level, the feature map is compressed by 1x1 convolution to obtain the reduced-dimensional feature map P i The role of 1x1 convolution is to reduce the number of channels of the feature map, reduce the computational complexity, and retain the most important key information:

[0212] P 5 =Conv1x1(C 5 ),P 4 =Conv1x1(C 4 ),P 3 =Conv1x1(C 3 ),P 2 =Conv1x1(C 2 ).

[0213] Top-Down Pathway:

[0214] FPN adopts a top-down feature fusion strategy. The feature map P at the highest level 5 The low-resolution feature map has the smallest spatial resolution and the richest semantic information, while the low-level feature map has a higher resolution and more spatial details. When fusion is performed, the low-resolution feature map is first upsampled to a higher resolution and then added element by element with the higher-resolution feature map:

[0215] P 4 =P4 +Upsample(P 5 );

[0216] P 3 =P 3 +Upsample(P 4 );

[0217] P 2 =P 2 +Upsample(P 3 ).

[0218] In this top-down manner, the global semantic information of high-level feature maps is propagated to low-level feature maps step by step, while retaining the low-level detail information.

[0219] Upsampling operation:

[0220] The upsampling operation enlarges the low-resolution feature map to a size that matches the high-resolution feature map through bilinear interpolation or deconvolution. 5 Upsample to P 4 Resolution:

[0221]

[0222] After upsampling, the size of the feature maps is consistent, so element-by-element addition can be directly calculated.

[0223] Convolution after fusion:

[0224] After each upsampling and feature fusion, FPN further processes the fused feature map by applying a 3x3 convolution operation. This convolution operation helps smooth the feature map and reduce aliasing effects while maintaining the combination of global semantic information and local details:

[0225] P i =Conv3x3(C i ).

[0226] Combination of FPN and Skip Connections:

[0227] In the improved SCAF-TransUNet, the CECA module has been introduced in the jump connection part. The feature map generated by FPN will be combined with the jump connection feature map from the encoder (through feature cascading). These jump connection feature maps are processed by CECA's channel and spatial attention mechanism, which can effectively improve the focus on important features:

[0228] P i =P i + SkipConnection (CECA(Ci )).

[0229] Upsampling operation:

[0230] In the SCAF-TransUNet decoder part, upsampling is performed by using the feature map fused by FPN. After each upsampling, the resolution of the feature map is gradually restored until the original resolution of the input image is reached:

[0231]

[0232] Among them, P i Represents the input feature map of the current layer, the upsampled feature map The chapter will be passed to the next level.

[0233] Finally, the resolution of the output image is restored to the size of the original image, and the segmentation result is:

[0234] Output= Upsample (P 2 ).

[0235] Among them, the introduction of FPN (Feature Pyramid Network) in the decoding stage of SCAF-TransUNet and its placement before upsampling can significantly improve the fusion capability of multi-scale features, thereby enhancing the accuracy of medical image segmentation. FPN fuses feature maps of different resolutions (such as 1 / 8, 1 / 4, 1 / 2, etc.) layer by layer, and integrates these feature maps into feature maps containing global semantic information and local details before upsampling. This multi-scale feature fusion can better capture liver tumors of different sizes and shapes, and is especially helpful in dealing with complex boundaries and small lesions. Upsampling after completing multi-scale feature fusion can ensure that the feature map already has rich details and global information before upsampling, making the upsampling process more robust and reducing the risk of blurred boundaries or lost details. Through such improvements, the model can significantly improve segmentation accuracy and boundary processing capabilities when processing complex medical images.

[0236] Step 4: Final Output

[0237] In the last layer of the decoder, the feature map after upsampling and feature fusion is compressed to the number of target categories through 1x1 convolution. Then, through Softmax activation, the category probability distribution is generated for each pixel, and the final output is the segmentation result of each pixel.

[0238] The network of the present invention is based on the improved scheme of TransUNet network, which combines SLIC superpixel preprocessing, CECA (Channel-Efficient Convolutional Attention) hybrid attention module and multi-scale feature fusion module FPN (Feature Pyramid Networks), aiming to improve the performance of liver tumor segmentation. First, SLIC superpixel preprocessing enables the model to focus on key areas such as tumors more accurately by segmenting liver images into semantically related superpixel areas, reducing computational redundancy and improving the efficiency and accuracy of feature extraction. Secondly, the CECA module introduced in the jump connection replaces the traditional fully connected layer with a one-dimensional convolution, simplifies the attention mechanism between channels, enhances the capture of channel dependencies, reduces computational overhead, and maintains the accuracy of feature extraction. This makes CECA show higher real-time performance and computational efficiency in liver tumor segmentation. Finally, FPN is added to the decoder to further enhance the fusion capability of multi-scale features. FPN fuses feature maps of different resolutions layer by layer, so that the model can effectively combine global semantic information and local detail features before upsampling, and improves the processing capability of complex liver boundaries and small tumors. Ultimately, these improvements significantly improve the accuracy and robustness of the liver tumor segmentation task.

[0239] Those skilled in the art will readily appreciate other embodiments of the present invention after considering the specification and practicing the schemes disclosed herein. The present invention is intended to cover any variations, uses or adaptations of the present invention that follow the general principles of the present invention and include common knowledge or customary techniques in the art that are not disclosed in this disclosure. The description and examples are to be considered exemplary only, and the true scope and spirit of the present invention are indicated by the claims.

Claims

1. A liver tumor segmentation method based on SCAF-TransUNet, characterized in that: include: Step 1: Input the liver tumor image and first use the superpixel preprocessing algorithm, which divides the image into multiple superpixel regions with similar features, reduces redundant information and retains local details of the image to form a superpixel image; Step 2: For the super-pixel image obtained after super-pixel preprocessing, the encoder part uses the convolutional neural network CNN to extract the multi-scale features of the input image, and divides the CNN feature map into small patches through the Transformer Layer of SCAF-TransUNet, and uses the self-attention mechanism to capture the long-distance dependencies in the image; Step 3: In the decoder stage of each layer, the upsampled feature map of the decoder is combined with the underlying encoder feature map transmitted through the skip connection. The encoder feature map is processed by the channel-efficient convolutional attention module during the transmission process, and finally the feature map weighted by channel attention and spatial attention is output; Step 4: In the last layer of the decoder, the feature map after upsampling and feature fusion is compressed to the target category number through 1x1 convolution. Then, through Softmax activation, the category probability distribution is generated for each pixel, and the final output is the segmentation result of each pixel to achieve liver tumor image segmentation.

2. The liver tumor segmentation method based on SCAF-TransUNet according to claim 1, characterized in that: The superpixel preprocessing of the acquired liver tumor CT image to be segmented includes: Step 1.1, initialize the seed: Suppose the image has N pixels and we want to divide the image into K superpixel regions, then the area of ​​each superpixel is N / K, and the distance between seed points is: Step 1.2, gradient adjustment seed point: In the n×n neighborhood of each seed point, usually n=3, the gradient value of the pixel is calculated and the seed is moved to the pixel position with the smallest gradient; Step 1.3, calculate the color distance d c : Among them, L, a, and b are the three channels of the Lab color space, p represents the current pixel, and c represents the superpixel seed point; Step 1.4: Calculate the spatial distance d s : Among them, x and y represent pixel coordinates respectively; Step 1.5, final distance metric D′: Combining color distance and spatial distance, a weighted metric is used to measure the distance from each pixel to the seed. In order to balance the influence of color distance and spatial distance, a weight parameter m is required. This value is used to control the relative importance of color and space, and is usually set to 10: in, is the step size between seed points, used to normalize spatial distances; Step 1.6, iterative optimization: The seed point and superpixel distribution are optimized by iterating the previous step multiple times. In each iteration, the seed point is gradually adjusted to the center of the superpixel area it affects, and finally stabilizes to form a superpixel image.

3. The liver tumor segmentation method based on SCAF-TransUNet according to claim 1, characterized in that: In the step 1, the connectivity between super-pixel regions needs to be enhanced. Since discontinuous super-pixel regions may be generated during segmentation, the connectivity enhancement step is finally performed to ensure the coherence of the super-pixel regions, remove isolated pixels, and optimize the discontinuous super-pixel regions.

4. The liver tumor segmentation method based on SCAF-TransUNet according to claim 1, characterized in that: The step of inputting the super-pixel image obtained through super-pixel preprocessing into the encoder part, extracting multi-scale features of the input image through a convolutional neural network (CNN) includes: In the encoder part, a convolutional neural network (CNN) is used to extract feature maps of different resolutions. For TransUNet, through convolution and pooling operations, the resolution of the image is gradually reduced to generate feature maps of different levels. Assuming that the size of the input image is H×W, the resolution of the feature map is gradually reduced. During the convolution operation, the input image will be gradually reduced to different feature maps. The feature map reduction process is as follows: C1: The input image size is H×W, where H represents the height of the input image, that is, the number of pixels in the vertical direction, and W represents the width of the input image, that is, the number of pixels in the horizontal direction; C2: After a pooling operation, the image resolution is reduced to Forming a larger low-level feature map; C3; pooling again, the image resolution is further reduced to C4: Finally, the image resolution is reduced to At this time, the feature map reflects more local information; The calculation formula of the convolutional layer operation is expressed as: F i =σ(W i *I i +b i ) Among them, F i Represents the feature map generated after the convolution operation; W i Represents the weight matrix of the convolution kernel; I i Represents the input image or the feature map of the previous layer; b i represents the bias term; * represents the convolution operation; σ is the nonlinear activation function.

5. The liver tumor segmentation method based on SCAF-TransUNet according to claim 1, characterized in that: The Transformer Layer of SCAF-TransUNet divides the CNN feature map into small patches and uses the self-attention mechanism to capture long-distance dependencies in the image, including: Step 2.1, visualization and embedding: First, the feature map is divided into P×P small blocks and expanded into a one-dimensional vector; each small block is linearly mapped to a high-dimensional space and the position information is added: Where E is the linear embedding matrix; E pos Expression position coding; Step 2.2, multi-head self-attention mechanism: Through the self-attention mechanism, the similarity between each small block is calculated, and the formula is: Among them, Q, K, V are query, key and value vectors respectively. The similarity between each position is calculated and the final output is obtained by weighting; K T represents the transpose of the key matrix, d k Represents the dimension of the key vector; Step 2.3, Multilayer Perceptron MLP and output update: After the self-attention mechanism, a multi-layer perceptron MLP is used for nonlinear changes, and finally the output of the Transformer layer is obtained: WITH l =MLP(LN(Z′ l ))+Z′ l Among them, Z l Describe the final output of the current layer; Z′ l represents the intermediate output after the multi-head self-attention mechanism; LN(Z′ l ) indicates that the intermediate output layer is normalized to make the data more stable; MLP indicates that nonlinear transformation is performed through multi-layer perceptron to improve the feature representation capability; +Z′ l Represents a residual connection, which is used to preserve input information and enhance gradient flow.

6. The liver tumor segmentation method based on SCAF-TransUNet according to claim 1, characterized in that: The step three includes: The feature map passed by the encoder is processed by the Channel-Efficient Convolutional Attention module, and the output is the feature map after channel attention weighting: Step 3.1, input feature map F: Assume that the input feature map is F, H, W, and C are the height, width, and number of channels of the feature map respectively; Step 3.2, global average pooling: Perform global average pooling on the input feature map F to generate a global representation of each channel; One-dimensional vector v∈R C , where each element represents the global information of a channel; the formula is as follows: Step 3.3, one-dimensional convolution instead of fully connected layer: Use 1D convolution instead of the fully connected layer to generate dependency weights between channels; the size of the convolution kernel determines the range of cross-channel dependency: W = Conv1D(v); Step 3.4, activation of channel weights: Use the Sigmoid function to map the weight W to the range of [0,1]: W′=σ(W) Step 3.5, feature map weighting: Use the channel weight W′ to weight each channel of the input feature map to generate a weighted channel feature map: F′=F C ·W′,c=1,2,....,C。 7. The liver tumor segmentation method based on SCAF-TransUNet according to claim 6, characterized in that: The step three includes: The channel-efficient convolutional attention module is used to process the feature map weighted by channel attention, and the output is the feature map weighted by spatial attention: Step 3.6, input feature map F′: After channel attention weighting, feature map F′∈R H×W×C Passed into the spatial attention module; Step 3.7, pooling operation: In the spatial attention module, global average pooling and global maximum pooling are applied to compress along the channel dimension to generate two spatial feature maps F avg ∈R H×W×C and F max ∈R H×W×C : F max =max(F′(i,j,c)),c=1,2....,C; Step 3.8, feature map concatenation: Concatenate the two pooled feature maps along the channel dimension to form a 2D feature map F concat ∈R H×W×2 ; Step 3.9, convolution operation: perform a 3x3 convolution on the concatenated feature map to generate a spatial attention weight map M s ∈R H×W : M s =σ(Conv3x3(F concat )) Where σ is the Sigmoid activation function, which limits the attention weight to between [0,1]; Step 3.10, spatial attention weighting: Use the spatial attention weight M s Spatial weighting of the previously weighted feature map F′: F″(i,j,c)=F′(i,j,c)·M s (i,j),i=1,2....,H,j=1,2....,W; Step 3.11, final output: the final output feature map F″(i, j, c)∈R H×W×C It is the feature map after channel attention and spatial attention weighting.

8. The liver tumor segmentation method based on SCAF-TransUNet according to claim 1, characterized in that: The step three also includes: At each layer of the decoder, the upsampled feature map is fused with the feature map processed by the Channel-Efficient Convolutional Attention module passed through the jump connection. The fused feature map F fusion It is passed as input to the PFN module for further processing to improve the expression and optimization of features: Feature extraction: In the encoder part, a convolutional neural network is used to extract feature maps of different resolutions. For SCAF-TransUNet, the resolution of the image is gradually reduced through convolution and pooling operations to generate feature maps of different levels. Assuming that the size of the input image is H×W, the resolution of the feature map is gradually reduced, and these multi-scale feature maps will be used as the input of the subsequent FPN. Each feature map C i They all contain spatial and semantic information at different levels; Horizontal connection: At each resolution level, the feature map is compressed by 1x1 convolution to obtain the reduced-dimensional feature map P i ; The role of 1x1 convolution is to reduce the number of channels of the feature map, reduce the computational complexity, and retain the most important key information: P5=Conv1x1(C5), P4=Conv1x1(C4), P3=Conv1x1(C3), P2=Conv1x1(C2); Top-down feature fusion: FPN adopts a top-down feature fusion strategy; the highest-level feature map P5 has the smallest spatial resolution and the richest semantic information, while the lower-level feature maps have higher resolution and more spatial details; when fusion is performed, the low-resolution feature map is first upsampled to a higher resolution, and then added element-by-element to the higher-resolution feature map: P4=P4+Upsample(P5) P3=P3+Upsample(P4) P2=P2+Upsample(P3) Through this top-down approach, the global semantic information of high-level feature maps is propagated to low-level feature maps step by step, while retaining the low-level detail information; Upsampling operation: The upsampling operation enlarges the low-resolution feature map to a size that matches the high-resolution feature map through bilinear interpolation or deconvolution; Convolution after fusion: After each upsampling and feature fusion, FPN further processes the fused feature map by applying a 3x3 convolution operation; this convolution operation helps smooth the feature map and reduce aliasing effects while maintaining the combination of global semantic information and local details: P i =Conv3x3(C i ); Combination of FPN and skip connection: In the improved SCAF-TransUNet, the skip connection part has introduced the Channel-Efficient Convolutional Attention module; the feature map generated by FPN will be combined with the skip connection feature map from the encoder: P i =P i + SkipConnection (CECA(C i ))。 9. The liver tumor segmentation method based on SCAF-TransUNet according to claim 8, characterized in that: In the SCAF-TransUNet decoder in step 3, upsampling is performed by using the feature map fused with FPN. After each upsampling, the resolution of the feature map is gradually restored until the original resolution of the input image is reached. Among them, P i Represents the input feature map of the current layer, the upsampled feature map The chapter will be passed to the next level; Finally, the resolution of the output image is restored to the size of the original image, and the segmentation result is: Output= Upsample (P2)。

Images