A liver cancer image lightweight segmentation method and system based on a multi-scale frequency-space fusion UNet

Abstract

The application discloses a liver cancer image lightweight segmentation method and system based on a multi-scale frequency-space fusion UNet, and belongs to the technical field of medical image processing and computer vision. The method acquires a liver contrast-enhanced CT image, performs pretreatment on the liver contrast-enhanced CT image, and inputs the liver contrast-enhanced CT image into an MSFA-UNet network; in an encoding stage, a GDRC module is used to establish directional context features, and a DCFD module is used for frequency-space joint down-sampling; deep layer semantics are strengthened at a bottleneck layer; in a decoding stage, a HAFU module is used for semantic gating and directional attention fusion on deep layer decoding features and encoding end jump features; finally, 1*1 convolution, Sigmoid mapping and threshold segmentation are performed to generate a tumor probability graph and a tumor segmentation mask. The application can improve the complex liver cancer lesion boundary recognition ability under lower parameter quantity and computation quantity, and is suitable for tumor region auxiliary delineation, preoperative planning and image omics analysis.

Description

Technical Field

[0001] This invention relates to the fields of medical image processing and computer vision technology, specifically to a lightweight automatic segmentation method and system for liver cancer images in contrast-enhanced CT images, and more particularly to a lightweight segmentation method and system for liver cancer images based on multi-scale frequency-space fusion UNet. This invention can be used for automatic assisted delineation, preoperative planning, radiomics analysis, and clinical auxiliary assessment of liver cancer lesions such as hepatocellular carcinoma (HCC) and intrahepatic cholangiocarcinoma (ICC). Background Technology

[0002] Contrast-enhanced computed tomography (CT) is an important imaging modality for the clinical management and non-invasive assessment of liver cancer, capable of revealing the spatial relationships between the liver, blood vessels, tumor, and surrounding tissues. Accurate automated segmentation of liver tumors provides the foundational regions of interest for minimally invasive surgery, tumor resection, radiotherapy target planning, radiomics analysis, and AI-assisted diagnosis.

[0003] In recent years, deep learning-based medical image segmentation methods have gradually replaced traditional manual delineation. Convolutional neural networks, represented by U-Net and its variants, fuse high-level semantic information with low-level spatial details through encoder-decoder structures and skip connections; models represented by Transformer capture long-distance dependencies through self-attention mechanisms; some studies have also attempted to introduce frequency domain mechanisms such as wavelet transform and discrete cosine transform to assist in spatial feature learning.

[0004] However, existing automatic segmentation methods for liver cancer still have the following problems in clinical deployment: First, tumor boundaries are complex and often adjacent to major blood vessels or the liver capsule, making existing networks susceptible to interference from vascular enhancement signals and cirrhotic background textures; Second, traditional convolutional networks have limited local receptive fields, and the downsampling stage usually uses max pooling or direct convolution, which easily leads to the loss of high-frequency details of small lesions and tumor edges; Third, some high-precision segmentation models have a large number of parameters and floating-point operations, making them difficult to deploy quickly in ordinary medical terminals or edge computing devices.

[0005] Therefore, there is a need for a lightweight liver cancer image segmentation technique that can preserve high-frequency boundary information in liver cancer CT images, establish directional context constraints in the spatial domain, and take into account both the number of parameters and the computational cost. Summary of the Invention

[0006] The present invention aims to solve the problems of blurred boundaries, easy loss of high-frequency details of small lesions, easy missegmentation of the tumor and blood vessel adjacent areas, and high model parameter quantity and computational complexity in the existing automatic segmentation of liver cancer CT images.

[0007] To address the aforementioned technical problems, this invention provides a lightweight segmentation method for liver cancer images based on a multi-scale frequency-space fusion UNet, comprising: acquiring contrast-enhanced CT images of the liver and performing preprocessing; inputting the preprocessed images into an MSFA-UNet network; establishing directional contextual features through a Global-Oriented Diffused Residual Convolution (GDRC) module during the encoding stage, and performing frequency-space joint downsampling through a Decomposition Context Frequency Downsampling (DCFD) module; further enhancing deep semantic features through a GDRC module at the bottleneck layer; fusing deep decoding features with encoding-end skip features through a Hybrid Attention Feature Upsampling (HAFU) module using semantic gating and directional attention during the decoding stage; and finally obtaining a tumor probability map and a tumor segmentation mask through 1×1 convolution, sigmoid mapping, and threshold segmentation.

[0008] This invention also provides a lightweight segmentation system for liver cancer images based on a multi-scale frequency-space fusion UNet, comprising an image preprocessing module, an encoded feature extraction module, a bottleneck semantic enhancement module, a decoded feature fusion module, and a segmentation output module. Each of these modules respectively performs the image input, feature encoding, semantic enhancement, attention decoding, and mask output processes in the segmentation method.

[0009] The implementation method of this system includes the following steps:

[0010] S1, acquire contrast-enhanced CT images of the liver, and preprocess the CT images to obtain network input images;

[0011] S2, the network input image is input into the encoder of the MSFA-UNet network. In each encoding level, the directional context features are first extracted by the Global-Oriented Dilated Residual Convolution (GDRC) module, and then downsampled by the Decomposed Context Frequency Downsampling (DCFD) module. The encoding features output by the corresponding encoding level are saved as skip connection features.

[0012] The GDRC module performs horizontal convolution, spatial dilated convolution, global projection, and vertical convolution on the input features. After channel concatenation and 1×1 convolution fusion of the outputs of each branch, a residual connection is established with the input features to obtain features with local multi-scale information, global semantic information, and horizontal and vertical contextual information.

[0013] The DCFD module first decomposes the input features into low-frequency and high-frequency information through the Hybrid Wavelet-Pooling Decomposition Module (HWPD), and then fuses the low-frequency and high-frequency paths. The fused frequency domain downsampled features are then input into the GDRC module for context enhancement to obtain frequency-space downsampled features with directional constraints.

[0014] S3, input the encoded features output from the deepest layer of the encoder into the bottleneck layer GDRC module to obtain deep semantic enhancement features;

[0015] S4. Input the deep semantic enhancement features into the decoder. In each decoding layer, the deep decoding features output by the previous decoding layer and the skip connection features saved in the corresponding encoding layer are received through the Hybrid Attention Feature Upsampling Module (HAFU). The skip connection features are semantically gated and filtered using the deep decoding features. The fused features are then calibrated with channel and directional attention to obtain the output features of the current decoding layer.

[0016] S5. Perform 1×1 convolution and Sigmoid mapping on the output features of the final decoding layer to obtain a tumor probability map, and binarize the tumor probability map according to a preset threshold to obtain a liver cancer tumor segmentation mask.

[0017] Furthermore, the GDRC module includes a horizontal convolution branch, a spatially dilated convolution branch, a global projection branch, a vertical convolution branch, a channel splicing unit, a 1×1 convolution fusion unit, and a residual connection unit;

[0018] Specifically, the horizontal convolution branch uses 1×K convolution to extract horizontal boundary features, the vertical convolution branch uses K×1 convolution to extract vertical boundary features, the spatial dilation convolution branch uses 3×3 dilation depth separable convolution with dilation rate r to expand the receptive field, and the global projection branch uses global average pooling and 1×1 convolution to obtain global semantic projection features.

[0019] The channel splicing unit splices the outputs of the above branches, the 1×1 convolutional fusion unit performs channel fusion on the spliced ​​features, and the residual connection unit adds the fused features to the input features to obtain the output features of the GDRC module.

[0020] Furthermore, the DCFD module is composed of an HWPD module and a GDRC module connected in series, and its output is expressed as follows:

[0021]

[0022] Where X represents the input feature of the DCFD module, HWPD(X) represents the frequency domain downsampling feature output by the hybrid wavelet-pooling decomposition module, and GDRC(HWPD(X)) represents the output after directional context enhancement of the frequency domain downsampling feature.

[0023] Furthermore, the HWPD module includes a Haar discrete wavelet transform unit, a low-frequency enhancement path, a high-frequency enhancement path, a channel splicing unit, and a channel attention unit;

[0024] The Haar discrete wavelet transform unit is used to decompose the input features into low-frequency approximate components and at least one high-frequency detail component.

[0025] The low-frequency enhancement path is used to concatenate the low-frequency approximate component with the max pooling result and average pooling result of the input feature, and obtain the low-frequency path output through 1×1 convolution, batch normalization and GELU activation.

[0026] The high-frequency enhancement path is used to concatenate the high-frequency detail components with the depth-separable convolution result with a stride of 2, and obtain the high-frequency path output through 1×1 convolution, batch normalization and GELU activation;

[0027] The channel attention unit is used to perform channel weight recalibration on the splicing result of the low-frequency path output and the high-frequency path output to obtain the output features of the HWPD module.

[0028] Furthermore, the bottleneck layer GDRC module is set between the deepest layer of the encoder and the decoder. It is used to establish long-distance semantic associations between the tumor region, vascular structure, liver capsule and liver parenchyma region in the low-resolution high semantic feature space after multi-level DCFD downsampling, through dilated convolution, horizontal branching, vertical branching and global projection branching, and use it as a high-level semantic prior for boundary recovery in the decoding stage.

[0029] Furthermore, the HAFU module includes an upsampling unit, a skip feature-guided attention module (SFGA), a feature additive fusion unit, and a hybrid directional pooling attention module (HDPA).

[0030] The upsampling unit is used to perform channel dimensionality reduction and spatial upsampling on the deep decoding features output by the previous decoding layer;

[0031] The SFGA module is used as a gating signal to filter the skip connection features of the corresponding coding level, using the oversampled deep decoding features as a gating signal, to obtain the semantically calibrated skip features.

[0032] The feature additive fusion unit is used to add the semantically calibrated skip features to the upsampled deep decoding features element by element;

[0033] The HDPA module is used to perform global channel modulation and horizontal and vertical spatial attention calibration on the fused features after element-wise addition to obtain the output features of the HAFU module.

[0034] Furthermore, the SFGA module performs 1×1 convolution mapping and batch normalization on the shallow skip connection features and the upsampled deep decoding features, respectively. The processed two types of features are added element-wise and an intermediate attention matrix is ​​constructed by GELU activation. Then, a spatial attention map is generated by 1×1 convolution, batch normalization and Sigmoid activation. The spatial attention map is used to modulate the shallow skip connection features element-wise, thereby outputting skip features that enhance the lesion region and suppress background noise.

[0035] Furthermore, the HDPA module includes a global channel modulation branch, a horizontal attention branch, and a vertical attention branch;

[0036] The global channel modulation branch is used to perform global average pooling and 1×1 convolution on the fused features, and generates global channel weights through Sigmoid activation.

[0037] The horizontal attention branch is used to extract lateral boundary attention based on horizontal pooling or 1×K direction convolution.

[0038] The vertical attention branch is used to extract longitudinal boundary attention based on vertical pooling or K×1 directional convolution.

[0039] The HDPA module applies global channel weights, lateral boundary attention, and vertical boundary attention to the fused features, and outputs the decoded features with sharpened boundaries through residual connections.

[0040] A lightweight segmentation system for liver cancer images based on multi-scale frequency-space fusion UNet includes:

[0041] The image preprocessing module is used to acquire contrast-enhanced CT images of the liver, and to normalize, resize, and perform data enhancement processing on the CT images to obtain network input images;

[0042] The coding feature extraction module is used to extract features from the network input image through a multi-level coding structure, wherein each coding level includes a GDRC module and a DCFD module. The GDRC module is used to extract directional context features, and the DCFD module is used to perform frequency-space joint downsampling.

[0043] The bottleneck semantic enhancement module is used to perform deep semantic enhancement on the deepest output features of the encoder through the bottleneck layer GDRC module.

[0044] The decoding feature fusion module is used to upsample the features output by the bottleneck semantic enhancement module through the multi-level HAFU module and perform attention fusion with the skip connection features saved at the corresponding coding level.

[0045] The segmentation output module is used to perform 1×1 convolution and Sigmoid mapping on the final decoded features to generate a tumor probability map and generate a tumor segmentation mask according to a preset threshold.

[0046] An electronic device or computer-readable storage medium, the electronic device comprising a processor and a memory, the memory storing a computer program that, when executed by the processor, implements the aforementioned lightweight segmentation method for liver cancer images based on multi-scale frequency-space fusion UNet; or, the computer-readable storage medium storing a computer program that, when executed by the processor, implements the aforementioned method.

[0047] Compared with the prior art, the present invention has at least the following beneficial effects:

[0048] By introducing horizontal, vertical, dilated convolution and global projection branches through the GDRC module, the ability to model directional context is enhanced with a lower number of parameters, thereby improving the ability to identify complex tumor-vascular adjacency boundaries.

[0049] By introducing Haar wavelet decomposition and hybrid pooling mechanism in the downsampling stage through the DCFD module, low-frequency shape information and high-frequency edge information are separated and fused, reducing the loss of spatial details caused by traditional downsampling.

[0050] By using the bottleneck layer GDRC module, long-distance dependencies between tumor regions, vascular structures and liver parenchyma are established in a low-resolution high semantic feature space, providing stable semantic priors for the decoding stage.

[0051] By employing SFGA semantic gating and HDPA directional pooling attention through the HAFU module, shallow background noise is suppressed and tumor boundaries are selectively restored during the decoding stage.

[0052] While maintaining high segmentation accuracy, it significantly reduces the number of model parameters and computational load, making it easy to deploy in clinical terminals or edge computing devices with limited computing resources. Attached Figure Description

[0053] Figure 1 Flowchart of the lightweight segmentation method for MSFA-UNet liver cancer images;

[0054] Figure 2 This is a diagram of the overall network structure of MSFA-UNet;

[0055] Figure 3 Here is a diagram of the Global-Oriented Dilated Residual Convolution (GDRC) module structure.

[0056] Figure 4 The structure diagram of the DCFD module for decomposing context frequency downsampling is shown below;

[0057] Figure 5 Diagram of hybrid wavelet-pooling decomposition HWPD and channel attention structure;

[0058] Figure 6 HAFU module structure diagram for hybrid attention feature upsampling;

[0059] Figure 7 This is a diagram of the collaborative attention fusion structure of SFGA and HDPA;

[0060] Figure 8 This is a schematic diagram of tumor probability map generation and segmentation mask output. Detailed Implementation

[0061] The specific embodiments of the present invention will be further described below with reference to the accompanying drawings. It should be understood that the following embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit the scope of protection of the present invention. Without departing from the core concept of the present invention, those skilled in the art can adaptively adjust the number of network layers, number of channels, convolutional kernel size, threshold parameters, and training parameters.

[0062] This invention provides a lightweight segmentation method and system for liver cancer images based on multi-scale frequency-space fusion UNet, the overall process of which is as follows: Figure 1 As shown, the overall network structure is as follows: Figure 2 As shown, this method takes contrast-enhanced CT images of the liver as input and outputs a tumor probability map and a tumor segmentation mask through an MSFA-UNet network composed of a global-directional dilated residual convolution module, a decomposed context frequency downsampling module, and a hybrid attention feature upsampling module. This method does not simply stack multiple neural network modules sequentially, but follows a continuous processing chain of "directional context establishment—frequency-space joint downsampling—deep semantic enhancement—attention decoding recovery." This allows the encoding stage to obtain frequency-space features with directional constraints, and the decoding stage to selectively recover effective details related to the tumor boundary, thus balancing segmentation accuracy and model lightweighting.

[0063] like Figure 1 As shown, a contrast-enhanced CT image of the patient's liver is first acquired. This CT image can be a two-dimensional CT slice or input into the network from three-dimensional CT volume data in a slice format. To enable CT images obtained from different scanning devices, scanning protocols, and image sizes to enter the same network structure, this embodiment preprocesses the original CT image. This preprocessing includes grayscale normalization, image size adjustment, necessary data enhancement, and channel dimension organization.

[0064] The preprocessed network input image is denoted as:

[0065]

[0066] Where X represents the liver CT image or image feature input to the network, H represents the image height, W represents the image width, and C represents the number of image channels. When the input is a single-channel CT image, C can be 1; when the input is constructed through channel duplication or multi-phase image fusion, C can be a value greater than 1.

[0067] Establish directional context features based on GDRC module

[0068] The preprocessed image first enters the feature extraction unit of the encoder. The feature extraction unit employs a Globally Oriented Dilated Residual Convolution (GDRC) module, whose structure is as follows: Figure 3 As shown, the GDRC module is used to simultaneously extract local multi-scale features, global semantic projection features, and horizontal and vertical anatomical structural features with low computational overhead.

[0069] For input feature X, the GDRC module includes horizontal convolution branches, spatially dilated convolution branches, global projection branches, and vertical convolution branches. The horizontal convolution branches are used to capture boundary cues extending horizontally between the tumor and blood vessels, and the liver capsule. The vertical convolution branches are used to capture longitudinal boundary and blood vessel orientation features. The spatially dilated convolution branches are used to expand the receptive field and extract local multi-scale context. The global projection branches are used to supplement global intensity distribution and regional semantic information.

[0070] In one specific implementation, the hole depth separable convolution DC can be expressed as:

[0071]

[0072] Where W_{DWd} represents the dilated depthwise convolution kernel, W_{PW} represents the pointwise convolution kernel, and represents the convolution operation. Local spatial features in GDRC can be obtained from cascaded DCs:

[0073]

[0074] The global projection branch can be represented as:

[0075]

[0076] The main branch output can be represented as:

[0077]

[0078] Horizontal branches and vertical branches are represented as follows:

[0079]

[0080]

[0081] Furthermore, the parallel branches of GDRC can also be uniformly represented as:

[0082]

[0083]

[0084]

[0085]

[0086] Where F_h represents horizontal features, F_s represents spatially dilated convolutional features, F_g represents global projection features, and F_v represents vertical features; DC_{1×K} represents 1×K direction convolution, DC_{K×1} represents K×1 direction convolution, DC_{3×3}^{(r)} represents 3×3 depth-separable convolution with dilation rate r, GAP represents global average pooling, Conv_{1×1} represents 1×1 convolution, BN represents batch normalization, and GELU represents the activation function.

[0087] Subsequently, the outputs of the above branches are concatenated along the channel dimension and fused using a 1×1 convolution. Then, a residual connection is established with the input features to obtain the output of the GDRC module:

[0088]

[0089]

[0090] Here, Concat represents channel concatenation, and R(X) represents residual mapping on the input feature X. When the number of channels of the input feature and the fused feature are the same, R(X) can be an identity mapping; when the number of channels is different, R(X) can be adjusted by using a 1×1 convolution.

[0091] Through the above structure, the GDRC module can enhance the model's ability to distinguish irregular morphology of liver cancer lesions, tumor-vascular adjacency boundaries, and background interference from liver parenchyma without significantly increasing the number of parameters.

[0092] Frequency-space joint downsampling based on DCFD module

[0093] After feature extraction at each level, the encoder performs downsampling through the decomposition context frequency downsampling module. The DCFD module structure is as follows: Figure 4As shown, the DCFD module consists of a hybrid wavelet-pooling decomposition module (HWPD) and a GDRC module connected in series. Its core principle is to first explicitly separate low-frequency shape information from high-frequency edge information during downsampling using the HWPD module, and then apply directional context constraints to the edge responses retained in the frequency domain using the GDRC module, thereby preventing the indiscriminate transmission of high-frequency texture noise.

[0094] like Figure 5 As shown, the HWPD module first performs Haar discrete wavelet transform on the input features X_f in the current encoding stage to obtain low-frequency approximate components and high-frequency detail components:

[0095]

[0096] Where X_f represents the input features in the current downsampling stage, Y_L represents the low-frequency approximation component, and Y_{HL}, Y_{LH}, and Y_{HH} represent high-frequency detail subbands in different directions, respectively. The low-frequency component mainly includes the shape of the liver region, the overall intensity distribution, and the main structure information of the tumor; the high-frequency component mainly includes the edge of the lesion, the edge of the blood vessel wall, the edge of the liver capsule, and information on local texture changes.

[0097] To balance global strength robustness and boundary detail preservation, the HWPD module sets up low-frequency and high-frequency enhancement paths. The low-frequency enhancement path processes the input features through max pooling and average pooling respectively, and then concatenates them with the wavelet low-frequency components:

[0098]

[0099]

[0100]

[0101] Where Out_{LF} represents the low-frequency path output, which is used to preserve the overall shape and intensity distribution of the image.

[0102] The high-frequency enhancement path processes the input features through a depthwise separable convolution with a stride of 2, and then concatenates the input features with the wavelet high-frequency components via channel concatenation.

[0103]

[0104]

[0105] Where DSC_{s=2} represents a depthwise separable convolution with a stride of 2, and Out_{HF} represents the high-frequency path output, which is used to preserve multi-directional edge information and local geometric changes.

[0106] Subsequently, the low-frequency path output and the high-frequency path output are fused at the channel level and then input into the channel attention module (CA) for recalibration.

[0107]

[0108]

[0109] Where σ represents the Sigmoid function, W_1 and W_2 represent the learnable mappings in channel attention, and GAP represents global average pooling. Through the CA module, the network can adaptively adjust the weights of low-frequency shape features and high-frequency edge features based on differences in tumor regions, liver parenchyma background, and vascular enhancement signals in different CT images.

[0110] The HWPD module output is then connected to the GDRC module to form a complete DCFD module:

[0111]

[0112] The technical significance of this cascaded structure lies in the following: the HWPD module can preserve low-frequency main structures and high-frequency edge details that are easily lost during downsampling, but simply preserving high frequencies cannot distinguish between tumor edges, blood vessel edges, and liver parenchyma texture edges; the GDRC module further constrains the high-frequency response anatomically through horizontal and vertical branches and dilated residual convolution branches, transforming it into frequency-space features with tumor contour directionality. Therefore, the DCFD module can reduce boundary loss caused by traditional pooling while lowering spatial resolution, and suppress false activation of background textures that may be caused by simple frequency domain enhancement.

[0113] Multi-level coding and bottleneck layer semantic enhancement

[0114] like Figure 2 As shown, MSFA-UNet sets up multiple coding levels during the encoding stage. Each coding level first extracts directional context features at the current scale through the GDRC module, and then performs frequency-space joint downsampling through the DCFD module. After each downsampling, the feature map space size is reduced to half of its original size, the number of channels is increased accordingly, and the coding features of this layer are saved as skip connection features for the subsequent decoding stage.

[0115] In one implementation, the encoder may include five scale levels: L1, L2, L3, L4, and L5. The L1 layer retains shallow texture features at or near the original resolution, while layers L2 through L5 progressively reduce spatial resolution and enhance semantic abstraction capabilities. The encoded features of each layer are denoted as E_1, E_2, E_3, E_4, and E_5, respectively.

[0116] After multi-level DCFD downsampling, the deepest encoded feature E_5 enters the bottleneck layer GDRC module. The bottleneck layer GDRC does not simply repeat shallow feature extraction; instead, it further establishes global directional consistency within a low-resolution, high-semantic space, integrating long-distance dependencies between the tumor region, vascular structures, liver capsule, and surrounding parenchymal regions. The bottleneck layer output is denoted as B.

[0117]

[0118] The output of this bottleneck layer serves as the initial input to the decoder. Through this design, the frequency-space boundary information retained during the encoding stage undergoes deep semantic integration before entering the decoding stage, providing a more stable high-level semantic prior for subsequent boundary recovery.

[0119] Attention decoding and fusion based on the HAFU module

[0120] The decoding stage employs the Hybrid Attention Feature Upsampling (HAFU) module for step-by-step upsampling and skip feature fusion, and its overall structure is as follows: Figure 6 As shown, the HAFU module is used to replace the simple splicing skip connections in traditional U-Net, avoiding the indiscriminate recovery of superficial high-frequency noise, vascular enhancement signals, and liver parenchyma texture.

[0121] For a given decoding level, let the deep semantic features output by the previous decoding layer be D_i, and the shallow skip features stored in the corresponding encoding layer be E_i. First, D_i is subjected to depthwise separable convolutional dimensionality reduction and upsampling to make its spatial resolution consistent with E_i:

[0122]

[0123] Where Up represents bilinear interpolation or transposed convolution upsampling, and DC represents depthwise separable convolution.

[0124] Subsequently, the HAFU module guides the attention module SFGA to filter shallow skip features through skip feature filtering. The SFGA structure is as follows: Figure 7 As shown on the left. The basic idea is to use the oversampled deep semantic features U_i as a gating signal to perform attention calibration on the shallow skip features E_i, preserving the boundaries and textures related to the tumor region while suppressing background noise in the liver parenchyma. SFGA can be represented as:

[0125]

[0126]

[0127] Where C_e, C_u, and C represent 1×1 convolutional mappings, and E_i' represents the skip features after semantic gating.

[0128] Then, the filtered skip features E_i' are added element-wise to the upsampled features U_i:

[0129]

[0130] Compared with traditional channel splicing, element-wise addition can reduce the number of parameters and computation caused by channel expansion, while preserving the effective boundary information at the encoding end.

[0131] Furthermore, the HAFU module performs global channel and orientation space calibration on the fused feature Z_i using the Hybrid Directional Pooling Attention (HDPA) module. The HDPA structure is as follows: Figure 7 As shown on the right. First, global average pooling is used to extract channel statistics and generate global channel modulation weights:

[0132]

[0133]

[0134] Where G_i represents the global channel attention weight, and Z_i^g represents the fused feature after global channel modulation.

[0135] Subsequently, max pooling is performed on Z_i^g, and directional boundary attention is extracted through horizontal and vertical branches respectively:

[0136]

[0137] Where H represents the horizontal feature extraction branch, V represents the vertical feature extraction branch, and L_i represents the local spatial orientation attention map. For narrow, wedge-shaped, and irregular tumor boundaries adjacent to the blood vessel wall or liver capsule, horizontal and vertical attention can enhance boundary-specific expression.

[0138] Finally, the HDPA output is:

[0139]

[0140] Where D_{i+1} represents the output feature of the current decoding layer. Through the synergy of SFGA and HDPA, the HAFU module first uses high-level semantics to filter shallow skip features, and then uses channel and directional attention to sharpen the boundaries, so that the frequency-space information retained in the encoding stage is not indiscriminately recovered, but is selectively enhanced under semantic constraints.

[0141] Tumor probability map and segmentation mask generation

[0142] The decoder executes the HAFU module step by step until the target spatial resolution is restored. The final decoded feature is denoted as D_{out}. Figure 8As shown, a 1×1 convolution is applied to D_{out} and mapped to a tumor probability map P using the Sigmoid function:

[0143]

[0144] Where P(x,y) represents the probability that pixel (x,y) belongs to the tumor region. The probability map is then binarized according to a preset threshold T to obtain the tumor segmentation mask M.

[0145]

[0146] Here, T can be set to 0.5 or adjusted based on the validation set performance. M(x,y)=1 indicates that the corresponding pixel is identified as a tumor region, and M(x,y)=0 indicates that the corresponding pixel is identified as a non-tumor region. The obtained segmentation mask can be used for automatic tumor region delineation, preoperative planning, radiomics feature extraction, and clinical auxiliary analysis.

[0147] Model training and loss function

[0148] During the model training phase, enhanced CT images and their manually annotated tumor masks are used as training samples. Let the network prediction probability map be Y_pred, and the ground truth mask be Y_true. In one implementation, a weighted combination of Dice loss and binary cross-entropy loss is used to train the network.

[0149]

[0150] Here, {Dice} is used to improve the overlap between the predicted region and the ground truth labeled region, and {BCE} is used to constrain pixel-level classification accuracy. During training, the AdamW optimizer and mixed-precision training methods can be used to reduce memory usage and improve training efficiency.

[0151] Model evaluation metrics and experimental results

[0152] To verify the segmentation performance and lightweight effect of the present invention, the Dice similarity coefficient, intersection-over-union ratio (IoU), Hausdorff distance (HD), number of parameters, and number of floating-point operations (FLOPs) can be used for evaluation.

[0153] The Dice similarity coefficient is expressed as:

[0154]

[0155] The intersection-union ratio is expressed as:

[0156]

[0157] Hausdorff distance is expressed as:

[0158]

[0159] Where A represents the predicted tumor region, B represents the actual labeled region, d(a,b) represents the Euclidean distance between the predicted boundary point a and the actual boundary point b, sup represents the supremum, and inf represents the infremum.

[0160] Experimental results on the LiTS2017 dataset show that the MSFA-UNet in this embodiment achieves a Dice of 86.01%, an IoU of 79.56%, an HD of 2.5466, 10.78M parameters, and 28.83G FLOPs. Compared with UNet, UNet++, Attention-UNet, nnUNet, Trans-UNet, and PDF-UNet, this invention achieves higher overlap segmentation accuracy and better boundary fitting ability while maintaining a lower parameter count and computational cost.

[0161] Performance comparison of the MSFA-UNet liver cancer segmentation system on the LiTS dataset: UNet's Dice is 81.89%, IoU is 74.81%, HD is 2.7685, parameter count is 31.38M, and FLOPs are 223.44G; UNet++'s Dice is 79.40%, IoU is 71.85%, HD is 2.9073, parameter count is 47.19M, and FLOPs are 800.91G; Attention-UNet's Dice is 80.07%, IoU is 72.85%, HD is 2.8782, parameter count is 34.88M, and FLOPs are 266.81G; nnUNet's Dice is 82.12%, IoU is 74.81%, HD is 2.7685, parameter count is 31.38M, and FLOPs are 223.44G; The Dice of Trans-UNet is 75.02%, IoU is 2.7568, HD is 31.39M, and FLOPs are 223.45G; the IoU of Trans-UNet is 80.63%, IoU is 73.38%, HD is 2.8587, HD is 105.91M, and FLOPs are 148.22G; the Dice of PDF-UNet is 82.98%, IoU is 76.02%, HD is 2.7362, HD is 42.06M, and FLOPs are 177.57G; the Dice of MSFA-UNet of this invention is 86.01%, IoU is 79.56%, HD is 2.5466, HD is 10.78M, and FLOPs are 28.83G.

[0162] Ablation comparison of the MSFA-UNet liver cancer segmentation system on the LiTS dataset: Without DCFD, GDRC, and HAFU, the Dice is 81.89%, IoU is 74.81%, HD is 2.7685, the number of parameters is 31.38M, and FLOPs are 223.44G; with only DCFD, the Dice is 84.43%, IoU is 77.74%, HD is 2.6386, the number of parameters is 19.63M, and FLOPs are 185.66G; with only GDRC, the Dice is 84.90%, IoU is 78.40%, HD is 2.5999, the number of parameters is 6.25M, and FLOPs are 49.18G; with only HAFU, the Dice is 83.85%, IoU is 76.99%, HD is 2.6614, the number of parameters is 19.95M, and FLOPs are [not specified in the original text]. The Dice was 71.80G; when DCFD and GDRC were introduced simultaneously, the Dice was 85.82%, IoU was 79.38%, HD was 2.5619, the number of parameters was 9.78M, and FLOPs was 58.26G; when DCFD and HAFU were introduced simultaneously, the Dice was 85.03%, IoU was 78.47%, HD was 2.5920, the number of parameters was 8.20M, and FLOPs was 34.30G; when GDRC and HAFU were introduced simultaneously, the Dice was 84.60%, IoU was 78.10%, HD was 2.6092, the number of parameters was 7.31M, and FLOPs was 19.46G; when DCFD, GDRC, and HAFU were introduced simultaneously, the Dice was 86.01%, IoU was 79.56%, HD was 2.5466, the number of parameters was 10.78M, and FLOPs was 28.83G.

[0163] Ablation experiments show that using DCFD, GDRC, or HAFU individually can improve segmentation performance, but the best results are achieved when all three are cascaded in the order specified in this invention. This demonstrates that GDRC provides directional context, DCFD preserves frequency-spatial boundary information during the downsampling phase, the bottleneck layer GDRC enhances deep semantic consistency, and HAFU selectively recovers effective boundaries during the decoding phase. Together, these three technologies form a collaborative technical chain of "encoding to preserve boundaries—semantic direction determination—decoding to suppress noise and restore."

[0164] The overall implementation process of this invention can be summarized as follows:

[0165] Input: Contrast-enhanced CT image X;

[0166] Output: Tumor probability map P and tumor segmentation mask M;

[0167] Step 1: Normalize, resize, and augment the input CT image X to obtain the network input features;

[0168] Step 2: Input the input features into the encoder. In each coding level, directional context features are first established through the GDRC module, and then frequency-space joint downsampling is performed through the DCFD module. The coding features of the corresponding level are saved as skip connection features.

[0169] Step 3: Input the deepest encoded features into the bottleneck layer GDRC module to obtain deep semantic enhancement features;

[0170] Step 4: Input the bottleneck layer output into the decoder. In each decoding layer, the HAFU module is used to receive the features of the previous decoding layer and the corresponding coding layer jump features. Semantic gating is performed by SFGA, and channel and directional attention calibration is performed by HDPA to obtain the current decoding layer output.

[0171] Step 5: Generate a tumor probability map P by passing the final decoded features through 1×1 convolution and the Sigmoid function;

[0172] Step 6: Binarize the probability map P according to the threshold T to obtain the tumor segmentation mask M;

[0173] Step 7: Output the tumor segmentation mask for auxiliary delineation of liver cancer lesion areas, preoperative planning, and radiomics analysis.

[0174] The technical implementation of the MSFA-UNet liver tumor segmentation algorithm is described below:

[0175] Step 1: Define the MSFA-UNet model, input the contrast-enhanced CT image X, and output the predicted tumor probability mask Y_pred.

[0176] Step 2: Enter the feature encoding stage, and pass the input image X to the GDRC module to extract the initial multi-scale spatial features X_1.

[0177] Step 3: Apply the DCFD module to the current feature map X_i to reduce the spatial size, output X_{i+1}, and store it in the buffer pool for subsequent skip connections.

[0178] Step 4: Enter the bottleneck layer, apply the GDRC module again to the deepest feature map X_5, capture deep semantics and output it to the decoder.

[0179] Step 5: Enter the feature decoding stage, retrieve the corresponding encoder layer jump feature from the cache pool, and input it and the current decoded feature into the HAFU module.

[0180] Step 6: In the HAFU module, depthwise separable convolutional dimensionality reduction and upsampling are performed on the decoded features; the SFGA module is called to perform attention calibration on the skip features; the denoised skip features are added element-wise to the upsampled features; the HDPA module is called to extract the global channel weights and bidirectional local spatial weights, and the features of the next decoding layer are output.

[0181] Step 7: Apply a 1×1 convolution and the Sigmoid function to the final decoded features to generate the predicted probability map Y_pred.

[0182] Step 8: During the training phase, the AdamW optimizer, mixed precision training, and Dice+BCE joint loss can be used; during the inference phase, the MSFA-UNet weights with the best validation set metrics are loaded and the prediction mask is output.

[0183] In summary, this invention enables the network to achieve joint feature modeling in the frequency and spatial domains during the encoding stage, deep semantic integration during the bottleneck stage, and effective boundary selective recovery during the decoding stage through the restrictive concatenation of GDRC, DCFD, and HAFU. This balances segmentation accuracy, boundary preservation capability, and lightweight deployment capability in complex liver cancer CT image segmentation tasks.

Claims

1. A lightweight segmentation method for liver cancer images based on multi-scale frequency-space fusion UNet, characterized in that, Includes the following steps: S1, acquire contrast-enhanced CT images of the liver, and preprocess the CT images to obtain network input images; S2, the network input image is input into the encoder of the MSFA-UNet network. In each encoding level, the directional context features are first extracted by the Global-Oriented Dilated Residual Convolution (GDRC) module, and then downsampled by the Decomposed Context Frequency Downsampling (DCFD) module. The encoding features output by the corresponding encoding level are saved as skip connection features. The GDRC module performs horizontal convolution, spatial dilated convolution, global projection, and vertical convolution on the input features. After channel concatenation and 1×1 convolution fusion of the outputs of each branch, a residual connection is established with the input features to obtain features with local multi-scale information, global semantic information, and horizontal and vertical contextual information. The DCFD module first decomposes the input features into low-frequency and high-frequency information through the Hybrid Wavelet-Pooling Decomposition Module (HWPD), and then fuses the low-frequency and high-frequency paths. The fused frequency domain downsampled features are then input into the GDRC module for context enhancement to obtain frequency-space downsampled features with directional constraints. S3, input the encoded features output from the deepest layer of the encoder into the bottleneck layer GDRC module to obtain deep semantic enhancement features; S4. Input the deep semantic enhancement features into the decoder. In each decoding layer, the deep decoding features output by the previous decoding layer and the skip connection features saved in the corresponding encoding layer are received through the Hybrid Attention Feature Upsampling Module (HAFU). The skip connection features are semantically gated and filtered using the deep decoding features. The fused features are then calibrated with channel and directional attention to obtain the output features of the current decoding layer. S5. Perform 1×1 convolution and Sigmoid mapping on the output features of the final decoding layer to obtain a tumor probability map, and binarize the tumor probability map according to a preset threshold to obtain a liver cancer tumor segmentation mask.

2. The lightweight segmentation method for liver cancer images based on multi-scale frequency-space fusion UNet according to claim 1, characterized in that, The GDRC module includes a horizontal convolution branch, a spatially dilated convolution branch, a global projection branch, a vertical convolution branch, a channel splicing unit, a 1×1 convolution fusion unit, and a residual connection unit. Specifically, the horizontal convolution branch uses 1×K convolution to extract horizontal boundary features, the vertical convolution branch uses K×1 convolution to extract vertical boundary features, the spatial dilation convolution branch uses 3×3 dilation depth separable convolution with dilation rate r to expand the receptive field, and the global projection branch uses global average pooling and 1×1 convolution to obtain global semantic projection features. The channel splicing unit splices the outputs of the above branches, the 1×1 convolutional fusion unit performs channel fusion on the spliced ​​features, and the residual connection unit adds the fused features to the input features to obtain the output features of the GDRC module.

3. The lightweight segmentation method for liver cancer images based on multi-scale frequency-space fusion UNet according to claim 1, characterized in that, The DCFD module is composed of an HWPD module and a GDRC module connected in series, and its output is expressed as follows: ； Where X represents the input feature of the DCFD module, HWPD(X) represents the frequency domain downsampling feature output by the hybrid wavelet-pooling decomposition module, and GDRC(HWPD(X)) represents the output after directional context enhancement of the frequency domain downsampling feature.

4. The lightweight segmentation method for liver cancer images based on multi-scale frequency-space fusion UNet according to claim 3, characterized in that, The HWPD module includes a Haar discrete wavelet transform unit, a low-frequency enhancement path, a high-frequency enhancement path, a channel splicing unit, and a channel attention unit. The Haar discrete wavelet transform unit is used to decompose the input features into low-frequency approximate components and at least one high-frequency detail component. The low-frequency enhancement path is used to concatenate the low-frequency approximate component with the max pooling result and average pooling result of the input feature, and obtain the low-frequency path output through 1×1 convolution, batch normalization and GELU activation. The high-frequency enhancement path is used to concatenate the high-frequency detail components with the depth-separable convolution result with a stride of 2, and obtain the high-frequency path output through 1×1 convolution, batch normalization and GELU activation; The channel attention unit is used to perform channel weight recalibration on the splicing result of the low-frequency path output and the high-frequency path output to obtain the output features of the HWPD module.

5. The lightweight segmentation method for liver cancer images based on multi-scale frequency-space fusion UNet according to claim 1, characterized in that, The bottleneck layer GDRC module is located between the deepest layer of the encoder and the decoder. It is used to further establish long-distance semantic associations between tumor regions, vascular structures, liver capsule and liver parenchyma regions in the low-resolution high semantic feature space after multi-level DCFD downsampling through dilated convolution, horizontal branching, vertical branching and global projection branching, and use them as high-level semantic priors for boundary recovery in the decoding stage.

6. The lightweight segmentation method for liver cancer images based on multi-scale frequency-space fusion UNet according to claim 1, characterized in that, The HAFU module includes an upsampling unit, a skip feature guided attention module SFGA, a feature additive fusion unit, and a hybrid directional pooling attention module HDPA. The upsampling unit is used to perform channel dimensionality reduction and spatial upsampling on the deep decoding features output by the previous decoding layer; The SFGA module is used as a gating signal to filter the skip connection features of the corresponding coding level, using the oversampled deep decoding features as a gating signal, to obtain the semantically calibrated skip features. The feature additive fusion unit is used to add the semantically calibrated skip features to the upsampled deep decoding features element by element; The HDPA module is used to perform global channel modulation and horizontal and vertical spatial attention calibration on the fused features after element-wise addition to obtain the output features of the HAFU module.

7. The lightweight segmentation method for liver cancer images based on multi-scale frequency-space fusion UNet according to claim 6, characterized in that, The SFGA module performs 1×1 convolution mapping and batch normalization on shallow skip connection features and upsampled deep decoding features, respectively. The processed two types of features are added element-wise and an intermediate attention matrix is ​​constructed by GELU activation. Then, a spatial attention map is generated by 1×1 convolution, batch normalization and Sigmoid activation. The spatial attention map is used to modulate the shallow skip connection features element-wise, thereby outputting skip features that enhance the lesion region and suppress background noise.

8. The lightweight segmentation method for liver cancer images based on multi-scale frequency-space fusion UNet according to claim 6, characterized in that, The HDPA module includes a global channel modulation branch, a horizontal attention branch, and a vertical attention branch; The global channel modulation branch is used to perform global average pooling and 1×1 convolution on the fused features, and generates global channel weights through Sigmoid activation. The horizontal attention branch is used to extract lateral boundary attention based on horizontal pooling or 1×K direction convolution. The vertical attention branch is used to extract longitudinal boundary attention based on vertical pooling or K×1 directional convolution. The HDPA module applies global channel weights, lateral boundary attention, and vertical boundary attention to the fused features, and outputs the decoded features with sharpened boundaries through residual connections.

9. A lightweight segmentation system for liver cancer images based on multi-scale frequency-space fusion UNet, characterized in that, include: The image preprocessing module is used to acquire contrast-enhanced CT images of the liver, and to normalize, resize, and perform data enhancement processing on the CT images to obtain network input images; The coding feature extraction module is used to extract features from the network input image through a multi-level coding structure, wherein each coding level includes a GDRC module and a DCFD module. The GDRC module is used to extract directional context features, and the DCFD module is used to perform frequency-space joint downsampling. The bottleneck semantic enhancement module is used to perform deep semantic enhancement on the deepest output features of the encoder through the bottleneck layer GDRC module. The decoding feature fusion module is used to upsample the features output by the bottleneck semantic enhancement module through the multi-level HAFU module and perform attention fusion with the skip connection features saved at the corresponding coding level. The segmentation output module is used to perform 1×1 convolution and Sigmoid mapping on the final decoded features to generate a tumor probability map and generate a tumor segmentation mask according to a preset threshold.

10. An electronic device or computer-readable storage medium, characterized in that, The electronic device includes a processor and a memory, wherein the memory stores a computer program that, when executed by the processor, implements the lightweight segmentation method for liver cancer images based on multi-scale frequency-space fusion UNet as described in any one of claims 1 to 8; or, the computer-readable storage medium stores a computer program that, when executed by the processor, implements the method described in any one of claims 1 to 8.

Images