Neural network model for liver CT image segmentation and liver cancer classification method based on neural network model

Abstract

The invention discloses a neural network model for liver CT image segmentation and a liver cancer classification method based on the neural network model. The neural network model adopts a feature encoder for performing down-sampling feature extraction on the liver CT image; a UNet + + model of a multi-scale feature fusion module connected between the feature encoders and the feature decoders and used for aggregating feature information of different levels, wherein the feature decoders are used for performing up-sampling on the features extracted by the feature encoders to recover spatial dimensions; the feature encoder is integrated with a high-efficiency channel attention module used for enhancing attention to a liver focus key area and inhibiting irrelevant background tissue information and a residual module used for maintaining a network gradient flow; through aggregation and fusion of different levels of feature information, more detail features are reserved when the image space dimension is restored, so that the segmentation precision of the liver and the lesion boundary thereof is remarkably improved, and the segmentation result is more accurate and robust.

Description

Technical Field

[0001] This invention belongs to the field of computer vision technology, and particularly relates to a neural network model for liver CT image segmentation and a liver cancer classification method based on the neural network model. Background Technology

[0002] Primary liver cancer is one of the most common malignant tumors worldwide, and its early and accurate identification is crucial for treatment planning and patient prognosis. Clinically, liver cancer is usually divided into HCC and non-HCC, which differ significantly in malignancy and treatment response. Therefore, accurate classification based on imaging is particularly important.

[0003] Currently, CT is widely used due to its fast imaging speed and high resolution. However, manual segmentation relies on experience, is inefficient, and highly subjective. Traditional algorithms such as thresholding, region growing, and watershed methods perform poorly under noise interference and blurred boundaries, exhibiting limited robustness. With the development of artificial intelligence technology, deep learning methods based on convolutional neural networks (CNNs), especially fully convolutional networks (FCNs), U-Net, and their variants, have been widely applied to medical image segmentation tasks to achieve automated analysis. However, when facing the specific challenge of liver CT image segmentation, the performance of these existing models remains unsatisfactory. Because the boundaries between the liver and its lesions and surrounding tissues are often blurred and have low contrast, and structures like U-Net are prone to losing subtle spatial information during multiple downsampling, the segmentation of boundaries is not accurate enough. Furthermore, the severe imbalance in the number of foreground and background pixels commonly found in medical images makes it difficult for models to fully focus on tumor regions that occupy a very small proportion of the image, resulting in missed detections of small lesions.

[0004] In the classification and discrimination of liver cancer, although deep learning models, represented by CNNs, possess powerful feature extraction capabilities, they still have significant limitations in practical clinical applications. On the one hand, subtypes of primary liver cancer (such as HCC and non-HCC) show subtle differences in imaging manifestations, and existing classification models have limited ability to capture these distinctive but insignificant medical features, making it difficult to further improve classification accuracy. On the other hand, many high-performance models have large parameters and high computational costs, making it difficult to achieve rapid deployment and application in efficient clinical environments. Summary of the Invention

[0005] To address the problems existing in the prior art, this invention proposes a neural network model for liver CT image segmentation and a liver cancer classification method based on the neural network model.

[0006] The technical solution of the present invention is as follows:

[0007] A neural network model for liver CT image segmentation, wherein the neural network model employs a UNet++ model having a feature encoder for downsampling feature extraction of liver CT images, a feature decoder for upsampling the features extracted by the feature encoder to restore spatial dimensions, and a multi-scale feature fusion module connected between the feature encoder and the feature decoder for aggregating feature information at different levels;

[0008] The feature encoder integrates an efficient channel attention (ECA) module for enhancing focus on key areas of liver lesions and suppressing irrelevant background tissue information, and a residual module for maintaining network gradient flow.

[0009] The neural network model is trained based on a preprocessed liver CT image dataset and using the Dice Loss loss function, and is used to output a mask for the liver region.

[0010] Furthermore, the feature encoder uses ResNet-34 as its backbone network and contains four layers; each layer consists of multiple ECA-Res modules, and the ECA-Res modules are composed of residual modules and efficient channel attention modules connected sequentially.

[0011] The residual module includes a first convolutional layer, a batch normalization layer, and a ReLU activation function layer connected in sequence, as well as a short connection structure between the second convolutional layer, the batch normalization layer, and the identity mapping, which is used to realize cross-layer feature propagation and gradient preservation.

[0012] The efficient channel attention module obtains the interaction weights between channels by using one-dimensional convolution operations in the channel dimension, thereby enhancing the model's response to key areas of liver lesions.

[0013] Furthermore, the feature decoder comprises four modules, each consisting of a deconvolutional layer with a stride of 2, a convolutional layer with a kernel size of 3x3, a batch normalization layer, and a ReLU activation function layer; the end of the feature decoder is a convolutional layer with a kernel size of 1x1 and a Sigmoid activation function.

[0014] Furthermore, the overall structure of the neural network model includes an input layer, a feature encoder, a feature decoder, an output layer, and a loss calculation unit L located at the top layer of the decoding end, connected in sequence.

[0015] The feature encoder consists of four sequentially connected ECA-Res modules, corresponding to the four main stages of the ResNet-34 network. Each ECA-Res module is composed of a residual structure and an efficient channel attention module connected sequentially. Each ECA-Res module internally includes two 3×3 convolutional layers, a batch normalization layer, and a ReLU activation function. An efficient channel attention module is introduced after the second convolutional layer to perform channel attention weighting, and feature transfer across layers is achieved through identity mapping residual connections. Adjacent ECA-Res modules are downsampled through a max pooling layer with a stride of 2.

[0016] The feature decoder includes four Double Conv modules. Each Double Conv module includes two 3×3 convolutional layers, a batch normalization layer, and a ReLU activation function, which are used to fuse and progressively upsample the features passed through skip connections. Adjacent Double Conv modules achieve upsampling through deconvolutional layers with a stride of 2 to restore spatial resolution.

[0017] The feature encoder and feature decoder are connected by a multi-level skip connection. Each level of the decoding layer receives feature information from the corresponding encoding layer and the next level of the decoding layer, forming a densely nested multi-scale fusion path.

[0018] The output of the neural network model is configured with a 1×1 convolutional layer and a Sigmoid activation function to generate pixel-level segmentation masks for the liver and lesion areas.

[0019] Furthermore, the expression for the loss function is:

[0020]

[0021] In the formula, This is the set of pixels representing the liver and lesion regions predicted by the model. This is a collection of pixels representing the liver and lesion areas that are actually annotated. This indicates the pixel-level overlap between the predicted result and the actual annotation. and These represent the number of foreground pixels in the predicted mask and the real mask, respectively.

[0022] Furthermore, the preprocessing includes data augmentation operations on the liver CT image dataset, the data augmentation operations including at least one of random rotation, horizontal or vertical flipping, affine transformation, brightness and contrast adjustment, and color saturation adjustment.

[0023] Furthermore, the samples in the dataset include liver CT images and corresponding labels, wherein the labels are pixel-level masks of the liver and lesion regions used for segmentation of liver CT images.

[0024] A liver cancer classification method based on a neural network model includes:

[0025] The preprocessed target liver CT image is input into any of the neural network models for liver CT image segmentation described above to obtain a mask for the liver region;

[0026] A mask of the liver region is input into the pre-tuned EfficientNet model through transfer learning to determine whether the target liver CT image belongs to HCC, non-HCC, or normal tissue.

[0027] Furthermore, the EfficientNet model is a variant model of EfficientNet-B0, EfficientNet-B1, EfficientNet-B2, EfficientNet-B3, or EfficientNet-B4 with a global average pooling layer, a fully connected layer, and a Softmax activation function layer added on top.

[0028] Furthermore, the specific method for fine-tuning the transfer learning includes freezing the bottom convolutional layers of the EfficientNet model and fine-tuning only the top convolutional layers and fully connected layers; the loss function used during fine-tuning is the cross-entropy loss function, and the optimizer is Adam.

[0029] Compared with the prior art, the present invention has the following beneficial effects:

[0030] This invention proposes a neural network model for liver CT image segmentation and a liver cancer classification method based on the neural network model. The neural network model integrates a feature encoder, a feature decoder, and a multi-scale feature fusion module. By aggregating and fusing feature information at different levels, it retains more detailed features when restoring the spatial dimension of the image, thereby significantly improving the segmentation accuracy of the liver and its lesion boundaries, making the segmentation results more accurate and robust.

[0031] The efficient channel attention (ECA) module and residual module introduced into the feature encoder of the neural network model in this invention effectively enhance the focus on key areas of liver lesions while suppressing interference from irrelevant background tissue information. This design solves the problem of the model's difficulty in focusing on small lesions in medical images due to the severe imbalance in the number of foreground and background pixels. The addition of the residual module maintains the stability of the network gradient flow, alleviates the gradient vanishing phenomenon in deep network training, and enables the model to learn features more stably and efficiently in complex CT images, thereby improving the ability to recognize blurred boundaries and low-contrast regions.

[0032] The neural network model training in this invention employs the Dice Loss function, specifically designed to address the class imbalance problem commonly encountered in medical image segmentation. This loss function emphasizes the overlap between foreground pixels and the predicted results, reducing the dominant influence of background pixels during training. This significantly improves the model's sensitivity to detecting small lesions, effectively reduces the false negative rate, and enhances the reliability of segmentation performance.

[0033] The neural network model constructed in this invention employs a preprocessed dataset. The preprocessing stage includes various data augmentation operations, such as random rotation, flipping, affine transformation, and brightness / contrast adjustment, effectively expanding the diversity of the training data and resolving the overfitting problem caused by limited data volume. This measure enhances the model's generalization ability under different imaging conditions, enabling it to better adapt to the diverse liver CT images encountered in actual clinical practice.

[0034] The hepatocellular carcinoma (HCC) classification method of this invention, based on a neural network model, inputs a segmented liver region mask into an EfficientNet model finely tuned through transfer learning. By focusing on the liver region rather than the entire image, it solves the problem of insufficient capture of subtle features by the classification model due to background interference, enabling the model to more accurately distinguish between HCC, non-HCC, and normal tissue. The selection of a variant of the EfficientNet model, with the addition of global average pooling, fully connected layers, and a Softmax activation function on top, optimizes computational efficiency while maintaining classification performance, meeting the speed and resource requirements of clinical deployment.

[0035] The liver cancer classification method of this invention adopts a transfer learning fine-tuning strategy. By freezing the bottom convolutional layers and optimizing only the top-level structure, combined with cross-entropy loss and Adam optimizer, the training time and computational cost are greatly reduced, and the adaptability and efficiency of the model in practical applications are improved. Attached Figure Description

[0036] Figure 1 A schematic diagram of the neural network model used for liver CT image segmentation;

[0037] Figure 2 This is a flowchart illustrating a liver cancer classification method based on a neural network model.

[0038] Figure 3 This is a flowchart illustrating the specific implementation of the present invention;

[0039] Figure 4(a) is a schematic diagram of the ECA-Res module structure; Figure 4(b) is a schematic diagram of the standard Res module structure.

[0040] Figure 5 This is a schematic diagram of the SE module;

[0041] Figure 6Comparison chart of U-Net, U-Net++, Res-UNet++, Eres-UNet++, and Eca-UNet++ losses;

[0042] Figure 7 Here is a diagram of the MBConv module structure;

[0043] Figure 8 A diagram illustrating the principles of transfer learning;

[0044] Figure 9 This is a network architecture diagram of EfficientNetB1;

[0045] Figure 10 Here is the accuracy curve of the Liver-EffNet model;

[0046] Figure 11 The graph shows the loss function of Liver-EffNet.

[0047] Figure 12 A comparison chart showing the impact of different data splits on Liver-EffNet performance; Detailed Implementation

[0048] The present invention will be further illustrated below with reference to the accompanying drawings and specific embodiments. It should be understood that these embodiments are for illustrative purposes only and are not intended to limit the scope of the invention. After reading this invention, any modifications of the invention in various equivalent forms by those skilled in the art will fall within the scope defined by the appended claims.

[0049] Example 1:

[0050] The present invention provides a neural network model for liver CT image segmentation. The neural network model employs a UNet++ model, which includes a feature encoder for downsampling feature extraction of liver CT images, a feature decoder for upsampling the features extracted by the feature encoder to restore spatial dimensions, and a multi-scale feature fusion module connecting the feature encoder and the feature decoder to aggregate feature information at different levels.

[0051] The feature encoder integrates an efficient channel attention (ECA) module for enhancing focus on key areas of liver lesions and suppressing irrelevant background tissue information, and a residual module for maintaining network gradient flow.

[0052] The neural network model is trained based on a preprocessed liver CT image dataset and uses the Dice Loss loss function to output a mask for the liver region.

[0053] Furthermore, the feature encoder uses ResNet-34 as its backbone network and contains four layers; each layer consists of multiple ECA-Res modules, which are composed of residual modules and efficient channel attention (ECA) modules connected sequentially.

[0054] The residual module includes a first convolutional layer, a batch normalization (BN) layer, and a ReLU activation function layer connected in sequence, as well as a short connection structure between the second convolutional layer, the batch normalization layer, and the identity mapping, which is used to realize cross-layer feature propagation and gradient preservation.

[0055] The efficient channel attention module obtains the interaction weights between channels by using one-dimensional convolution operations in the channel dimension, which can achieve efficient attention modeling without fully connected layers, thereby enhancing the model's ability to respond to key areas of liver lesions;

[0056] Specifically, each layer of the feature encoder consists of three parts: convolution + BN + ReLU, convolution + BN + downsampling + ECA, and max pooling, which are used to extract spatial features at different scales and suppress irrelevant background information.

[0057] The above structural design enables the model to take into account both global contextual information and local feature details during the downsampling stage, thereby improving the segmentation accuracy of low-contrast lesions.

[0058] Furthermore, the feature decoder comprises four modules, each consisting of a deconvolutional layer with a stride of 2, a convolutional layer with a kernel size of 3x3, a batch normalization (BN) layer, and a ReLU activation function layer; the end of the feature decoder consists of a convolutional layer with a kernel size of 1x1 and a Sigmoid activation function.

[0059] Furthermore, such as Figure 1 As shown, the overall structure of the neural network model includes an input layer, a feature encoder, a feature decoder, an output layer, and a loss calculation unit L located at the top layer of the decoding end, which are connected in sequence. The loss calculation unit L represents the loss value used to calculate the difference between the segmentation result and the true label, and is used to guide the network training.

[0060] The feature encoder consists of four sequentially connected ECA-Res modules, corresponding to the four main stages (layer 1 to layer 4) of the ResNet-34 network, processing feature maps of different resolutions respectively. Each ECA-Res module is composed of a residual structure and an efficient channel attention module connected sequentially, used for multi-scale feature extraction and channel weighting of liver CT images. Each ECA-Res module internally includes two 3×3 convolutional layers, a batch normalization (BN) layer, and a ReLU activation function. An efficient channel attention module is introduced after the second convolutional layer to perform channel attention weighting, and feature transfer across layers is achieved through identity mapping residual connections. Adjacent ECA-Res modules are downsampled through a max pooling layer with a stride of 2 (max pool 2×2).

[0061] The feature decoder includes four Double Conv modules. Each Double Conv module includes two 3×3 convolutional layers, a batch normalization layer, and a ReLU activation function, which are used to fuse and progressively upsample the features passed through skip connections. Adjacent Double Conv modules are upsampled through deconvolutional layers with a stride of 2 (up-conv 2×2) to restore spatial resolution.

[0062] The feature encoder and feature decoder are connected via multi-level skip connections. Each decoding layer receives feature information from the corresponding encoding layer and its next-level decoding layer, forming a densely nested multi-scale fusion path. This path is used to fully fuse shallow detail features with deep semantic features, thereby improving the accuracy of boundary recognition for the liver and lesion regions. In this example, the multi-scale feature fusion module is implemented through multi-level skip connections, that is, establishing a lateral connection channel between the feature encoder and the feature decoder, enabling feature-level fusion of each layer's encoded features with the corresponding decoding layer, thus achieving the interaction and synthesis of semantic information at different scales.

[0063] The output of the neural network model is set with a 1×1 convolutional layer and a Sigmoid activation function to generate a pixel-level segmentation mask for the liver and lesion areas.

[0064] The training method for the neural network model includes: after standardizing and data augmenting the liver CT images, the augmented samples are input into the Eres-UNet++ network, and the segmentation results are output through the encoding, skip connection and decoding processes. The Dice Loss is calculated through the loss function unit L to optimize the network parameters, thereby improving the segmentation performance and robustness.

[0065] Furthermore, the expression for the loss function is:

[0066]

[0067] In the formula, The set of pixels in the liver and lesion regions predicted by the model, that is, the set of pixels in the segmentation mask output by the neural network that are identified as the foreground (liver or lesion). It is a set of pixels representing the liver and lesion areas that are actually annotated, that is, the actual mask area annotated on the liver CT image by humans or medical experts. This indicates the pixel-level overlap between the predicted result and the actual annotation. and These represent the number of foreground pixels in the predicted mask and the real mask, respectively.

[0068] This loss function measures model performance by maximizing the overlap between the predicted result and the true labeled region. It can effectively address the pixel imbalance between the foreground (liver, lesion) and background in liver CT images, thereby improving the model's segmentation accuracy and stability.

[0069] Furthermore, the preprocessing includes data augmentation operations on the liver CT image dataset, including at least one of random rotation, horizontal or vertical flipping, affine transformation, brightness and contrast adjustment, and color saturation adjustment.

[0070] Furthermore, the samples in the dataset include liver CT images and their corresponding labels, which are pixel-level masks of the liver and lesion regions used for liver CT image segmentation.

[0071] Example 2:

[0072] This invention provides a liver cancer classification method based on a neural network model, such as... Figure 2 As shown, it includes:

[0073] S1. Input the preprocessed target liver CT image into the neural network model for liver CT image segmentation of the present invention to obtain a mask for the liver region;

[0074] S2. Input the mask of the liver region into the pre-tuned EfficientNet model through transfer learning to determine whether the target liver CT image belongs to HCC, non-HCC, or normal tissue.

[0075] Furthermore, the EfficientNet model includes variants such as EfficientNet-B0, EfficientNet-B1, EfficientNet-B2, EfficientNet-B3, or EfficientNet-B4, which have global average pooling layers, fully connected layers, and Softmax activation function layers added on top.

[0076] Furthermore, the specific methods for fine-tuning transfer learning include freezing the bottom convolutional layers of the EfficientNet model and only fine-tuning the top convolutional and fully connected layers; the loss function used during fine-tuning is the cross-entropy loss function, and the optimizer is Adam.

[0077] Example 3:

[0078] This example demonstrates the application of the neural network model and liver cancer classification method of this invention through simulation, in order to specifically illustrate the design process of this invention and verify its feasibility and inventiveness. Figure 3 As shown, the main steps are as follows:

[0079] 1) Data acquisition and preprocessing: Acquire liver CT image dataset: normal liver and liver CT images of liver with liver cancer; perform data preprocessing: image denoising, standardization, cropping, etc.

[0080] 2) Network structure design: A deep supervised network structure based on efficient channel attention and Res-UNet++ is designed, including a feature encoder, decoder and multi-scale feature fusion module, as well as key components such as efficient channel attention and residual connections, for liver CT image segmentation tasks.

[0081] 3) Network training and optimization: Train the designed network using the dataset, and optimize the network parameters by using loss functions such as cross-entropy loss function; use appropriate optimizers and learning rate scheduling strategies to ensure that the network converges stably during the training process.

[0082] 4) Image segmentation and result prediction: The trained network is used to segment the liver CT image and generate a mask for the liver region; the segmentation results are input into the Liver-EffNet model and the results are classified as: HCC, non-HCC, and normal.

[0083] 5) Experimental evaluation and results analysis: Evaluate the prediction results, including the calculation of indicators such as accuracy, recall, and F1 score; analyze the experimental results and compare the performance differences of different models.

[0084] 6) Results Presentation: The research findings will be visualized, including image segmentation results and liver cancer prediction results;

[0085] Furthermore, regarding the datasets, this example uses two public datasets. The first experimental dataset comes from the public liver tumor segmentation dataset LITS (Liver Tumor Segmentation), which contains 130 training samples and 70 test samples, manually labeled by four radiologists. To better address the issue of imbalanced data samples, images with small proportions of liver and liver tumors in the slices were removed. The training samples from the LITS dataset were then randomly divided into 80% training set, 10% validation set, and 10% test set. In the experiment, the number of filtered training samples was 10273. This example also uses the LIDC-IDRI lung nodule dataset collected by the National Cancer Institute to study early cancer detection and validate the model's performance. A total of 1018 cases were included. Each image was diagnosed and annotated by four physicians.

[0086] Data preprocessing and data augmentation are key tools for improving the training effect of deep learning. They can significantly improve the quality of the training sample set, enabling the model to more accurately fit the feature distribution of the training data. Data augmentation techniques can expand the diversity of training data, making it closer to real-world scenarios, thereby enhancing the model's generalization ability and effectively suppressing overfitting. Specific data preprocessing and data augmentation processes include, but are not limited to, translation, rotation, folding, and flipping. These operations help enrich the training dataset and improve the model's robustness and performance.

[0087] Since the LITS dataset consists of three-dimensional images, while the developed segmentation algorithm is two-dimensional, the data was first converted from three-dimensional to two-dimensional. Then, the data was enhanced through rotation, horizontal and vertical flipping, affine transformation, brightness contrast adjustment, and color saturation adjustment.

[0088] Rotation: The probability of using this operation is 0.5 for a random rotation of the input image.

[0089] Horizontal and vertical folding: For the input image, horizontal or vertical folding is performed randomly with a probability of 0.5.

[0090] Affine transformation: First, three points to be processed are randomly selected from the original image. Then, a random number is added to the x and y coordinates of these three points within the interval [-A, a] to obtain three points at new positions, where 'a' is the affine displacement factor. Affine transformation distorts the image and makes the data distribution more complex.

[0091] Brightness contrast and color saturation: For the input image, brightness and contrast are randomly adjusted to enhance the diversity of the focal area. Brightness contrast and color saturation significantly increase the contrast of the internal tissues of the liver area after treatment, making details more prominent.

[0092] Finally, the parameters in the network are trained, and the data-augmented experimental dataset is sent into the experimental system for training to observe the actual training situation.

[0093] In image segmentation networks using CNNs, there are two main types: FCN as the baseline and U-Net as the baseline. The UNet++ architecture is a network structure that extracts more multi-scale features by reconstructing skip connections, and then fuses them from the encoder to the decoder, integrating different sizes of U-Net structures into a new design.

[0094] The original structure of U-Net consists of two parts. The first part is feature extraction, similar to the VGG network. Each pooling layer has a scale. This part includes the original image, with a total of five scales; the second part is the upsampling part, upsampling once at a time; fusion is performed at the same scale as the number of channels corresponding to feature extraction. However, the fixed structure imposes two limitations on it:

[0095] The greater the network depth, the greater the computational workload, but the better the computational results are not necessarily.

[0096] Different datasets determine different optimal network depths.

[0097] To address the limitations of UNet, Zhou et al. proposed UNet++, one of its significant improvements being the introduction of different levels of U-Net coverage and grid skip connections between modules. By combining long and short connections, UNet++ can capture features at different levels and fuse them through feature stacking, thereby employing a shallower U-Net structure and reducing the differences in feature map scale during fusion.

[0098] UNet++ can extract features at all depth levels, and the network can learn the importance of features at different depths, making it well-suited for medical image segmentation where both detailed and global information need to be considered simultaneously. Furthermore, UNet++ shares a single feature extractor, requiring only one encoder to be trained, while different decoder paths reduce features at different levels. The flexible encoder configuration allows for the use of various modules instead. However, a problem with UNet++ is that the convolutional layers with 1x1 kernels obtained from the right-hand feature maps at each layer reduce the dimensionality of the features, thus diminishing their spatial information and hindering the acquisition of overall feature information.

[0099] The residual refers to the difference between the predicted value and the actual value. Typically, the feature extraction capability of a neural network is improved by increasing the number of layers, which means stacking multiple convolutional layers together.

[0100] However, as the number of network layers increases, problems such as vanishing and exploding gradients often arise, leading to a decline in network performance. To address these issues, ResNet introduces skip connections, allowing for good gradient flow while increasing network depth. It's important to note that ResNet employs a residual network structure, the basic idea of ​​which is to introduce residual blocks into the network, allowing for deeper layers while maintaining strong adaptability.

[0101] The basic residual block consists of two 3×3 convolutional layers, as shown in Figure 4(a). The residual block contains two paths: one is the new feature obtained by passing the feature map F(x) through the convolutional layer, and the other is the original feature of the input x. The outputs of these two paths are weighted and summed to obtain the final output of the residual block, which can be expressed by the following formula:

[0102]

[0103] Where x is the value output from the previous layer to the neuron, ω is the weight passed from x to the neuron, y is the output value obtained by the activation function of x, F(x¸ω) is the residual path, and x is the identity mapping. This is an element-wise addition.

[0104] Figure 4(b) shows a typical fully connected network. The difference lies in the fact that the ResNet network structure employs cross-layer connections, and residual mappings are used to replace the original feature maps.

[0105] The purpose of attention mechanisms is to discover relevant details and suppress irrelevant details. Among these, channel attention mechanisms have been shown by researchers to effectively improve the performance of convolutional neural networks. ECA, in particular, is a lightweight channel attention module that explicitly models the interdependencies between feature channels.

[0106] Its core is an extension of SENet (Squeeze-and-Excitation networks), combining two aspects of SENet. It replaces fully connected layers with one-dimensional convolutions, automatically learning the importance of each feature channel. Useful features are then promoted according to this importance level, while features less useful for the task at hand are suppressed. The SE module can be flexibly applied to existing network architectures. A schematic diagram of the module is shown below. Figure 5 As shown, it consists of two parts:

[0107] Global Mean Pooling (GAP) Generation The feature map; two fully connected layers generate the weights for each channel.

[0108] While dimensionality reduction in the SE module lowers complexity, dimensionality reduction between two fully connected layers is detrimental to weight learning. Therefore, this invention introduces an ECA module that considers each channel and its... Adjacent channels. Capture local cross-channel interactions. Its structure consists of three parts: GAP generation. Calculate the size of the feature map; calculate the size of the adaptive convolution kernel; apply the kernel size to the one-dimensional convolution to obtain the weights for each channel.

[0109] Among them, the channel dimension C and the convolution kernel The size is proportional, and the number of convolution kernels is set to 2. The formula for raising a power is as follows:

[0110] ;

[0111] The formula for convolution kernel size is as follows: ;

[0112] in, The value of γ represents an odd number close to t, with a value of 2 and a value of 1 for b. The ECA module involves fewer parameters, avoiding a reduction in feature dimension, increasing the interaction of information between channels, and effectively improving the performance of the model.

[0113] To leverage the powerful image representation capabilities of deep learning to improve the accuracy of liver and liver tumor segmentation, this invention proposes a liver and liver tumor segmentation network based on efficient channel attention and Res-UNet++, abbreviated as ResNet-34, using the autoencoder UNet++ as the basic architecture. The network structure is as follows: Figure 1 As shown, the ResNet-34 network is a 34-layer ResNet residual convolutional neural network, consisting of three parts: independent convolutional structures, convolutional residual structures, average pooling, and fully connected structures.

[0114] In independent convolutional structures, a size of [size missing] is used. The image is convolved with a 64-channel convolution kernel, and then... Max pooling is performed on the convolutional kernels. The convolutional residual structure consists of four different convolutional residual modules, each module's residual branch consisting of two convolutions, batch regularization, and a ReLU activation function. The residual module operations are performed 3, 4, 6, and 3 times for each convolutional residual structure, respectively.

[0115] During downsampling, the feature map is fed into different attention modules through max pooling and two consecutive operations to construct information for each channel or spatial channel, generate each weight value, and then perform batch normalization again. Finally, each channel of the original feature map is linearly weighted using its weights to pass it to the next layer of the network structure. Therefore, this model contains four attention modules. By introducing a channel attention mechanism, the network can autonomously learn channel weights, assigning greater weights to important image channels and extracting more critical image information.

[0116] The encoder is also known as the downsampling path for feature extraction. To address the vanishing gradient and network degradation issues caused by increasing depth, Eres-UNet++ uses ResNet34 to extract multi-scale features from the encoder portion. To further enhance useful features in the downsampling layers and suppress useless feature responses, an ECA-res model is constructed for each residual block in ResNet based on the Efficient Channel Attention (ECA) module. This section consists of four parts, each consisting of... Convolution + BN + ReLU Convolution + BN + Subsampling + ECA The maximum pool consists of...

[0117] This module utilizes an attention mechanism to assign weights to the input sequence and suppresses irrelevant background features. It ensures easier removal of background regions when encoding liver and liver tumor slices, effectively avoiding the introduction of global contextual information. Network performance may decrease with increasing network depth. Experimental results show that the ECA-res module improves overall network performance compared to the original ResNet34.

[0118] In feature extraction using deep learning network architectures, skip connections are used to fuse specific shallow features with abstract deep features. This invention employs hybrid skip connections for feature fusion, located between the encoder and decoder. Dashed lines in the network structure diagram represent skip connections on convolutional blocks. The proposed network, Eres-UNet++, reduces semantic gaps and is inspired by UNet++. By integrating long and short connections to aggregate local and global information, it effectively improves the accuracy of small object segmentation.

[0119] Feature reconstruction is performed using a decoder; this part consists of four modules, each consisting of... Composed of deconvolution layers with a step size of 2, Convolutional layer + BN layer + ReLU layer, finally utilizing Convolutional layers and sigmoid activation functions alter the channels to obtain the final image. Furthermore, the main goal of the decoder is to utilize high-level features to recover the normality of the feature map as much as possible while preserving spatial information.

[0120] The loss function, also known as the cost function, evaluates the degree of difference between the model's predictions and the actual values. The training process is the process of minimizing the loss function. The smaller the loss function, the closer the model's predictions are to the true values, indicating better model robustness.

[0121] Given that medical images often suffer from sample imbalance, this invention selects Dice Loss as the loss function. Dice Loss is derived from the Dice coefficients, and the relationship between them is shown below: ;

[0122] The formula for DiceLoss is as follows: ;in, express and The intersection of sets, where X and Y represent the number of elements.

[0123] Semantic segmentation networks have many commonly used evaluation metrics, among which confusion matrix is ​​a popular image evaluation method. It evaluates the prediction result by calculating the quantitative difference between the predicted image pixels and the manually labeled pixels. Confusion matrix allows for the calculation of evaluation metrics with fixed thresholds, using different evaluation functions to assess the model's ability. Specifically, True Positive (TP) means the label is target and the image is classified as target; True Negative (TN) means the label is background and the image is classified as background; False Positive (FP) means the label is background but the image is classified as target; and False Negative (FN) means the label is target but the image is classified as background.

[0124] Four metrics are specifically used to evaluate the overall performance of the model: Dice coefficient (DC), Accuracy (ACC), mean intersection ratio (MioU), frequency weighted intersection ratio (FWIoU), For liver and liver tumor segmentation by Precision, Recall, and F1-Score.

[0125] The dice coefficient is a set of similarity measurement functions, commonly used to calculate the similarity between two samples. The calculation formula is as follows: ;

[0126] Accuracy refers to the proportion of correctly predicted pixels out of all pixels. Furthermore, the higher the accuracy, the better the pixel classification performance. The calculation formula is as follows: ;

[0127] mIoU is an important metric for measuring image segmentation accuracy. It is calculated by determining the borrow value for each category, using the following formula: ; ;

[0128] FwIoU assigns weights based on the frequency of occurrence of each category, then multiplies these weights by the IoU of each category and sums them up. The calculation formula is as follows: ;

[0129] Precision is a measure of accuracy. It represents the proportion of pixels that are actually divided into regions of interest that are positive, and is calculated using the following formula: ;

[0130] Recall is the proportion of correctly segmented pixels out of all samples; these pixels are actually positive examples. It is also known as the true positive rate, and is calculated using the following formula: ;

[0131] The F1-Score is the harmonic mean of precision and recall. Its value ranges from [0,1]. A value closer to 1 indicates better segmentation. The calculation formula is as follows: ;

[0132] To verify the effectiveness of the proposed Eres-UNet++ network in liver segmentation, ablation experiments were conducted to compare the various modules of U-Net and this network, and four metrics were used for evaluation: Accuracy, IoU, F1-Score, and Recall. Here, Accuracy is used as an example. Figure 6 As shown in the figure, the comparison reveals that the segmentation performance of the method proposed in this invention is superior to other methods, with an accuracy improvement of 0.63% compared to UNet++.

[0133] Building upon this, a novel liver cancer category predictor based on EfficientNet architecture and transfer learning, called Liver-EffNet, is introduced to classify liver images, as detailed below:

[0134] (1) Introduction to the EfficientNet model: EfficientNet is an efficient convolutional neural network architecture that improves model performance through compound scaling of the network's depth, width, and resolution. The basic structure of this network includes the following key points:

[0135] ① Composite scaling: Composite scaling aims to balance the depth of the network. ,width The concept involves using three dimensions—scale, resolution, and r—to design more efficient and high-performing convolutional neural networks. The core of this concept is achieving balance by applying the same scaling ratio across all three dimensions of the network. ; , .in, These represent the weights for network depth, width, and resolution, respectively, and are solved using constrained optimal parameters. This represents hyperparameters.

[0136] ②MBConv Block: EfficientNet uses a type of block called Mobile Inverted BottleneckConvolution (MBConv), such as... Figure 7 As shown, this is a lightweight convolutional block that contains depthwise separable convolutions and dilated convolutions, making the model lightweight.

[0137] ③ Global average pooling: At the top of the model, there is usually a global average pooling layer that aggregates the average value of each channel of the feature map into a scalar value.

[0138] ④ Fully connected layers and Softmax: Finally, one or more fully connected layers are used to map the final features to the output class and obtain the classification probability through the Softmax function. ;in Let C be the output value of the i-th node, and C be the number of output nodes, i.e. the number of categories.

[0139] (2) Using EfficientNet for transfer learning

[0140] Variant Selection: Since EfficientNetB5-EfficientNetB7 are larger variants of EfficientNet with deeper networks and more parameters, the models may overfit the training set. Therefore, EfficientNetB0-B4 is used as the backbone for liver image classification, and transfer learning is applied to it.

[0141] Transfer learning: Compared to traditional machine learning methods, convolutional neural networks (CNNs) can automatically extract low-level and high-level feature maps from the model's convolutional base layers, pooling layers, and batch normalization layers. The resulting one-dimensional feature vectors are then fed into fully connected layers for classification. However, one drawback of CNNs is that they require a large amount of data to train the model and need to avoid underfitting and overfitting. To overcome these problems, transfer learning techniques can be applied. Transfer learning trains the architecture on benchmark datasets such as ImageNet, acquiring prior knowledge that can then be transferred to liver image classification tasks.

[0142] Based on the above background, transfer learning is performed on five variants of the pre-trained EfficientNet (EfficientNet B0-B4). For example... Figure 8As shown, the model is first trained on the benchmark dataset ImageNet, and then transferred to the target dataset. Due to the differences between the source and target datasets, the model is fine-tuned on CT scan slices of the liver for each variant of the EfficientNet model to adapt to the target images.

[0143] (3) Fine-tuning EfficientNetB0-B4 to construct Liver-EffNet for classification

[0144] Due to differences between the source and target datasets, EfficientNetB0-B4 underwent explicit fine-tuning on CT scan slices of the liver to adapt to the target images. Further modifications were made by adding a top layer to the model's classification head; this fine-tuned EfficientNetB0-B4 is referred to as Liver-EffNet. Feature maps were then extracted and passed to fully connected layers for classification. The network architecture of EfficientNetB1 is as follows... Figure 9 As shown.

[0145] Load the pre-trained model: Use the EfficientNetB0-B4 model that has been pre-trained on the large-scale image dataset ImageNet.

[0146] Constructing the top layer for the image classification task: A top layer for image classification is added to EfficientNetB0-B4. This top layer includes a global average pooling layer, a fully connected layer, and a softmax layer. The output size of the fully connected layer at the top matches the number of classes in the liver cancer classification task. Since the task is to classify liver cancer images into HCC, non-HCC, and normal classes, the task has 3 classes, hence the output size is 3.

[0147] Choose the layers to fine-tune: the bottom convolutional layers can preserve low-level features, while the top convolutional and fully connected layers need to be fine-tuned according to the number of classifications in the liver image.

[0148] Define the loss function: For the liver image classification problem, use the cross-entropy loss function: ;in, It is the one-hot encoding of the real label. It is the predicted probability of the model.

[0149] The hyperparameter settings after fine-tuning the model are shown in the table below:

[0150] Table 1 Liver-EffNet Hyperparameter Settings

[0151]

[0152] (4) Model performance evaluation measures and results analysis

[0153] The dataset consists of 1097 CT scan images from 110 patients: 120 non-HCC, 561 HCC, and the remaining 416 normal. The results will be evaluated using accuracy, F1 score, precision, recall, ROC curves, and confusion matrix.

[0154] The overall accuracy, F1 score, precision, and recall of this model are shown in the table below:

[0155] Table 2 Detailed Results of the Liver-EffNet Model

[0156]

[0157] As shown in the table above, the proposed model Liver-EffNet achieves an accuracy of 98.64% and a precision of 98.63%.

[0158] The accuracy, F1 score, precision, and recall of each variant in this model are shown in the table below:

[0159] Table 3 Detailed results of various variants of the Liver-EffNet model

[0160]

[0161] As shown in the table above, EfficientNet-B1 performs best among the Liver-EffNet models.

[0162] Confusion matrix: A confusion matrix is ​​a summary of prediction results and helps measure different evaluation metrics, such as accuracy, recall, and precision. It summarizes the overall performance of the model by breaking down the number of correct and incorrect predictions for each class.

[0163] The training effect of the Liver-EffNet model is as follows: Figures 10-11 As shown, with the increase of training times, the training accuracy and validation accuracy of the Liver-EffNet model gradually increase, and the difference between the two is very small during training, indicating that the Liver-EffNet model has good generalization ability.

[0164] The Liver-EffNet model was evaluated using the above experimental evaluation measures. Taking EfficientNetB1 as an example, the performance of Liver-EffNet based on EfficientNetB1 was compared with that of existing methods on the IQ-OTH / NCCD dataset.

[0165] Table 4. Performance comparison of Liver-EffNet with existing methods on the IQ-OTH / NCCD dataset after data augmentation.

[0166]

[0167] Liver-EffNet has demonstrated superior performance in liver cancer classification tasks compared to other existing models. One reason for this superior performance is its use of a combination of convolutional and pooling layers to extract features from images, with each layer building upon the features extracted in the previous layer. This allows the model to learn complex representations of images, which is crucial for accurate classification. The proposed architecture has been shown to be more efficient and versatile than traditional ML algorithms (such as SVM and RF) and other DL-based CNN architectures (such as VGG16, ResNet50, and DenseNet121).

[0168] To evaluate the generality and robustness of Liver-EffNet for liver cancer classification, this invention conducts an ablation study to investigate the impact of various factors on model performance and to gain a deeper understanding of its behavior under different conditions. This includes analyses of computational complexity, the impact of different data splits, the effect of the presence or absence of transfer learning on the model, and the training of proposed models for different liver cancer classifications.

[0169] Computational Complexity: A comprehensive analysis of the computational complexity of the proposed fine-tuned EfficientNetB0-B4 was conducted. The evaluation considered various factors, including the number of network parameters, floating-point operations, network size, training time, inference time, and test accuracy.

[0170] Table 5. EfficientNets variants and other deep learning architectures fine-tuned with data augmentation.

[0171] The computational cost of training the same dataset with the same hyperparameter settings

[0172]

[0173] Based on the computational costs of EfficientNet B0-B4 and other architectures, it was found that EfficientNetB0 has a relatively simple structure, consisting of 40,100 parameters. However, as we move to higher-order models, such as EfficientNetB1, the number of parameters increases to 65,100. Furthermore, EfficientNetB2-B4 exhibit even greater complexity, with 77,000, 107,000, and 175,500 parameters, respectively. Moreover, the depth of the model, especially the number of convolutional layers, affects the number of parameters in a deep neural network architecture, making the architecture more complex.

[0174] The impact of different data splits: Using data augmentation techniques, the performance of the model is evaluated using training and test sets with different ratios. The data is split into 80:20, 70:30, and 90:10 ratios.

[0175] Table 6. Exploring the impact of 70:30 data split on model performance

[0176]

[0177] Table 7. Exploring the impact of 80:20 data split on model performance

[0178]

[0179] Table 8. Exploring the impact of 90:10 data split on model performance

[0180] The performance impact of different data splits on Liver-EffNet is compared in the following figure. Figure 12 As shown, the model's performance is affected by the data split, indicating that the size of the training set plays a crucial role in the effectiveness of transfer learning methods. By allocating a large portion (80%) of the dataset to the training set, providing the model with ample data for learning and generalization helps prevent overfitting.

[0181] Comparison of models with and without transfer learning: In ablation studies, experiments are conducted to analyze the impact of using transfer learning on specific types of datasets. Setting the ImageNet weights to None when training the pre-trained model represents using only the pre-trained model architecture.

[0182] Table 9. Impact of non-transfer learning on model performance

[0183]

[0184] Table 10. Impact of Transfer Learning on Model Performance

[0185]

[0186] Without transfer learning, the following observations were made: ① The model tends to overfit during training and testing. ② The model favors non-HCC classes because it includes more images in the test set. ③ The model cannot distinguish between HCC and normal classes, therefore its generalization performance for HCC classes is poor.

[0187] Therefore, transfer learning has been shown to improve the generalization performance of models. By learning from natural images, these models can capture robust and transferable features applicable to medical imaging tasks, thereby improving the performance and accuracy of liver image classification. Furthermore, training deep neural networks from scratch on large medical imaging datasets can be computationally expensive. Combining transfer learning with EfficientNet can significantly reduce the required training time and computational resources, allowing for more efficient iteration and experimentation with different architectures and hyperparameters.

[0188] Further research: To further analyze and highlight the robustness of the proposed fine-tuned EfficientNetB1, additional experiments were conducted to classify different types of liver cancer from CT scan images. This dataset consists of 1000 CT scan images of three cancer types: hepatocellular carcinoma (HCC), intrahepatic cholangiocarcinoma (ICC), and mixed hepatocellular carcinoma-cholangiocarcinoma (cHCC-CCA). The dataset is divided into training, testing, and validation sets.

[0189] Table 11 Dataset Partition Results

[0190]

[0191] The normal class, hepatocellular carcinoma (HCC), intrahepatic cholangiocarcinoma (ICC), and mixed hepatocellular carcinoma-cholangiocarcinoma (cHCC-CCA) classes were coded as 0, 1, 2, and 3, respectively. After the experiment, the liver cancer image subdivision results are as follows:

[0192] Table 12. Results of Liver Cancer Image Subdivision

[0193]

[0194] Using the EfficientNetB1 variant of the Liver-EffNet model for segmentation of liver cancer images, the overall accuracy on the training and test sets reached 99.00% and 94.00%, respectively. The proposed model demonstrates good performance in multi-class classification of liver cancer types and significant performance on all evaluation metrics. Due to its complex nature, the proposed model achieved relatively low recall and test precision on the ICC.

[0195] This invention demonstrates significant advantages in liver CT image segmentation and hepatocellular carcinoma (HCC) classification tasks. The improved Eres-UNet++ network outperforms traditional U-Net and its variants in both segmentation accuracy and boundary preservation, effectively addressing the spatial information loss problem caused by deep convolution. Simultaneously, the EfficientNet classifier, fine-tuned using transfer learning, exhibits higher accuracy and robustness in distinguishing between HCC, non-HCC, and normal tissues. Experimental results show that this invention surpasses comparative methods in key metrics such as Dice coefficient and IoU, and achieves superior performance in training efficiency and model stability, significantly reducing the risk of overfitting and improving the consistency and reliability of diagnostic results. Therefore, this invention is particularly suitable for medical image analysis scenarios with high requirements for segmentation accuracy, computational efficiency, and clinical applicability.

Claims

1. A neural network model for liver CT image segmentation, characterized in that, The neural network model adopts a UNet++ model having a feature encoder for down-sampling feature extraction on a liver CT image, a feature decoder for up-sampling to restore a spatial dimension on the features extracted by the feature encoder, and a multi-scale feature fusion module connected between the feature encoder and the feature decoder and used for aggregating different level feature information; The feature encoder is integrated with an efficient channel attention ECA module for strengthening attention on a key region of a liver lesion and suppressing irrelevant background tissue information and a residual module for maintaining network gradient flow; The neural network model is trained based on a pre-processed liver CT image dataset and adopts a Dice Loss loss function to output a mask of a liver region.

2. The neural network model for liver CT image segmentation according to claim 1, wherein, The feature encoder takes ResNet-34 as a backbone network and includes four levels; each level is composed of a plurality of ECA-Res modules, and each ECA-Res module is composed of a residual module and an efficient channel attention module connected in sequence. The residual module includes a first convolutional layer, a batch normalization layer and a ReLU activation function layer connected in sequence, and a second convolutional layer, a batch normalization layer and a short connection structure of an identity mapping, for realizing cross-layer transmission of features and gradient preservation. The efficient channel attention module obtains the interaction weight between channels by using one-dimensional convolution operation in the channel dimension, thereby enhancing the response capability of the model to the key region of the liver lesion.

3. The neural network model for liver CT image segmentation according to claim 2, wherein, The feature decoder includes four modules, each of which is composed of a deconvolution layer with a step of 2, a convolutional layer with a kernel size of 3x3, a batch normalization layer and a ReLU activation function layer in sequence; and the end of the feature decoder is a convolutional layer with a kernel size of 1x1 and a Sigmoid activation function.

4. The neural network model for liver CT image segmentation according to claim 3, wherein The overall structure of the neural network model includes an input layer, a feature encoder, a feature decoder, an output layer and a loss calculation unit L located at the top layer of the decoding end connected in sequence; The feature encoder is composed of four ECA-Res modules connected in sequence, corresponding to four main stages of the ResNet-34 network; each ECA-Res module is composed of a residual structure and an efficient channel attention module connected in sequence; the internal part of each ECA-Res module includes two 3x3 convolutional layers, a batch normalization layer and a ReLU activation function, and an efficient channel attention module is introduced after the second convolutional layer for channel attention weighting, and cross-layer feature transmission is realized through identity mapping residual connection; adjacent ECA-Res modules are down-sampled through a maximum pooling layer with a step of 2; The feature decoder includes four Double Conv modules, each of which includes two 3x3 convolutional layers, a batch normalization layer and a ReLU activation function, for fusing and gradually up-sampling the features transmitted through the jump connection; adjacent Double Conv modules are up-sampled through a deconvolution layer with a step of 2 to restore the spatial resolution. The feature encoder and the feature decoder are connected through multi-stage skip connections, each decoding layer receives feature information from the corresponding encoding layer and the next decoding layer, forming a densely nested multi-scale fusion path; The output end of the neural network model is provided with a convolution layer with a convolution kernel size of 1*1 and a Sigmoid activation function, which is used to generate a pixel-level segmentation mask of the liver and lesion regions.

5. The neural network model for liver CT image segmentation according to claim 1, wherein, The expression of the loss function is: In the formula, is the pixel set of the liver and lesion region predicted by the model; is the pixel set of the liver and lesion region labeled in reality; represents the overlapping region of the prediction result and the real label at the pixel level; and respectively represent the number of foreground pixels of the predicted mask and the real mask.

6. The neural network model for liver CT image segmentation according to claim 1, wherein, The preprocessing includes a data enhancement operation on the liver CT image dataset, and the data enhancement operation includes at least one of random rotation, horizontal or vertical flipping, affine transformation, brightness contrast adjustment, and color saturation adjustment.

7. The neural network model for liver CT image segmentation according to claim 1, wherein, The samples in the dataset include liver CT images and corresponding labels, and the labels are pixel-level masks of liver and lesion regions for liver CT image segmentation. 8.A liver cancer classification method based on a neural network model, characterized in that, Comprise: inputting the preprocessed target liver CT image into the neural network model for liver CT image segmentation according to any one of claims 1-7 to obtain a liver region mask; inputting the liver region mask into the pre-migration learning fine-tuned EfficientNet model to determine whether the target liver CT image belongs to HCC, non-HCC or normal tissue. 9.The liver cancer classification method based on the neural network model according to claim 8, characterized in that, The EfficientNet model is an EfficientNet-B0, EfficientNet-B1, EfficientNet-B2, EfficientNet-B3 or EfficientNet-B4 variant model with a global average pooling layer, a fully connected layer and a Softmax activation function layer added at the top. 10.The liver cancer classification method based on the neural network model according to claim 8, characterized in that, The specific method of the transfer learning fine-tuning includes freezing the convolution layers at the bottom of the EfficientNet model, and only fine-tuning the convolution layers and the fully connected layers at the top; the loss function used during fine-tuning is a cross-entropy loss function, and the optimizer is Adam.

Images