Medical image segmentation method based on lightweight encoder-decoder

Abstract

The invention discloses a medical image segmentation method based on a lightweight encoder-decoder, and relates to the field of image segmentation, and the method comprises the steps: obtaining a to-be-segmented image from a disclosed medical image data set, carrying out the preprocessing of the data set, and dividing the data set into a training set and a test set; the method comprises the following steps: constructing a lightweight encoding-decoding network, wherein the lightweight encoding-decoding network comprises a Ghost bottleneck encoder and an MIViT decoder; inputting the image data in the training set into a Ghost bottleneck encoder to obtain a feature map, inputting the feature map into an MIViT decoder to fuse local and global information, and outputting a prediction segmentation image result by using a Sigmoid activation function; training and optimizing the constructed network model; quantifying the trained network model, and deploying the network model on an edge device; and inputting the image data in the test set into the edge device to obtain a segmentation result. According to the method, medical image analysis can be efficiently carried out on equipment with limited computing resources, and the computing burden and the storage requirement are reduced while relatively high precision is ensured.

Description

Technical Field

[0001] The present invention relates to the technical field of image segmentation, and in particular to a medical image segmentation method based on a lightweight encoder-decoder. Background Art

[0002] With the widespread application of deep learning technology in medical image analysis, deep learning-based medical image segmentation models have become the core tools of many clinical diagnosis and treatment programs. However, deploying these deep learning models on resource-constrained devices, especially in hardware environments such as mobile devices, embedded systems, and edge computing devices, still faces many challenges. These devices are usually limited in computing power, memory, and storage space, so it is necessary to design efficient models with fewer parameters and low floating-point operations to ensure real-time and accuracy. Current solutions usually rely on lightweight convolutional neural network (CNN) architectures. These architectures reduce the computational requirements of the model while ensuring high accuracy by reducing the number of parameters and computational complexity to meet the needs of resource-constrained environments. However, traditional CNN layers, especially convolution and pooling operations, usually have strong spatial inductive biases, which makes them inherently limited in capturing global context information. Although convolutional neural networks can effectively extract local features, when faced with large-scale complex structures or lesion areas, traditional CNN methods may not be able to fully understand the global context of the image, resulting in reduced segmentation accuracy. Therefore, when performing medical image segmentation on resource-constrained devices, how to keep the model lightweight while improving its ability to capture global contextual information has become a major challenge in current research. Summary of the invention

[0003] The purpose of the present invention is to overcome the deficiencies of the prior art and provide a medical image segmentation method based on a lightweight encoder-decoder to address the challenges of running medical image segmentation tasks on low-resource devices.

[0004] The objective of the present invention is achieved through the following technical solutions:

[0005] A medical image segmentation method based on a lightweight encoder-decoder, comprising:

[0006] Step 1: Obtain the image to be segmented from a public medical image dataset, preprocess each image in the dataset, and divide the preprocessed dataset into a training set and a test set;

[0007] Step 2: Build a lightweight encoder-decoder network, including a Ghost bottleneck encoder and a MIViT decoder. The Ghost bottleneck encoder includes a standard convolutional layer and a Ghost convolutional layer, which are used to extract feature maps of the image, and the MIViT decoder includes a MobileViT module and an Involution module, which are used to capture global information and fuse it with local information.

[0008] Step 3: Input the image data in the training set into the Ghost bottleneck encoder to obtain a feature map, input the feature map into the MIViT decoder to fuse local and global information, and use the Sigmoid activation function to output the predicted segmentation image result;

[0009] Step 4: Train and optimize the constructed lightweight encoder-decoder network model;

[0010] Step 5: quantize the trained lightweight encoder-decoder network model and deploy the quantized model on the edge device;

[0011] Step 6: Input the image data in the test set to the edge device deployed with a lightweight encoder-decoder network model to obtain the medical image segmentation results.

[0012] Furthermore, the step 1 specifically includes:

[0013] Obtain the original images to be segmented from the brain tumor segmentation dataset BraTS, the lung nodule CT image dataset LUNA16, and the skin lesion segmentation dataset ISIC;

[0014] Normalize the original image, that is, scale the pixel values ​​of the image to the [0,1] interval. The formula is as follows: Among them, p orig is the original pixel value, p norm is the normalized pixel value;

[0015] Resize all images to 256*256;

[0016] The normalized and resized image dataset is divided into a training set and a test set;

[0017] Some original images are selected from the training set and rotated at random angles within the range of [-30°, 30°]. Another part of the images is randomly cropped and an area is added to the training set after rotation and cropping.

[0018] Furthermore, the image data in the training set is input into the Ghost bottleneck encoder to obtain a feature map, which specifically includes:

[0019] Input image data To the standard convolution layer, W is the image width, H is the image height, the standard convolution kernel size is K*K, and the standard convolution operation is completed using the following formula: in, is the output feature map, W out is the width of the output feature map, H out is the height, C in and C out Represents the number of input and output channels respectively;

[0020] Input image data I to the Ghost convolution layer to generate a low-dimensional feature map, and use the following formula to complete the Ghost convolution operation:

[0021] Merge the feature maps obtained from the standard convolution layer and the Ghost convolution layer:

[0022] Furthermore, the MobileViT module is a lightweight Transformer module, which is used to capture global information through the self-attention mechanism while retaining the convolution structure to process local information. The core formula is as follows:

[0023]

[0024] Among them, d k is the dimension of the key; Q, K, V represent the query, key, and value matrices, respectively, which are obtained by applying three different linear transformations to the input feature map, as follows:

[0025] Q=I out *W1

[0026] K=I out *W2

[0027] V=I out *W3

[0028] Among them, W1, W2 and W3 are weight matrices used to generate queries, keys and values ​​respectively;

[0029] The Involution module is used to enhance local features by performing spatial convolution operations on the input feature map. The formula is as follows:

[0030]

[0031] Among them, f(·,·) represents the reverse convolution operation, x i,j is the input feature at position (i, j), and w is the convolution kernel.

[0032] Furthermore, in step 4, the Dice coefficient loss is used to evaluate the segmentation accuracy when training the lightweight encoder-decoder network model. The formula is as follows:

[0033]

[0034] Among them, p i is the predicted value, t i is the true value.

[0035] Furthermore, in step 4, the Adam optimizer is used to optimize the lightweight encoder-decoder network model, and the learning rate of each parameter is adaptively adjusted to obtain the optimized segmentation network model.

[0036] Furthermore, in step 5, the trained and optimized model is quantized using the INT8 quantization method.

[0037] The beneficial effects of the present invention are:

[0038] 1) The lightweight encoder-decoder network model provided by the present invention can effectively capture the global context information in the image by combining the advantages of convolutional neural networks (CNN) and visual transformers (ViT). This design not only enhances the model's ability to understand complex structures and lesion areas, but also maintains the advantages of low parameters and low computational complexity. In addition, by introducing an efficient feature generation strategy, the network model significantly reduces the amount of computation and storage requirements while ensuring the model's expressiveness. Through this mechanism, the model can retain important feature information with fewer parameters, thereby improving computational efficiency.

[0039] 2) Since the lightweight encoder-decoder network model can adapt to hardware platforms with limited computing resources such as edge devices or embedded systems, the medical image segmentation method based on a lightweight encoder-decoder provided by the present invention can efficiently perform medical image analysis on devices with limited computing resources, and significantly reduce the computing burden and storage requirements while ensuring high accuracy. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] Figure 1 A flowchart of a medical image segmentation method based on a lightweight encoder-decoder provided in an embodiment of the present invention;

[0041] Figure 2 A model structure diagram of a lightweight encoder-decoder network provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0042] The technical solution of the present invention will be clearly and completely described below in conjunction with the embodiments. Obviously, the described embodiments are only part of the embodiments of the present invention, rather than all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of the present invention.

[0043] See also Figure 1-Figure 2 , the present invention provides a technical solution:

[0044] A medical image segmentation method based on a lightweight encoder-decoder, such as Figure 1 As shown, including:

[0045] Step 1: Obtain the image to be segmented from a public medical image dataset, preprocess each image in the dataset, and divide the preprocessed dataset into a training set and a test set;

[0046] In this embodiment, the step 1 specifically includes:

[0047] Obtain the original images to be segmented from the brain tumor segmentation dataset BraTS, the lung nodule CT image dataset LUNA16, and the skin lesion segmentation dataset ISIC;

[0048] Normalize the original image, that is, scale the pixel values ​​of the image to the [0,1] interval. The formula is as follows: Among them, p orig is the original pixel value, p norm is the normalized pixel value;

[0049] Resize all images to 256*256;

[0050] The normalized and resized image dataset is divided into a training set and a test set;

[0051] Some original images are selected from the training set and rotated at random angles within the range of [-30°, 30°]. Another part of the images is randomly cropped and an area is added to the training set after rotation and cropping.

[0052] The present invention normalizes the images in the image preprocessing stage to avoid the problem of unstable model training due to different pixel value ranges, and adjusts all images to a uniform size so that they can be input into the constructed neural network model in batches. Finally, in order to increase the diversity of data, thereby improving the generalization ability of the model and preventing overfitting, some random and slight changes are made to the image data in the training set to obtain more enhanced images to expand the number of training sets.

[0053] Step 2: Build a lightweight encoder-decoder network, including the Ghost bottleneck encoder and MIViT decoder, such as Figure 2 As shown, the Ghost bottleneck encoder includes a standard convolution layer and a Ghost convolution layer, which are used to extract feature maps of the image, and the MIViT decoder includes a MobileViT module and an Involution module, which are used to capture global information and fuse it with local information.

[0054] Step 3: Input the image data in the training set into the Ghost bottleneck encoder to obtain a feature map, input the feature map into the MIViT decoder to fuse local and global information, and use the Sigmoid activation function to output the predicted segmentation image result;

[0055] In this embodiment, the image data in the training set is input into the Ghost bottleneck encoder to obtain a feature map, which specifically includes:

[0056] Input image data To the standard convolution layer, W is the image width, H is the image height, the standard convolution kernel size is K*K, and the standard convolution operation is completed using the following formula: in, is the output feature map, W out is the width of the output feature map, H out is the height, C in and C out Represents the number of input and output channels respectively;

[0057] Input image data I to the Ghost convolution layer to generate a low-dimensional feature map, and use the following formula to complete the Ghost convolution operation:

[0058] Merge the feature maps obtained from the standard convolution layer and the Ghost convolution layer:

[0059] The Ghost bottleneck encoder generates “ghost features” through lightweight convolution, generates multiple low-dimensional feature maps through less calculation, and efficiently fuses them in the subsequent processing stage, reducing the computational complexity.

[0060] In this embodiment, the MIViT decoder includes a MobileViT module and an Involution module; the MobileViT module is a lightweight Transformer module, which is used to capture global information through a self-attention mechanism while retaining a convolution structure to process local information. The core formula is as follows:

[0061]

[0062] Among them, d k is the dimension of the key; Q, K, V represent the query, key, and value matrices, respectively, which are obtained by applying three different linear transformations to the input feature map, as follows:

[0063] Q=I out *W1

[0064] K=I out *W2

[0065] V=I out *W3

[0066] Among them, W1, W2 and W3 are weight matrices used to generate queries, keys and values ​​respectively. These matrices are also part of the model and will be learned and updated in the subsequent training process;

[0067] The Involution module is used to enhance local features by performing spatial convolution operations on the input feature map. The formula is as follows:

[0068]

[0069] Among them, f(·,·) represents the reverse convolution operation, x i,j is the input feature at position (i, j), and w is the convolution kernel.

[0070] The MIViT decoder combines the MobileViT and Involution modules to process the fusion of global context information and local features. The Involution module uses a deconvolution operation to reduce the computational complexity.

[0071] Step 4: Train and optimize the constructed lightweight encoder-decoder network model;

[0072] Furthermore, in step 4, the Dice coefficient loss is used to evaluate the segmentation accuracy when training the lightweight encoder-decoder network model. The formula is as follows:

[0073]

[0074] Among them, p i is the predicted value, t i is the true value.

[0075] In step 4, the Adam optimizer is used to optimize the lightweight encoder-decoder network model, and the learning rate of each parameter is adaptively adjusted to obtain the optimized segmentation network model.

[0076] Step 5: Quantize the trained lightweight encoder-decoder network model and deploy the quantized model on the edge device. Specifically, use the INT8 quantization method to quantize the trained and optimized model. The quantized model can be inferred on the edge device with lower computing resources.

[0077] Step 6: Input the image data in the test set to the edge device deployed with a lightweight encoder-decoder network model to obtain the medical image segmentation results.

[0078] The cropped 256*256 size images in the test set are used as the input of the network model. Feature extraction is performed through the Ghost bottleneck module. Then the MIViT module is used to fuse local and global information, gradually restore the details of the segmented image, and finally the Sigmoid activation function is used to output the segmentation result. The output image size is consistent with the input image.

[0079] When deploying a model, you need to save the model first, and then optimize it based on hardware resources, computing power, and system requirements, such as using quantization, pruning, and other technologies to accelerate reasoning, and provide services through APIs. Continuous monitoring and version management ensure system stability and updates, while also paying attention to data security and privacy protection to ensure that the model runs efficiently, reliably, and securely in the actual environment.

[0080] The present invention provides a lightweight encoder-decoder network model, which is designed to efficiently capture global context information while maintaining low parameter volume and computational complexity, and is particularly suitable for edge devices or embedded systems with limited computing resources. The model significantly reduces computing and storage overhead while ensuring expressiveness by introducing an efficient feature generation strategy. This strategy enables the model to retain key information under the condition of fewer parameters, thereby improving computational efficiency. Furthermore, the model combines the advantages of convolutional neural networks (CNNs) and visual transformers (ViTs), and can effectively capture the global context features of images, thereby improving segmentation accuracy. This design not only enhances the recognition ability of complex structures and lesion areas, but also maintains low parameters and low computational complexity, avoiding the problem of excessive consumption of computing resources in traditional methods. Therefore, the method can provide more accurate results in medical image segmentation tasks while avoiding the problem of excessive consumption of computing resources commonly seen in traditional methods.

[0081] The medical image segmentation method based on the lightweight encoder-decoder of the present invention can efficiently perform medical image analysis on devices with limited computing resources, and significantly reduce the computing burden and storage requirements while ensuring high accuracy. First of all, the invention can help more medical institutions achieve efficient and accurate medical image diagnosis. Especially in mobile medical devices, remote diagnosis and edge computing platforms, the application of this technology can reduce hardware costs and improve diagnostic efficiency, so that patients in primary care and remote areas can also benefit from advanced medical technology. In addition, the invention reduces the hardware cost of the equipment by reducing the demand for computing resources, while improving the segmentation speed and real-time performance while ensuring the accuracy of the model. This will help reduce the labor cost in medical image analysis, especially in large-scale medical data processing and medical record management, thereby bringing higher cost performance to medical institutions and patients.

[0082] Furthermore, the invention can effectively improve the accuracy and speed of medical image segmentation, especially when faced with complex structures and lesion areas. By improving the level of automated diagnosis, it can reduce the workload of doctors, speed up disease analysis and treatment decisions, and help doctors detect lesions earlier, so as to achieve early diagnosis and early treatment, thereby improving the treatment effect and survival rate of patients. While improving the ability to capture global contextual information and keeping the model lightweight, it will help promote technological innovation in the field of medical image processing. By reducing the computational complexity of the model, it will encourage more intelligent hardware devices to enter the medical field and promote the widespread application of medical artificial intelligence technology in clinical environments.

[0083] The above is only a preferred embodiment of the present invention. It should be understood that the present invention is not limited to the form disclosed herein, and should not be regarded as excluding other embodiments, but can be used in various other combinations, modifications and environments, and can be modified within the scope of the concept described herein through the above teachings or the technology or knowledge of the relevant field. The changes and modifications made by those skilled in the art shall not deviate from the spirit and scope of the present invention, and shall be within the scope of protection of the claims attached to the present invention.

Claims

1. A medical image segmentation method based on a lightweight encoder-decoder, characterized in that: include: Step 1: Obtain the image to be segmented from a public medical image dataset, preprocess each image in the dataset, and divide the preprocessed dataset into a training set and a test set; Step 2: Build a lightweight encoder-decoder network, including a Ghost bottleneck encoder and a MIViT decoder. The Ghost bottleneck encoder includes a standard convolutional layer and a Ghost convolutional layer, which are used to extract feature maps of the image, and the MIViT decoder includes a MobileViT module and an Involution module, which are used to capture global information and fuse it with local information. Step 3: Input the image data in the training set into the Ghost bottleneck encoder to obtain a feature map, input the feature map into the MIViT decoder to fuse local and global information, and use the Sigmoid activation function to output the predicted segmentation image result; Step 4: Train and optimize the constructed lightweight encoder-decoder network model; Step 5: quantize the trained lightweight encoder-decoder network model and deploy the quantized model on the edge device; Step 6: Input the image data in the test set to the edge device deployed with a lightweight encoder-decoder network model to obtain the medical image segmentation results.

2. The medical image segmentation method based on a lightweight encoder-decoder according to claim 1, characterized in that: The step 1 specifically includes: Obtain the original images to be segmented from the brain tumor segmentation dataset BraTS, the lung nodule CT image dataset LUNA16, and the skin lesion segmentation dataset ISIC; Normalize the original image, that is, scale the pixel values ​​of the image to the [0,1] interval. The formula is as follows: Among them, p orig is the original pixel value, p norm is the normalized pixel value; Resize all images to 256*256; The normalized and resized image dataset is divided into a training set and a test set; Some original images are selected from the training set and rotated at random angles within the range of [-30°, 30°]. Another part of the images is randomly cropped and an area is added to the training set after rotation and cropping.

3. The medical image segmentation method based on a lightweight encoder-decoder according to claim 1, characterized in that: The image data in the training set is input into the Ghost bottleneck encoder to obtain a feature map, which specifically includes: Input image data To the standard convolution layer, W is the image width, H is the image height, the standard convolution kernel size is K*K, and the standard convolution operation is completed using the following formula: in, is the output feature map, W out is the width of the output feature map, H out is the height, C in and C out Represents the number of input and output channels respectively; Input image data I to the Ghost convolution layer to generate a low-dimensional feature map, and use the following formula to complete the Ghost convolution operation: Merge the feature maps obtained from the standard convolution layer and the Ghost convolution layer:

4. The medical image segmentation method based on a lightweight encoder-decoder according to claim 1, characterized in that: The MobileViT module is a lightweight Transformer module that captures global information through a self-attention mechanism while retaining the convolutional structure to process local information. The core formula is as follows: Among them, d k is the dimension of the key; Q, K, V represent the query, key, and value matrices, respectively, which are obtained by applying three different linear transformations to the input feature map, as follows: Q=I out *W1 K=I out *W2 V=I out *W3 Among them, W1, W2 and W3 are weight matrices used to generate queries, keys and values ​​respectively; The Involution module is used to enhance local features by performing spatial convolution operations on the input feature map. The formula is as follows: Among them, f(·,·) represents the reverse convolution operation, x i,j is the input feature at position (i, j), and w is the convolution kernel.

5. The medical image segmentation method based on a lightweight encoder-decoder according to claim 1, characterized in that: In step 4, the Dice coefficient loss is used to evaluate the segmentation accuracy when training the lightweight encoder-decoder network model. The formula is as follows: Among them, p i is the predicted value, t i is the true value.

6. The medical image segmentation method based on a lightweight encoder-decoder according to claim 1, characterized in that: In step 4, the Adam optimizer is used to optimize the lightweight encoder-decoder network model, and the learning rate of each parameter is adaptively adjusted to obtain the optimized segmentation network model.

7. The medical image segmentation method based on a lightweight encoder-decoder according to claim 1, characterized in that: In step 5, the INT8 quantization method is used to quantize the trained and optimized model.

Images