Efficient medical image ROI region extraction method and system with lower parameters based on deep learning network
By proposing the EMIS-Net model in medical image segmentation, using feature extraction, information compression and efficient feature aggregation modules, the existing deep learning network parameters are solved, and efficient and accurate ROI area extraction is achieved, which is suitable for a variety of medical image segmentation tasks.
Patent Information
- Application Number
- CN202411791644.7
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-12-06
- Publication Date
- 2025-03-11
AI Technical Summary
The existing deep learning networks have problems with large amounts of parameters and high computational complexity in medical image segmentation, making it difficult to achieve efficient and accurate ROI area extraction.
A lightweight and efficient deep learning model EMIS-Net is proposed. Through feature extraction module LGFE, information compression module ICM and efficient feature aggregation module EFAM, the calculation ratio of the encoding and decoding stage is adjusted, and a lightweight module is introduced to improve segmentation accuracy and reduce model complexity.
The EMIS-Net model achieves excellent segmentation performance on multiple data sets, including Kvasir-SEG, ISIC2018 and GLaS data sets, with Dice coefficients reaching 90.0%, 90.4%, and 91.6%, and has low parameter quantity and calculation complexity, making it suitable for resource-constrained environments.
Smart Images
Figure CN119600277A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to a method and system for extracting ROI regions of medical images efficiently and with lower parameters based on a deep learning network, and belongs to the technical field of medical image processing. Background Art
[0002] In the field of computer-aided diagnosis, medical image segmentation technology is the key to improving diagnostic accuracy and efficiency. At present, many studies are devoted to improving the performance of segmentation algorithms, aiming to improve the model's ability to process medical images through deep learning methods. Taking the Kvasir-SEG dataset as an example, its images are derived from clinical colonoscopy, which is crucial for the diagnosis of early colon cancer. At present, the research on medical image segmentation by deep learning mainly adopts the network structure based on U-Net and its variants. Although these structures have achieved certain results in image segmentation, they still have problems such as large number of parameters and high computational complexity. Therefore, the present invention provides a medical image ROI region extraction method based on a deep learning network with high efficiency and lower parameters. Summary of the invention
[0003] In order to address the shortcomings of the existing methods, the present invention provides an efficient and lower-parameter medical image ROI region extraction method based on a deep learning network to address the problems of large parameters and high computational complexity of the existing deep learning network models, and proposes a lightweight and efficient deep learning model EMIS-Net, which effectively improves the segmentation accuracy and reduces the model complexity by adjusting the stage calculation ratio of encoding and decoding and introducing a lightweight module.
[0004] The technical solution of the present invention is: a medical image ROI region extraction method with high efficiency and lower parameters based on a deep learning network, the method comprising:
[0005] Step 1, obtain medical images and preprocess them;
[0006] Step 2: Input the preprocessed medical image into the constructed deep learning model, and use the feature extraction module (LGFE) in the deep learning model to extract the details and contour information in the medical image;
[0007] Step 3: Use the information compression module (ICM) to compress the information in the encoding process to reduce the information loss in the encoding and decoding process;
[0008] Step 4: Use the decoder to extract features of different scales from the compressed medical image;
[0009] Step 5: Fuse features of different scales through the efficient feature aggregation module (EFAM) to enhance the model’s understanding of details and context;
[0010] Step 6: Input the fused feature map into the convolution module (OutConv) for the final medical image segmentation processing to achieve accurate segmentation of the lesion area in the medical image.
[0011] Furthermore, in the Step 1, the original image of the medical image is cropped with the lesion as the center, and the cropped size is 256×256. The medical image is randomly flipped and rotated before being input into the deep learning model.
[0012] Furthermore, the feature extraction module in Step 2 includes two branches;
[0013] The Ghost module is introduced in the first branch mainly to solve the problems of redundant information and excessive model parameters caused by the convolution module. The Ghost module can increase the width and depth of the network without significantly increasing the number of parameters, so that the model can use limited computing resources more efficiently. After obtaining the output of the Ghost module, the CA attention mechanism can be combined to improve the positioning of the lesion, which can improve the accuracy of segmentation.
[0014] Introducing dilated convolution in the second branch can expand the receptive field without increasing memory and computational overhead to obtain richer contextual information;
[0015] The input of the first branch is processed sequentially through a depth-wise separable convolution (DWconv) with a convolution kernel size of 5×5, batch normalization, an activation function (GELU), a Ghost module, a CA block, and batch normalization to obtain the output of the first branch;
[0016] The input of the second branch is processed by a convolution with a convolution kernel size of 1×1 and a dilation rate of 7 to obtain the output of the second branch;
[0017] The output result of the first branch is added to the output result of the second branch (Addition), and then an activation function (GELU) is passed to obtain the final output feature.
[0018] Furthermore, in the Step 3, the information compression module (ICM) is mainly a narrow-wide-narrow structure, which increases the dimension of the feature map in the second convolution layer, which helps the network learn more complex features. The expanded feature map is compressed back to the original dimension in the third convolution layer, which helps the network focus on the most important features. At the same time, it can also reduce the amount of calculation and improve the efficiency and performance of the network.
[0019] Step 3 includes:
[0020] Perform the first convolution to extract features of the input information compression module;
[0021] The features extracted by the first convolution are convolved for the second time to 4 times the original number of channels before extracting the features;
[0022] The features extracted by the second convolution are convolved for the third time to reduce the number of channels back to the original number and then extract the features;
[0023] The results of the three convolutions are processed by the batch normalization layer, and the features of the original input information compression module are added. The result of the addition is activated by the activation function GELU to obtain the final output.
[0024] Furthermore, in Step 4, the structure of the decoder is the same as the feature extraction module in Step 2.
[0025] Furthermore, in the Step 5, the efficient feature aggregation module EFAM mainly uses a 1×1 convolution kernel and different dilation rates; the 1×1 convolution kernel has a small number of parameters and low computational complexity, and different dilation rates will not introduce additional parameter consumption and computational complexity while expanding the receptive field;
[0026] Step 5 includes:
[0027] Step 5.1, the input of the efficient feature aggregation module EFAM is processed by three parallel convolutional layers with different expansion rates and a convolution kernel size of 1×1, and the results are output respectively;
[0028] Step 5.2, the output results of the three branches are spliced to obtain the spliced result;
[0029] Step 5.3, input the spliced result into a convolution layer with a convolution kernel size of 1×1 to obtain the final output.
[0030] In the Step 6, the fused feature map is input into the convolution module (OutConv) for final image segmentation processing to achieve accurate segmentation of the lesion area in the medical image.
[0031] The present invention also provides a medical image ROI region extraction system based on a deep learning network that is efficient and has lower parameters, including: a module for executing the above-mentioned medical image ROI region extraction method based on a deep learning network that is efficient and has lower parameters.
[0032] The present invention also provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, the above-mentioned medical image ROI region extraction method with high efficiency and lower parameters based on a deep learning network is implemented.
[0033] The present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that when the computer program is executed by a processor, the above-mentioned medical image ROI region extraction method with high efficiency and lower parameters based on a deep learning network is implemented.
[0034] The present invention also provides a computer program product, including a computer program, characterized in that when the computer program is executed by a processor, it implements the above-mentioned medical image ROI area extraction method with high efficiency and lower parameters based on a deep learning network.
[0035] The beneficial effects of the present invention are:
[0036] 1. This paper proposes a lightweight and efficient deep learning model EMIS-Net for medical image segmentation. By combining the feature extraction module LGFE, the information compression module ICM and the efficient feature aggregation module EFAM, the segmentation accuracy is effectively improved, the model complexity is reduced, and the computational efficiency is improved;
[0037] 2. The EMIS-Net model in the present invention extracts details and contour information from the image through the feature extraction module LGFE, reduces the loss of information in the encoding and decoding process through the information compression module ICM, and makes full use of the extracted feature information through the efficient feature aggregation module EFAM, thereby achieving excellent segmentation performance on multiple data sets, including Kvasir-SEG, ISIC2018 and GLaS data sets;
[0038] 3. The EMIS-Net model of the present invention reduces the number of model parameters and computational complexity by optimizing the network structure and computational ratio, making the model more suitable for resource-constrained environments while maintaining efficient segmentation performance. It is suitable for a variety of medical image segmentation tasks and has a wide range of clinical application value.
[0039] 4. The EMIS-Net model introduces the feature extraction module LGFE, the information compression module ICM and the efficient feature aggregation module EFAM. These modules reduce the number of parameters while enhancing the model's ability to extract details and contour information;
[0040] 5. The present invention not only improves the efficiency and accuracy of medical image segmentation, but also provides powerful technical support for clinical diagnosis. BRIEF DESCRIPTION OF THE DRAWINGS
[0041] Figure 1 It is a schematic diagram of the process structure of the method of the present invention;
[0042] Figure 2 This is a structural diagram of the feature extraction module, information compression module, and multi-scale feature fusion module of the method of the present invention. DETAILED DESCRIPTION
[0043] Example 1: Figure 1-Figure 2 As shown, a medical image ROI region extraction method based on a deep learning network is efficient and has lower parameters, and the method includes:
[0044] Step 1. Obtain a medical image and crop the original image with the lesion as the center. The cropped size is 256×256. The medical image is randomly flipped and rotated before being input into the deep learning model.
[0045] Step 2: Input the preprocessed medical image into the constructed deep learning model, and use the feature extraction module (LGFE) in the deep learning model to extract the details and contour information in the medical image;
[0046] The feature extraction module in Step 2 includes two branches;
[0047] The Ghost module is introduced in the first branch mainly to solve the problems of redundant information and excessive model parameters caused by the convolution module. The Ghost module can increase the width and depth of the network without significantly increasing the number of parameters, so that the model can use limited computing resources more efficiently. After obtaining the output of the Ghost module, the CA attention mechanism can be combined to improve the positioning of the lesion, which can improve the accuracy of segmentation.
[0048] The input of the first branch is processed sequentially through a depth-wise separable convolution (DWconv) with a convolution kernel size of 5×5, batch normalization, an activation function (GELU), a Ghost module, a CA block, and batch normalization to obtain the output of the first branch;
[0049] The input of the second branch is processed by a convolution with a kernel size of 1×1 and a dilation rate of 7 to obtain the output of the second branch. The dilated convolution is introduced in the second branch, which can expand the receptive field without increasing the memory overhead and computational overhead to obtain richer context information.
[0050] The output result of the first branch is added to the output result of the second branch (Addition), and then an activation function (GELU) is passed to obtain the final output feature.
[0051] Step 3. Use the information compression module (ICM) to compress the information in the encoding process to reduce information loss in the encoding and decoding process. The information compression module (ICM) is mainly a narrow-wide-narrow structure, which increases the dimension of the feature map in the second convolution layer, which helps the network learn more complex features. In the third convolution layer, the expanded feature map is compressed back to the original dimension, which helps the network focus on the most important features. At the same time, it can also reduce the amount of calculation and improve the efficiency and performance of the network.
[0052] Perform the first convolution to extract features of the input information compression module;
[0053] The features extracted by the first convolution are convolved for the second time to 4 times the original number of channels before extracting the features;
[0054] The features extracted by the second convolution are convolved for the third time to reduce the number of channels back to the original number and then extract the features;
[0055] The results of the three convolutions are processed by the batch normalization layer, and the features of the original input information compression module are added. The result of the addition is activated by the activation function GELU to obtain the final output.
[0056] Step 4: Use a decoder to extract features of different scales from the compressed medical image; the structure of the decoder is the same as the feature extraction module in Step 2.
[0057] Step 5: Fuse features of different scales through the efficient feature aggregation module (EFAM) to enhance the model’s understanding of details and context;
[0058] The efficient feature aggregation module EFAM mainly uses 1×1 convolution kernels and different dilation rates. The 1×1 convolution kernel has a small number of parameters and low computational complexity, and different dilation rates will not introduce additional parameter consumption and computation while expanding the receptive field.
[0059] Step 5 includes:
[0060] Step 5.1, the input of the efficient feature aggregation module EFAM is processed by three parallel convolutional layers with different expansion rates and a convolution kernel size of 1×1, and the results are output respectively;
[0061] Step 5.2, the output results of the three branches are spliced to obtain the spliced result;
[0062] Step 5.3, input the spliced result into a convolution layer with a convolution kernel size of 1×1 to obtain the final output.
[0063] Step 6: Input the fused feature map into the convolution module (OutConv) for the final medical image segmentation processing to achieve accurate segmentation of the lesion area in the medical image.
[0064] The present invention also provides a medical image ROI region extraction system based on a deep learning network that is efficient and has lower parameters, including: a module for executing the above-mentioned medical image ROI region extraction method based on a deep learning network that is efficient and has lower parameters.
[0065] The present invention also provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, the above-mentioned medical image ROI region extraction method with high efficiency and lower parameters based on a deep learning network is implemented.
[0066] The present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that when the computer program is executed by a processor, the above-mentioned medical image ROI region extraction method with high efficiency and lower parameters based on a deep learning network is implemented.
[0067] The present invention also provides a computer program product, including a computer program, characterized in that when the computer program is executed by a processor, it implements the above-mentioned medical image ROI area extraction method with high efficiency and lower parameters based on a deep learning network.
[0068] To verify the effectiveness of the method of the present invention, the present invention evaluates the performance of the present invention on three public data sets of different diseases. The data used in the present invention are Kvasir-SEG (polyp), ISIC2018 (skin cancer), and GLaS (gland).
[0069] (1) The Kvasir-SEG dataset is a publicly available polyp segmentation dataset from a clinical colonoscopy program, including 1,000 polyp images and their corresponding ground truth labels. The images were carefully annotated manually by a physician and then verified by an experienced gastroenterologist. The image resolutions in Kvasir-SEG range from 332×487 to 1920×1072 pixels. In this paper, 800 images are used for training and 200 images are used for testing.
[0070] (2) The segmentation dataset of ISIC2018 (International Skin Imaging Collaboration 2018 Challenge) is a high-quality data resource designed specifically to promote and develop skin lesion image analysis technology. This dataset has a significant impact in the field of medical image segmentation, especially in the research of dermatology and early detection of skin cancer. It consists of 2594 images and 2594 corresponding true labels, which are carefully annotated by dermatologists and provide accurate segmentation boundaries of lesion areas. The original size of ISIC2018 ranges from 720×540 to 6708×4439. In this invention, we adopted a random partitioning, allocating 2076 images for training and 518 for testing.
[0071] (3) The GlaS dataset consists of 165 images, which are H&E stained T3 or T4 colorectal adenocarcinoma tissue sections. Each section comes from a different patient, and these sections come from different laboratory environments, resulting in high variability in staining distribution and tissue structure between patients. The use of the GlaS dataset is of great significance to medical image segmentation research and colorectal cancer-related research. It provides a standardized evaluation platform for researchers to advance medical image segmentation algorithms and provides valuable information for improving the diagnosis and treatment of colorectal adenocarcinoma. In the present invention, 85 images are used for training and 80 images are used for testing.
[0072] The present invention is implemented in Python 3.8 and Pytorch 1.12.1. The input image is randomly flipped and rotated for data augmentation. The input image size is set to 256×256, the batch size is set to 4, and the model corresponding to the method of the present invention is trained on an NVIDIA GeForce RTX 3080 GPU. The model uses the Dice loss function and the AdamW optimizer, the weight decay is set to 5e-5, the learning rate is set to 0.0005, and the Epoch is set to 300.
[0073] The present invention uses the dice coefficient (Dice), mean intersection over union (mIoU), accuracy (Acc), computational complexity (FLops) and parameter quantity to evaluate the segmentation performance of the method in medical image segmentation.
[0074]
[0075] Among them, TP (True Positive) is the positive sample predicted by the model as the positive class, FN (False Negative) is the positive sample predicted by the model as the negative class, TN (True Negative) is the negative sample predicted by the model as the negative class, and FP (False Positive) is the negative sample predicted by the model as the positive class.
[0076] Furthermore, on three public data sets, the performance of the method of the present invention is compared with that of medical image ROI region extraction methods such as UNet, SegNet, and Attention UNet, and the experimental results are shown in Tables 1-3. It can be seen that the method of the present invention has achieved the following results in terms of dice coefficient, average intersection-over-union ratio, and accuracy: (1) Kvasir-SEG: 90.0%, 89.1%, and 96.9%; (2) ISIC2018: 90.4%; 90.1%, 96.6; (3) GLaS: 91.6%, 85.0%, and 91.6%; all higher than other algorithms. In addition, the number of parameters of the method of the present invention is 2.2M, and the computational complexity of the method of the present invention is only 3.7G, which is more efficient than other algorithms in the experiment, proving that the method of the present invention is more efficient and lightweight in medical image segmentation.
[0077] Table 1 shows the objective evaluation comparison of different methods on the Kvasir-SEG dataset
[0078]
[0079] Table 2 shows the objective evaluation comparison of different methods on the ISIC2018 dataset.
[0080]
[0081]
[0082] Table 3 shows the objective evaluation comparison of different methods on the GLaS dataset.
[0083]
[0084] The EMIS-Net model corresponding to the method of the present invention introduces the feature extraction module LGFE, the information compression module ICM and the efficient feature aggregation module EFAM, which reduce the number of parameters while enhancing the model's ability to extract details and contour information. Through experiments on three datasets, Kvasir-SEG, ISIC2018 and GLaS, EMIS-Net demonstrated excellent performance, with Dice coefficients of 90.0%, 90.4% and 91.6% respectively, which are better than other segmentation models mentioned in the experiment. This research result not only improves the efficiency and accuracy of medical image segmentation, but also provides strong technical support for clinical diagnosis.
[0085] The specific implementation modes of the present invention are described in detail above in conjunction with the accompanying drawings, but the present invention is not limited to the above implementation modes, and various changes can be made within the knowledge scope of ordinary technicians in this field without departing from the purpose of the present invention.
Claims
1. An efficient and low-parameter ROI region extraction method for medical images based on a deep learning network, characterized by: The method comprises: Step 1, obtain medical images and preprocess them; Step 2: Input the preprocessed medical image into the constructed deep learning model, and use the feature extraction module in the deep learning model to extract the details and contour information in the medical image; Step 3, using the information compression module to compress the information in the encoding process; Step 4: Use the decoder to extract features of different scales from the compressed medical image; Step 5: Fuse features of different scales through an efficient feature aggregation module; Step 6: Input the fused feature map into the convolution module for the final medical image segmentation processing.
2. The method for extracting ROI regions of medical images based on a deep learning network with high efficiency and lower parameters according to claim 1, characterized in that: In the Step 1, the medical image is cropped with the lesion as the center, and the medical image is randomly flipped and rotated before being input into the deep learning model.
3. The method for extracting ROI regions of medical images based on a deep learning network with high efficiency and lower parameters according to claim 1, characterized in that: The feature extraction module in Step 2 includes two branches; Step 2 includes: The input of the first branch is processed sequentially through a depth-wise separable convolution with a convolution kernel size of 5×5, batch normalization, activation function, Ghost module, CA block, and batch normalization to obtain the output of the first branch; The input of the second branch is processed by a convolution with a convolution kernel size of 1×1 and a dilation rate of 7 to obtain the output of the second branch; The output of the first branch is added to the output of the second branch, and then an activation function is passed to obtain the final output feature.
4. The method for extracting ROI regions of medical images based on a deep learning network with high efficiency and lower parameters according to claim 1, characterized in that: The Step 3 includes: Perform the first convolution to extract features of the input information compression module; The features extracted by the first convolution are convolved for the second time to 4 times the original number of channels before extracting the features; The features extracted by the second convolution are convolved for the third time to reduce the number of channels back to the original number and then extract the features; The results of the three convolutions are processed by the batch normalization layer, and the features of the original input information compression module are added. The result of the addition is activated by the activation function GELU to obtain the final output.
5. The method for extracting ROI regions of medical images based on a deep learning network with high efficiency and lower parameters according to claim 1, characterized in that: In the Step 4, the structure of the decoder is the same as the feature extraction module in Step 2.
6. The method for extracting ROI regions of medical images based on a deep learning network with high efficiency and lower parameters according to claim 1, characterized in that: The Step 5 includes: Step 5.1, the input of the efficient feature aggregation module is processed by three parallel convolutional layers with different expansion rates and a convolution kernel size of 1×1, and the results are output respectively; Step 5.2, the output results of the three branches are spliced to obtain the spliced result; Step 5.3, input the spliced result into a convolution layer with a convolution kernel size of 1×1 to obtain the final output.
7. An efficient and lower-parameter medical image ROI region extraction system based on deep learning network, characterized by: include: A module for executing the medical image ROI region extraction method based on a deep learning network with high efficiency and lower parameters as described in any one of claims 1 to 6.
8. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the program, the method for extracting ROI regions of medical images with high efficiency and lower parameters based on a deep learning network is implemented as described in any one of claims 1 to 6.
9. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the method for extracting ROI regions of medical images with high efficiency and lower parameters based on a deep learning network is implemented as described in any one of claims 1 to 6.
10. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the method for extracting ROI regions of medical images with high efficiency and lower parameters based on a deep learning network is implemented as described in any one of claims 1 to 6.