A lightweight medical image segmentation method, device, medium and electronic equipment

Abstract

The present application provides a kind of lightweight medical image segmentation method, device, medium and electronic equipment, utilize the target lesion in the medical image to be segmented in pre-trained lightweight medical image segmentation model and carry out segmentation, in lightweight medical image segmentation model, down-sampling unit carries out down-sampling processing to medical image to be segmented;First EERM layer carries out feature extraction to the down-sampling processing result and obtains preliminary feature extraction result;First segmentation unit carries out down-sampling processing to preliminary feature extraction result and then carries out feature extraction operation based on ConvLSTM;Second segmentation unit carries out feature extraction operation based on ConvLSTM to preliminary feature extraction result;Third segmentation unit is used to carry out feature extraction operation based on ConvLSTM to preliminary feature extraction result after carrying out up-sampling processing;Result output unit fuses three feature extraction results and then outputs image segmentation result.The present application can run in the device with relatively lower graphics card configuration, model parameter is less, and can give consideration to calculation speed and calculation accuracy.

Description

Technical Field

[0001] This invention relates to the field of medical image segmentation technology, and in particular to a lightweight medical image segmentation method, apparatus, medium and electronic device. Background Technology

[0002] This section is intended to provide background or context for the embodiments set forth in the claims. The description herein is not an admission that it is prior art simply because it is included in this section.

[0003] With the continuous development of medical image segmentation technology, many excellent medical segmentation models have emerged, such as U-Net, 3D U-Net, and V-Net. However, as is well known, image segmentation methods are very computationally intensive, especially 3D image segmentation methods, and they also consume a lot of video memory. Therefore, these image segmentation methods require high-end graphics cards to run, which makes intelligent medical image segmentation products expensive and difficult to popularize, and ordinary doctors can hardly experience the convenience brought by artificial intelligence. Therefore, the research on ultra-lightweight segmentation models is very necessary.

[0004] Most common neural network architectures use U-Net, built with ordinary double convolutions, as the backbone network. This results in many models having a large number of parameters, slow computation speed, and high memory consumption, making it impossible to achieve ultra-lightweight segmentation models. Consequently, these models cannot run on computers with low-end graphics card configurations. Summary of the Invention

[0005] The present invention aims to provide a lightweight medical image segmentation method, device, medium and electronic device that can run on devices with relatively low graphics card configuration, has few model parameters, and can balance calculation speed and calculation accuracy.

[0006] In a first aspect, embodiments of the present invention provide a lightweight medical image segmentation method, comprising:

[0007] Acquire the medical image to be segmented;

[0008] A pre-trained lightweight medical image segmentation model is used to segment the target lesion in the medical image to be segmented, and the image segmentation result is obtained.

[0009] The lightweight medical image segmentation model includes a downsampling unit, a first ERM layer, a first segmentation unit, a second segmentation unit, a third segmentation unit, and a result output unit.

[0010] The downsampling unit is used to downsample the medical image to be segmented to obtain a downsampling result; the first ERM layer is used to extract features from the downsampling result to obtain a preliminary feature extraction result; the first segmentation unit is used to downsample the preliminary feature extraction result and then perform a ConvLSTM-based feature extraction operation to obtain a first feature extraction result; the second segmentation unit is used to perform a ConvLSTM-based feature extraction operation on the preliminary feature extraction result to obtain a second feature extraction result; the third segmentation unit is used to upsample the preliminary feature extraction result and then perform a ConvLSTM-based feature extraction operation to obtain a third feature extraction result; the result output unit is used to fuse the first feature extraction result, the second feature extraction result, and the third feature extraction result to output an image segmentation result.

[0011] In some implementations, acquiring the medical image to be segmented includes:

[0012] Acquire raw medical images;

[0013] The original medical image is segmented along the z-axis to obtain image blocks, which are used as the medical images to be segmented.

[0014] In some implementations, the downsampling unit includes a first downsampling layer, a second downsampling layer, and a third downsampling layer connected in sequence, wherein the input of the first downsampling layer is the medical image to be segmented, and the third downsampling layer is connected to the first EERM layer;

[0015] The result output unit comprises a first activation function layer, a second ERM layer, a first upsampling layer, a second activation function layer, a third ERM layer, a second upsampling layer, a fourth ERM layer, and a third upsampling layer connected in sequence. The input to the first activation function layer includes the sum of the first feature extraction result and the second feature extraction result. The input to the second ERM layer includes the sum of the output of the first activation function layer and the output of the third downsampling layer. The input to the third ERM layer includes the sum of the third feature extraction result and the output of the first upsampling layer. The input to the fourth ERM layer includes the sum of the output of the second upsampling layer and the output of the first downsampling layer.

[0016] In some implementations, the first segmentation unit includes a first sub-downsampling layer, a first sub-EERM layer, at least two first sub-ConvLSTM layers, a second sub-EERM layer, and a first sub-upsampling layer connected in sequence; the second segmentation unit includes a third sub-EERM layer and at least two second sub-ConvLSTM layers connected in sequence; and the third segmentation unit includes a second sub-upsampling layer, a fourth sub-EERM layer, and at least two third sub-ConvLSTM layers connected in sequence.

[0017] The initial hidden state and initial cell state output by the first first sub-ConvLSTM layer among the at least two first sub-ConvLSTM layers are provided to the first second sub-ConvLSTM layer among the at least two second sub-ConvLSTM layers; the initial hidden state and initial cell state output by the first second sub-ConvLSTM layer among the at least two second sub-ConvLSTM layers are provided to the first third sub-ConvLSTM layer among the at least two third sub-ConvLSTM layers.

[0018] In some implementations, the first EERM layer, the second EERM layer, the third EERM layer, the fourth EERM layer, the first sub-EERM layer, the second sub-EERM layer, the third sub-EERM layer, and the fourth sub-EERM layer all include at least one EERM module.

[0019] In some implementations, an scSE attention mechanism layer is set before the last activation function layer in the EERM module.

[0020] In some implementations, the method further includes:

[0021] Obtain a medical image dataset containing the target lesion, wherein the medical image dataset includes several image blocks obtained by segmenting the original medical image set along the z-axis;

[0022] The medical image dataset is divided into a training set, a validation set, and a test set according to a set ratio;

[0023] The lightweight medical image segmentation model must be trained for at least a preset number of epochs;

[0024] The trained lightweight medical image segmentation model is determined based on the loss or Dice metric of the lightweight medical image segmentation model determined on the validation set.

[0025] Secondly, embodiments of the present invention provide a lightweight medical image segmentation device, comprising:

[0026] The acquisition module is used to acquire the medical image to be segmented;

[0027] The segmentation module is used to segment the target lesion in the medical image to be segmented using a pre-trained lightweight medical image segmentation model, and obtain the image segmentation result.

[0028] The lightweight medical image segmentation model includes a downsampling unit, a first ERM layer, a first segmentation unit, a second segmentation unit, a third segmentation unit, and a result output unit.

[0029] The downsampling unit is used to downsample the medical image to be segmented to obtain a downsampling result; the first ERM layer is used to extract features from the downsampling result to obtain a preliminary feature extraction result; the first segmentation unit is used to downsample the preliminary feature extraction result and then perform a ConvLSTM-based feature extraction operation to obtain a first feature extraction result; the second segmentation unit is used to perform a ConvLSTM-based feature extraction operation on the preliminary feature extraction result to obtain a second feature extraction result; the third segmentation unit is used to upsample the preliminary feature extraction result and then perform a ConvLSTM-based feature extraction operation to obtain a third feature extraction result; the result output unit is used to fuse the first feature extraction result, the second feature extraction result, and the third feature extraction result to output an image segmentation result.

[0030] Thirdly, embodiments of the present invention provide a computer-readable storage medium storing a computer program, which, when executed by at least one processor, implements the method described in the first aspect.

[0031] Fourthly, embodiments of the present invention provide an electronic device, including a memory and at least one processor, wherein the memory stores a computer program, and the computer program, when executed by the at least one processor, implements the method as described in the first aspect.

[0032] The embodiments of the present invention can bring at least the following beneficial effects:

[0033] The lightweight medical image segmentation method provided in this embodiment uses a pre-trained lightweight medical image segmentation model to segment target lesions in the medical image to be segmented. In the lightweight medical image segmentation model, the medical image to be segmented is first reduced in size by downsampling, and then the three designed branches are input to extract features based on EERM and ConvLSTM for information on larger, medium and smaller lesions, respectively. This improves the computation speed and accuracy, consumes less video memory, and can support running on devices with relatively low graphics card configurations. Attached Figure Description

[0034] To more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings used in the embodiments will be briefly described below. It should be understood that the following drawings only show some embodiments of the present invention and should not be regarded as a limitation on the scope.

[0035] Figure 1 This is a flowchart of a lightweight medical image segmentation method provided by an embodiment of the present invention;

[0036] Figure 2 This is a schematic diagram of a lightweight medical image segmentation model structure provided in an embodiment of the present invention;

[0037] Figure 3 This is an example of a lightweight medical image segmentation model structure provided in an embodiment of the present invention;

[0038] Figure 4 This is a schematic diagram of the EERM module provided in an embodiment of the present invention;

[0039] Figure 5 This is a schematic diagram of the scSE attention mechanism layer provided in an embodiment of the present invention;

[0040] Figure 6 This is a schematic diagram of a lightweight medical image segmentation device provided in an embodiment of the present invention. Detailed Implementation

[0041] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. The components of the embodiments of the present invention described and shown in the accompanying drawings can generally be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of the present invention provided in the accompanying drawings is not intended to limit the scope of the claimed invention, but merely to illustrate selected embodiments of the invention. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without inventive effort are within the scope of protection of the present invention.

[0042] Example 1

[0043] This embodiment provides a lightweight medical image segmentation method, such as Figure 1 As shown, it includes:

[0044] Step S101: Obtain the medical image to be segmented.

[0045] In some implementations, acquiring the medical image to be segmented may include:

[0046] Step S101a: Obtain the original medical image;

[0047] Step S101b: Divide the original medical image along the z-axis to obtain image blocks, which are used as the medical image to be segmented.

[0048] In practical applications, medical images can refer to CT images. Before inputting the raw CT image into a pre-trained lightweight medical image segmentation model, preprocessing can be performed to segment the raw CT image along the Z-axis, resulting in image blocks of at least four layers. Taking 16 layers as an example, after segmentation, a 16*512*512 (Z*Y*X) image block is obtained. It should be noted that if further scaling is required, the X and Y axes of the image block can be scaled to 256*256 to form a 16*256*256 image block. Of course, depending on different layer and scaling requirements, the raw CT image can also be preprocessed into image blocks of 8*64*64, 16*64*64, etc.

[0049] In addition, the medical image to be segmented should at least contain a medical image of the region where the target lesion is located. For example, if the target lesion is a lung nodule, the image to be segmented should at least contain a medical image of the lung region.

[0050] Step S102: Use a pre-trained lightweight medical image segmentation model to segment the target lesion in the medical image to be segmented, and obtain the image segmentation result.

[0051] The lightweight medical image segmentation model in this embodiment can be a two-dimensional or three-dimensional lightweight medical image segmentation model. The dimension of the model is not limited in this embodiment.

[0052] like Figure 2 As shown, the lightweight medical image segmentation model includes a downsampling unit 100, a first EERM layer 200, a first segmentation unit 300, a second segmentation unit 400, a third segmentation unit 500, and a result output unit 600; where EERM stands for Extremely Efficient Residual Module.

[0053] The downsampling unit 100 is used to downsample the medical image to be segmented, obtaining a downsampling result to reduce the size of the medical image input to the model, thus improving computational speed. The first ERM layer 200 is used to extract features from the downsampling result to obtain a preliminary feature extraction result. The first ERM layer 200, with its low parameter count, improves segmentation accuracy while reducing computational load. The first segmentation unit 300 performs a feature extraction operation based on ConvLSTM (Convolutional Long Short-Term Memory Artificial Neural Network) after downsampling the preliminary feature extraction result to obtain a first feature extraction result. By downsampling followed by ConvLSTM processing, information from larger lesions can be extracted more efficiently. The first segmentation unit 400 obtains accurate segmentation results by magnifying the information (main information) to a certain scale; the second segmentation unit 400 performs ConvLSTM-based feature extraction on the preliminary feature extraction results to obtain the second feature extraction results. This segmentation unit can obtain accurate segmentation results for information of medium-sized lesions; the third segmentation unit 500 performs ConvLSTM-based feature extraction on the preliminary feature extraction results to obtain the third feature extraction results. This segmentation unit can obtain accurate segmentation results for information of smaller lesions by magnifying it to a certain scale; the result output unit 600 merges the first feature extraction results, the second feature extraction results, and the third feature extraction results to output the image segmentation results.

[0054] By combining EERM with ConvLSTM, a medical image segmentation model was implemented. This model greatly improves segmentation efficiency and can run on devices with relatively low graphics card configurations (such as desktops or laptops), for example, devices equipped with NVIDIA GeForce GTX 1650 graphics cards.

[0055] like Figure 3 As shown, the downsampling unit 100 may include a first downsampling layer (downsampling layer 1), a second downsampling layer (downsampling layer 2), and a third downsampling layer (downsampling layer 3) connected in sequence. The input of the first downsampling layer is the medical image to be segmented, and the third downsampling layer is connected to the first EERM layer. The medical image to be segmented may be a 16*512*512 image block. After three downsampling operations, the size of the input image is reduced. Then, the first EERM layer performs feature extraction on the downsampled image processing result, and the results are input into the three branches of the first segmentation unit 300, the second segmentation unit 400, and the third segmentation unit 500, respectively, to segment larger, medium, and smaller lesions.

[0056] like Figure 3As shown, the output unit may include a first activation function layer (activation function layer 1), a second EERM layer, a first upsampling layer (upsampling layer 1), a second activation function layer (activation function layer 2), a third EERM layer, a second upsampling layer (upsampling layer 2), a fourth EERM layer, and a third upsampling layer (upsampling layer 3) connected in sequence. The input to the first activation function layer includes the sum of the first and second feature extraction results. The input to the second EERM layer includes the sum of the output of the first activation function layer and the output of the third downsampling layer. The input to the third EERM layer includes the sum of the third feature extraction result and the output of the first upsampling layer. The input to the fourth EERM layer includes the sum of the output of the second activation function layer and the output of the second downsampling layer. The input to the fourth EERM layer includes the sum of the output of the second upsampling layer and the output of the first downsampling layer. After processing the fusion results of the three branches (first segmentation unit 300, second segmentation unit 400, and third segmentation unit 500) through the EERM layer, and then performing another upsampling process, the results are restored to the original size of 16*512*512, yielding the medical image segmentation result.

[0057] In this embodiment, after three downsampling operations in the aforementioned downsampling unit, the size of the input image is reduced. Then, information on larger, medium, and smaller lesions is obtained through three branches. The information from the three branches is fused and then upsampled to obtain the segmentation result. Throughout the segmentation process, the input images entering the three branches are relatively small. Furthermore, the segmentation method that combines EERM and ConvLSTM with fewer parameters and higher accuracy effectively improves both the computation speed and segmentation accuracy, achieving lightweight image segmentation.

[0058] like Figure 3 As shown, the first segmentation unit 300 includes a first sub-downsampling layer (downsampling layer 1-1), a first sub-EERM layer, at least two first sub-ConvLSTM layers, a second sub-EERM layer, and a first sub-upsampling layer (upsampling layer 1-1) connected in sequence; the second segmentation unit 200 includes a third sub-EERM layer and at least two second sub-ConvLSTM layers connected in sequence; the third segmentation unit includes a second sub-upsampling layer (upsampling layer 3-1), a fourth sub-EERM layer, and at least two third sub-ConvLSTM layers connected in sequence.

[0059] Specifically, the initial hidden state H and initial cell state C output by the first of the first of the at least two first sub-ConvLSTM layers are provided to the first of the at least two second sub-ConvLSTM layers; the initial hidden state H and initial cell state C output by the first of the first of the at least two second sub-ConvLSTM layers are provided to the first of the at least two third sub-ConvLSTM layers.

[0060] In one example, there are four sub-ConvLSTM layers: ConvLSTM layer 1-1, ConvLSTM layer 1-2, ConvLSTM layer 1-3, and ConvLSTM layer 1-4; four sub-ConvLSTM layers: ConvLSTM layer 2-1, ConvLSTM layer 2-2, ConvLSTM layer 2-3, and ConvLSTM layer 2-4; and four sub-ConvLSTM layers: ConvLSTM layer 3-1, ConvLSTM layer 3-2, ConvLSTM layer 3-3, and ConvLSTM layer 3-4. The initial hidden state H and initial cell state C output by ConvLSTM layer 1-1 are provided as inputs to ConvLSTM layer 2-1, and the initial hidden state H and initial cell state C output by ConvLSTM layer 2-1 are provided as inputs to ConvLSTM layer 3-1.

[0061] This embodiment uses ConvLSTM to enhance the semantic relationships between different dimensions of the feature map. At the same time, the initial hidden state (H) and initial cell state (C) of the ConvLSTM are reused between branches, which can further improve the segmentation accuracy of the model.

[0062] In this embodiment, each ERM layer can incorporate an attention mechanism to facilitate the capture of detailed object features and improve segmentation accuracy. In some implementations, the first ERM layer (attention ERM layer 1), second ERM layer (attention ERM layer 2), third ERM layer (attention ERM layer 3), fourth ERM layer (attention ERM layer 4), first sub-EERM layer (attention ERM layer 1-1), second sub-EERM layer (attention ERM layer 1-2), third sub-EERM layer (attention ERM layer 2-1), and fourth sub-EERM layer (attention ERM layer 3-1) each include at least one ERM module with an added attention mechanism. The number of ERM modules in each ERM layer can be arbitrarily expanded as needed; for example, attention ERM layer 1 can contain 5 ERM modules with added attention mechanisms. Each attention ERM module has few parameters and low computational cost, enabling fast computation even on computers with low-end graphics card configurations.

[0063] In some implementations, an scSE attention mechanism layer is set before the last activation function layer in the EERM module. For example... Figure 4 As shown, the EERM module includes the scSE attention mechanism and dilated convolution, which is beneficial for segmenting small objects. Meanwhile, because the ConvLSTM module does not consume excessive GPU memory during computation, the model can run on computers with low-end GPUs. The scSE attention mechanism is beneficial for focusing on the segmentation of small objects, and the multi-layer dilated convolution structure also increases the model's receptive field, better capturing the detailed features of objects and improving the segmentation accuracy of small targets.

[0064] like Figure 4 In the EERM module structure shown, the input feature map is processed by performing 3D convolution with a kernel size of k in each dimension. After each 3D convolution, a batch normalization (BN) layer is applied. After processing in all three dimensions, a 1*1*1 convolution kernel is used for another 3D convolution. The resulting data is then processed by batch normalization and activation function layers to obtain the first set of processing results. The first set of processing results is then input into a structure identical to the one described above for a second processing step. In this second processing step, dilated convolutions with a size of d are added. The specific value of d can be set according to actual needs. An scSE attention mechanism layer is inserted before the last activation function layer. The feature map calculated by the scSE attention mechanism layer is added to the original feature map input to the EERM module, and then processed by the activation function layer to obtain the output feature map.

[0065] The reason for inserting the scSE attention mechanism module before the last activation function layer of the EERM module in this embodiment is that the cSE attention mechanism module has good anti-interference capabilities, but its convergence speed is relatively slow and it is not easy to quickly focus on the region of interest. The sSE attention mechanism is very sensitive to the region of interest that is of great interest, but it is also quite sensitive to noise and is easily affected by noise interference. By adding the scSE attention mechanism module, the model's anti-interference capabilities can be improved while its convergence speed can be increased, allowing it to quickly focus on the region of interest.

[0066] The structure of the scSE attention mechanism layer is as follows: Figure 5 As shown, firstly, adaptive average pooling is performed on the input feature map, reducing the image length L, width W, and height H to 1. After two convolutional layers, the feature map is multiplied with the original input feature map to obtain the cSE feature map output. Secondly, convolution is performed on the input feature map, reducing the number of channels C to 1. The feature map is then multiplied with the original input feature map to obtain the sSE feature map output. Finally, the cSE and sSE feature maps are added together to obtain the scSE feature map output.

[0067] In this embodiment, each activation function layer can employ activation functions such as Sigmoid activation function, ReLU activation function, LeakyReLU activation function, and H-Swish activation function.

[0068] In some implementations, the method of this embodiment further includes the model training process in steps S201 to S204:

[0069] Step S201: Obtain a medical image dataset containing the target lesion. The medical image dataset includes several image blocks obtained by segmenting the original medical image set along the z-axis.

[0070] Adjust the window width and window level of the original medical image dataset. The original medical image dataset can be selected from, but is not limited to, the LUNA16 dataset.

[0071] Taking a lung nodule as the target lesion as an example, the window width and window level of the lung window are set. The window width (WW) ranges from 1500 to 2000 HU, and the window level (WL) ranges from -450 to -600 HU. If lesions in other organs need to be segmented, the window width and window level need to be adjusted to the corresponding window width and window level for those organs. Then, the medical image after window width and window level adjustment is normalized, mapping pixel values ​​[0, 255] to [0, 1] to improve model training speed. Normalization methods include, but are not limited to, zero-mean normalization.

[0072] Taking lung nodule segmentation as an example, the region of interest (ROI) can be preserved or cropped to extract the corresponding ROI as the original medical image, thereby improving the accuracy of segmentation. The specific method for ROI preservation is as follows: first, obtain the segmented lung region, designate this region as the ROI, then preserve the ROI while removing other regions. For example, the removal method can be to set the pixel value of the region to be removed to 0. The specific method for ROI cropping is as follows: first, obtain the lung ROI, calculate its length, width, and height, crop the lung region according to the parameters, and then scale the cropped region back to the original image size.

[0073] After obtaining the original medical images, data augmentation processing is performed on the original medical image data.

[0074] In some examples, the probability of motion blur in data augmentation is 0.04–0.1, the probability of horizontal image flipping is 0.1–0.3, the probability of vertical image flipping is 0.1–0.3, the probability of random translation, resizing, and rotation transformations of the image is 0.2–0.4, the probability of image generating mesh distortion, optical distortion, or elastic transformation is 0.1–0.3, and the probability of image generating Gaussian noise is 0.05–0.1.

[0075] The original medical image, after data augmentation, is segmented along the Z-axis to obtain a predetermined number of image patches, each consisting of at least four layers, resulting in a medical image dataset containing the target lesion. For example, with 16 layers, the segmentation yields image patches of size 16*512*512 (Z*Y*X). It should be noted that if further scaling is required, the X and Y axes of the image patches can be scaled to 256*256 to form 16*256*256 image patches.

[0076] Step S202: Divide the medical image dataset into a training set, a validation set, and a test set according to a set ratio.

[0077] Step S203: Train the lightweight medical image segmentation model for at least a preset number of cycles.

[0078] Step S204: Determine the trained lightweight medical image segmentation model based on the loss or Dice metric of the lightweight medical image segmentation model determined on the validation set.

[0079] In a specific example, the preset quantity can be 888 sets, the preset ratio can be 8:1:1, and the preset number of cycles can be set to ≥30 rounds. At least 800 sets of image patches are divided into training, validation, and test sets in an 8:1:1 ratio. The training is performed for the preset number of cycles. If the loss between the model's validation output and the true label is less than a first preset threshold, or if the Dice metric of the model's validation output reaches a second preset threshold, then the trained image segmentation model is output.

[0080] The first threshold is set to, for example, 0.5, but it can also be set to other thresholds according to actual needs. This embodiment does not limit this threshold to a single value. After training for a preset number of cycles, if the Dice index value is greater than 0.5, the model corresponding to the largest Dice index value is selected as the lightweight medical image segmentation model for segmenting lung nodules. If the Dice index value is less than 0.5, training can continue for 1 to 2 cycles. After continuing training for 1 to 2 cycles, if the Dice index value is still less than 0.5, the model corresponding to the largest Dice index value can be directly selected.

[0081] In some implementations, the loss function used to train the model is the sum of the BCE loss function and the Tversky loss function;

[0082] The expression for the loss function is: loss BCE +loss Tversky ,

[0083] BCE loss function BCE The formula for calculation is:

[0084]

[0085] Where n represents the total number of samples, x represents a sample, set A(x) represents the lung nodule annotation data drawn by doctors, and set B(x) represents the lung nodule annotation data generated by the model.

[0086] Tversky loss function Tversky The formula for calculation is:

[0087]

[0088] Wherein, set A(x) represents the labeled data generated by the model, set B(x) represents the labeled data drawn by the doctor; |A(x)-(x)| represents false positives (FP), |B(x)-(x)| represents false negatives (FN), and s represents the smoothing coefficient, preferably s = 1e -5 The calculation is relatively accurate.

[0089] The Tversky loss function has two parameters, α and β. Its advantage is that by adjusting these two parameters, the balance between false positives (FP) and false negatives (FN) can be controlled, thereby affecting the segmentation accuracy of smaller and larger segmentation regions.

[0090] The parameter α ranges from 0.2 to 0.35, and the parameter β ranges from 0.6 to 0.75. In one example, α = 0.3 and β = 0.7. Setting the parameters α and β increases the weight of image segmentation sensitivity, which is beneficial for segmenting small regions (organs) such as lung nodules.

[0091] The Dice metric was chosen as the evaluation metric for both the model validation and test sets. The formula for calculating the Dice metric is as follows:

[0092]

[0093] Where x represents a sample, set A(x) is the true label, set B(x) is the predicted label obtained from the segmentation, |A(x)∩B(x)| represents the common element between set A(x) and set B(x), |A(x)| represents the number of elements in A, |B(x)| represents the number of elements in B, and s represents the smoothing coefficient, preferably s = 1e -5 The calculation is relatively accurate.

[0094] In some cases, the trained image segmentation model can also be output based on the detection rate. For example, if the detection rate is greater than a third set threshold, the trained image segmentation model can be output.

[0095] Specifically, the detection rate and false positives (FPs) of lung nodule segmentation results were tested using a test set. The lung nodule detection rate / true positive rate (TPR) ratio is calculated as follows:

[0096]

[0097] Where TP is the number of correctly detected lung nodules, and FN is the number of actual lung nodules that were not detected.

[0098] To visually demonstrate the advantages of this invention, this embodiment conducts experiments to verify the model of this invention.

[0099] The deep learning hardware platform used in this experiment is as follows: CPU: Intel(R) Xeon(R) Gold6230R CPU @ 2.10GHz; GPU: NVIDIA GeForce RTX 3090; RAM: 128GB. The software platform is: Windows 10 21H2 operating system; programming language: Python 3.7; development framework: PyTorch 1.13.1 + cu117.

[0100] Based on the LUNA16 public dataset, different lung nodule segmentation models were trained using image patches of size 16*256*256, and their performance was compared.

[0101] The following table compares the number of 3D neural network parameters, maximum memory usage, frames per second (FPS), and lung nodule detection rate in the method of this embodiment with those in related technologies.

[0102]

[0103] As can be seen, this embodiment has the fewest model parameters and achieves a balance between computational speed and computational accuracy.

[0104] Example 2

[0105] This embodiment provides a lightweight medical image segmentation device, such as... Figure 6 As shown, it includes:

[0106] The acquisition module 601 is used to acquire the medical image to be segmented;

[0107] The segmentation module 602 is used to segment the target lesion in the medical image to be segmented using a pre-trained lightweight medical image segmentation model, and obtain the image segmentation result.

[0108] The lightweight medical image segmentation model includes a downsampling unit, a first ERM layer, a first segmentation unit, a second segmentation unit, a third segmentation unit, and a result output unit.

[0109] The downsampling unit is used to downsample the medical image to be segmented to obtain the downsampling result; the first ERM layer is used to extract features from the downsampling result to obtain the preliminary feature extraction result; the first segmentation unit is used to downsample the preliminary feature extraction result and then perform a ConvLSTM-based feature extraction operation to obtain the first feature extraction result; the second segmentation unit is used to perform a ConvLSTM-based feature extraction operation on the preliminary feature extraction result to obtain the second feature extraction result; the third segmentation unit is used to upsample the preliminary feature extraction result and then perform a ConvLSTM-based feature extraction operation to obtain the third feature extraction result; the result output unit is used to fuse the first feature extraction result, the second feature extraction result, and the third feature extraction result to output the image segmentation result.

[0110] The specific implementation methods of each module are detailed in Example 1, and will not be repeated in this example. This example possesses at least all the beneficial effects of Example 1.

[0111] Example 3

[0112] This invention provides a computer-readable storage medium storing a computer program, which, when executed by at least one processor, implements the method of Embodiment 1.

[0113] Example 4

[0114] This invention provides an electronic device, including a memory and at least one processor. The memory stores a computer program, which, when executed by the at least one processor, implements the method of Embodiment 1.

[0115] The aforementioned processor may be implemented using an Application Specific Integrated Circuit (ASIC), a Digital Signal Processor (DSP), a Digital Signal Processing Device (DSPD), a Programmable Logic Device (PLD), a Field Programmable Gate Array (FPGA), a controller, a Microcontroller Unit (MCU), a microprocessor, or other electronic components to execute the above methods.

[0116] The aforementioned computer-readable storage medium can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as Static Random Access Memory (SRAM), Electrically Erasable Programmable Read-Only Memory (EEPROM), Erasable Programmable Read-Only Memory (EPROM), Programmable Read-Only Memory (PROM), Read-Only Memory (ROM), magnetic storage, flash memory, magnetic disk, or optical disk.

[0117] In the several embodiments provided in this invention, it should be understood that the disclosed apparatus and methods can also be implemented in other ways. The apparatus and method embodiments described above are merely illustrative.

[0118] It should be noted that, in this document, the terms "first," "second," etc., used in the specification, claims, and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. The terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0119] While the embodiments disclosed in this invention are as described above, the content is merely for the purpose of facilitating understanding of the invention and is not intended to limit the invention. Any person skilled in the art to which this invention pertains may make any modifications and variations in form and detail of the implementation without departing from the spirit and scope disclosed herein; however, the scope of patent protection for this invention shall still be determined by the scope defined in the appended claims.

Claims

1. A lightweight medical image segmentation method, characterized by, The method comprises: obtaining a medical image to be segmented; segmenting a target lesion in the medical image to be segmented by using a pre-trained lightweight medical image segmentation model to obtain an image segmentation result; wherein the lightweight medical image segmentation model comprises a downsampling unit, a first EERM layer, a first segmentation unit, a second segmentation unit, a third segmentation unit, and a result output unit; the downsampling unit is configured to perform downsampling processing on the medical image to be segmented to obtain a downsampling processing result; the first EERM layer is configured to perform feature extraction on the downsampling processing result to obtain a preliminary feature extraction result; the first segmentation unit is configured to perform ConvLSTM-based feature extraction on the preliminary feature extraction result after downsampling processing to obtain a first feature extraction result; the second segmentation unit is configured to perform ConvLSTM-based feature extraction on the preliminary feature extraction result to obtain a second feature extraction result; the third segmentation unit is configured to perform ConvLSTM-based feature extraction on the preliminary feature extraction result after upsampling processing to obtain a third feature extraction result; and the result output unit is configured to fuse the first feature extraction result, the second feature extraction result, and the third feature extraction result and output an image segmentation result; the first segmentation unit comprises a first sub-downsampling layer, a first sub-EERM layer, at least two first sub-ConvLSTM layers, a second sub-EERM layer, and a first sub-upsampling layer connected in sequence; the second segmentation unit comprises a third sub-EERM layer and at least two second sub-ConvLSTM layers connected in sequence; and the third segmentation unit comprises a second sub-upsampling layer, a fourth sub-EERM layer, and at least two third sub-ConvLSTM layers connected in sequence; wherein an initial hidden state and an initial cell state output by a first first sub-ConvLSTM layer in the at least two first sub-ConvLSTM layers are provided to a first second sub-ConvLSTM layer in the at least two second sub-ConvLSTM layers; and an initial hidden state and an initial cell state output by a first second sub-ConvLSTM layer in the at least two second sub-ConvLSTM layers are provided to a first third sub-ConvLSTM layer in the at least two third sub-ConvLSTM layers.

2. The lightweight medical image segmentation method of claim 1, wherein, The method further comprises: obtaining an original medical image; slicing the original medical image along the z-axis to obtain image blocks as the medical image to be segmented.

3. The lightweight medical image segmentation method according to claim 1, wherein the downsampling unit comprises a first downsampling layer, a second downsampling layer, and a third downsampling layer connected in sequence, wherein the input of the first downsampling layer is the medical image to be segmented, and the third downsampling layer is connected to the first EERM layer. ​ The result output unit comprises a first activation function layer, a second EERM layer, a first up-sampling layer, a second activation function layer, a third EERM layer, a second up-sampling layer, a fourth EERM layer and a third up-sampling layer connected in sequence, wherein the input of the first activation function layer comprises the result of adding the first feature extraction result and the second feature extraction result, the input of the second EERM layer comprises the result of adding the output of the first activation function layer and the output of the third down-sampling layer, the input of the second activation function layer comprises the result of adding the third feature extraction result and the output of the first up-sampling layer, the input of the third EERM layer comprises the result of adding the output of the second activation function layer and the output of the second down-sampling layer, and the input of the fourth EERM layer comprises the result of adding the output of the second up-sampling layer and the output of the first down-sampling layer.

4. The lightweight medical image segmentation method according to any one of claims 1 to 3, characterized in that, The first EERM layer, the second EERM layer, the third EERM layer, the fourth EERM layer, the first sub-EERM layer, the second sub-EERM layer, the third sub-EERM layer and the fourth sub-EERM layer each comprise at least one EERM module.

5. The lightweight medical image segmentation method of claim 4, wherein, The last activation function layer in the EERM module is provided with an scSE attention mechanism layer before it.

6. The lightweight medical image segmentation method of claim 1, wherein, Also comprising: obtaining a medical image data set containing a target lesion, the medical image data set comprising a plurality of image blocks obtained by slicing an original medical image set along a z-axis; dividing the medical image data set into a training set, a validation set and a test set according to a set proportion; training the lightweight medical image segmentation model for at least a preset number of periods; determining the trained lightweight medical image segmentation model according to the loss or Dice index of the lightweight medical image segmentation model determined based on the validation set.

7. A lightweight medical image segmentation device for implementing the lightweight medical image segmentation method of claim 1, characterized by, Comprising: an acquisition module configured to acquire a medical image to be segmented; a segmentation module configured to segment a target lesion in the medical image to be segmented by using a pre-trained lightweight medical image segmentation model to obtain an image segmentation result; wherein the lightweight medical image segmentation model comprises a down-sampling unit, a first EERM layer, a first segmentation unit, a second segmentation unit, a third segmentation unit and a result output unit; The downsampling unit is configured to perform downsampling processing on the medical image to be segmented to obtain a downsampling processing result; the first EERM layer is configured to perform feature extraction on the downsampling processing result to obtain a preliminary feature extraction result; the first segmentation unit is configured to perform downsampling processing on the preliminary feature extraction result and then perform ConvLSTM-based feature extraction to obtain a first feature extraction result; the second segmentation unit is configured to perform ConvLSTM-based feature extraction on the preliminary feature extraction result to obtain a second feature extraction result; the third segmentation unit is configured to perform upsampling processing on the preliminary feature extraction result and then perform ConvLSTM-based feature extraction to obtain a third feature extraction result; and the result output unit is configured to fuse the first feature extraction result, the second feature extraction result and the third feature extraction result and then output an image segmentation result.

8. A computer-readable storage medium, characterized in that, The computer program is stored on the computer readable storage medium and is executed by at least one processor to implement the method in any one of claims 1 to 6.

9. An electronic device, comprising: The device comprises a memory and at least one processor, and the memory stores a computer program which is executed by the at least one processor to implement the method in any one of claims 1 to 6.

Images