Deep learning-based lung nodule fine segmentation method and device, and storage medium

Abstract

The present application relates to a kind of lung nodule fine segmentation method, device and storage medium based on deep learning, the method includes the following steps: constructing the lung nodule segmentation model based on deep learning of multi-scale output;Get the lung CT image to be segmented, the lung CT image to be segmented is preprocessed, obtains 3D image block to be segmented;With the 3D image block to be segmented as the input of the lung nodule segmentation model, different scale output results are obtained;Each different scale output result is fused and processed, and final segmentation result is obtained.Compared with prior art, the present application has the advantages of high segmentation accuracy.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of medical image processing, in particular to a lung nodule fine segmentation method and device based on deep learning and a storage medium. BACKGROUND

[0002] Lung cancer is one of the highest mortality tumor diseases. More than 1.3 million people die of lung cancer every year in the world. Since the clinical manifestations of lung cancer are not obvious at the early stage, more than 70% of lung cancer patients are basically in the advanced stage of lung cancer when diagnosed, and early intervention and treatment are the key to improve the survival rate of lung cancer patients. Lung cancer at the early stage generally shows lung nodules. At present, the most common method for early detection of lung cancer is to perform imaging examination by computed tomography (CT), and then radiologists perform screening and diagnosis, radiologists usually only qualitatively evaluate lung nodules according to medical knowledge and experience, in this background, a computer-aided diagnosis (CAD) system for lung cancer based on CT images is proposed, the emergence of computer-aided diagnosis technology based on CT images can assist radiologists in diagnosis, and serve as a "second opinion" for clinical diagnosis. Precise segmentation of lung nodules not only helps doctors to quantitatively evaluate the growth trend of nodules, but also is the basis for early computer-aided diagnosis of lung cancer.

[0003] Current research on lung nodule segmentation methods at home and abroad has made some progress, but there are still some deficiencies, and there is a lot of room for improvement and improvement. SUMMARY

[0004] The purpose of the present application is to overcome the defects of the prior art and provide a lung nodule fine segmentation method and device based on deep learning with high segmentation accuracy and a storage medium.

[0005] The purpose of the present application can be achieved by the following technical solutions:

[0006] A lung nodule fine segmentation method based on deep learning, comprising the following steps:

[0007] Constructing a lung nodule segmentation model based on deep learning with multi-scale output;

[0008] Obtaining a lung CT image to be segmented, pre-processing the lung CT image to be segmented, and obtaining a 3D image block to be segmented;

[0009] Taking the 3D image block to be segmented as the input of the lung nodule segmentation model, obtaining different scale output results;

[0010] Fusing each different scale output result to obtain a final segmentation result.

[0011] Further, the preprocessing includes a normalization operation and an image cropping process.

[0012] Further, the normalization operation is specifically:

[0013] The HU value greater than 400 in the CT image is defined as 1, and the HU value less than-1000 is defined as 0.

[0014] Further, the lung nodule segmentation model takes a 3D U-Net network as a basic structure, and multiple outputs of different scales are introduced in the decoding part of the 3D U-Net network.

[0015] Further, the data set used when training the lung nodule segmentation model includes CT images and corresponding segmentation mask images formed based on the annotation information.

[0016] Further, the fusion processing is:

[0017] The output result of the smallest scale is taken as the basic output, and the output results of other scales are sequentially fused with other output results of relatively smaller scales from small to large in scale, and the last output result is taken as the final segmentation result.

[0018] Further, when training the lung nodule segmentation model, the model parameters are updated based on a batch average dice loss function, the loss value of each scale output result and the segmented label image is calculated based on the batch average dice loss function, the final loss value is obtained by weighting the loss values of each scale, and the model parameters are updated using the final loss value.

[0019] Further, the expression of the batch average dice loss function is:

[0020]

[0021] In the formula, Q is the number of samples in each batch, w, h, and l correspond to the length, width, and height of the 3D image block respectively, g corresponds to the foreground area in the label image, and e is the prediction result corresponding to the deep learning model based on three-dimensional context information.

[0022] The application also provides a lung nodule fine segmentation device based on deep learning, comprising:

[0023] One or more processors;

[0024] Memory; and

[0025] One or more programs stored in the memory, the one or more programs including instructions for performing the lung nodule fine segmentation based on deep learning as described above.

[0026] The present invention also provides a computer-readable storage medium including one or more programs executable by one or more processors of an electronic device, the one or more programs including instructions for performing the deep learning-based fine segmentation of lung nodules as described above.

[0027] Compared with the prior art, the present invention has the following beneficial effects:

[0028] 1. This invention improves the segmentation effect of the model by refining the segmentation process of prediction results at multiple different scales.

[0029] 2. In this invention, when preprocessing the training dataset, both CT images and segmentation mask images are preprocessed simultaneously to improve data reliability and thus improve the model's segmentation accuracy.

[0030] 3. This invention updates model parameters using the batch average Dice loss function, which has high accuracy. Attached Figure Description

[0031] Figure 1 This is a schematic diagram of the process of the present invention;

[0032] Figure 2 This is a schematic diagram of a multi-scale output lung nodule segmentation model constructed according to the present invention. Detailed Implementation

[0033] The present invention will now be described in detail with reference to the accompanying drawings and specific embodiments. These embodiments are based on the technical solution of the present invention and provide detailed implementation methods and specific operating procedures. However, the scope of protection of the present invention is not limited to the following embodiments.

[0034] Example 1

[0035] This embodiment provides a deep learning-based method for refined segmentation of lung nodules, such as... Figure 1 As shown, the process includes the following steps: constructing a multi-scale output lung nodule segmentation model; acquiring a lung CT image to be segmented, preprocessing the lung CT image to be segmented to obtain a 3D image block to be segmented; using the 3D image block to be segmented as the input of the lung nodule segmentation model to obtain output results at different scales; fusing the output results at different scales to gradually refine the segmentation of the lung nodules and obtain the final segmentation result.

[0036] In this method, the lung nodule segmentation model uses a 3D U-Net network as its basic structure, and its construction and training process includes the following steps:

[0037] (1) Obtain CT image dataset

[0038] The data used in this embodiment is from the LIDC / IDRI dataset, including 1018 CT image sequences, and the present application selects at least 1055 nodules labeled by three radiologists.

[0039] (2) Preprocessing of the dataset

[0040] The preprocessing includes normalization operation and image cropping processing, forming 3D image blocks containing lung nodules.

[0041] This step normalizes the CT image data to the range of [0, 1] through preprocessing, wherein the value of HU greater than 400 in the CT image is defined as 1, and the value of HU less than -1000 is defined as 0. Then the image containing the lung nodule is cropped into a 3D image block, and the corresponding segmentation mask image is also cropped into a 3D image block of the same size, wherein the segmentation mask image comes from the nodule contour labeling information of the corresponding xml file in LIDC.

[0042] (3) Model construction

[0043] The lung nodule segmentation model is based on a 3D U-Net network, including an encoding part and a decoding part, and in the decoding part of the 3D U-Net, multiple feature maps of different scales can be obtained. These feature maps of different scales are resampled to the same size as the original image, and the smallest scale feature map output in the decoding part of the 3D U-Net is taken as the basic output of the network. Other scale feature maps are sequentially fused with the output results of relatively smaller feature maps according to the scale size, to further refine, supplement and optimize the segmentation effect. Each level output fuses the output of the smaller scale feature map as the final output of this level, and the last segmentation result is taken as the final prediction result of the model. Figure 1

[0044] In this embodiment, the structure of the lung nodule segmentation model based on deep learning is as shown in Figure 2 , including four segmentation prediction results, from front to back, sequentially refining and supplementing the previous prediction results. Finally, the last segmentation result (Output4) in the four prediction results is taken as the final prediction result of the model.

[0045] (4) Model training

[0046] The preprocessed 3D image block is input into the multi-scale output deep learning segmentation model for training, to obtain the optimal parameters of the segmentation model.

[0047] ​In the training process, the outputs of each level of the segmentation network are obtained, and the batch average dice loss function is used to optimize the outputs of each level. The optimization method is to calculate the loss value of each level output and the segmented label image by using the batch average dice loss function respectively, and the loss of each level is weighted to obtain the final loss value, and the obtained final loss value is used to update the parameters of the segmentation model. After iteration, the optimal model parameters are saved.

[0048] In this embodiment, the multi-scale output prediction results are obtained by using the model constructed and trained by the optimal parameters obtained above, and each prediction result is fused. Finally, the last segmentation result (Output4) in the prediction result is taken as the final prediction result of the model.

[0049] In the above method, first, the CT image data and segmentation annotation results are obtained from the LIDC data set, and the LIDC data is preprocessed and the image block is cropped. Then, the lung nodule fine segmentation model based on deep learning is constructed, and finally the most fine segmentation result is taken as the final segmentation result of the model, so as to realize more accurate segmentation of lung nodule in lung.

[0050] As shown in Table 1, compared with the 3D U-Net method, the lung nodule fine segmentation method based on deep learning improves the dice coefficient, sensitivity and ppv.

[0051] Table 1: Comparison table of segmentation result evaluation

[0052] dice coefficient sensitivity ppv 3D U-Net 0.8294 0.8395 0.8574 refinement segmentation method 0.8479 0.8500 0.8711

[0053] If the above method is realized in the form of a software function unit and sold or used as an independent product, it can be stored in a computer readable storage medium. Based on this understanding, the technical solutions of the present application essentially or the part that contributes to the prior art or part of the technical solutions can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes a plurality of instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: a U disk, a mobile hard disk, a read-only memory (ROM, Read-Only Memory), a random access memory (RAM, Random Access Memory), a magnetic disk or an optical disk, and various media that can store program codes.

[0054] Embodiment 2

[0055] The electronic device of one embodiment for implementing fine segmentation of a lung nodule includes one or more processors, a memory, and one or more programs stored in the memory, the one or more programs including instructions for performing the method described in Embodiment 1.

[0056] The preferred embodiments of the present application are described in detail above. It should be understood that modifications and variations can be resorted to without departing from the spirit of this application, as described above. Accordingly, the scope of the application is governed by the following claims and their equivalents in which references to "a" and "an" should be construed as referring to "one or more" unless otherwise indicated.

Claims

1. A deep learning-based lung nodule fine segmentation method, characterized in that, The method comprises the following steps: constructing a lung nodule segmentation model based on deep learning and multi-scale output; obtaining a lung CT image to be segmented, pre-processing the lung CT image to be segmented, and obtaining a 3D image block to be segmented; taking the 3D image block to be segmented as the input of the lung nodule segmentation model to obtain different scale output results; performing fusion processing on each different scale output result to obtain a final segmentation result; the fusion processing is: taking the output result of the smallest scale as the basic output, and sequentially fusing the output results of other scales with the scale being relatively smaller from small to large, and taking the last output result as the final segmentation result; when training the lung nodule segmentation model, updating the model parameters based on a batch average dice loss function, calculating the loss value of each scale output result and a segmented label image by using the batch average dice loss function, weighting the loss values of each scale to obtain a final loss value, and updating the model parameters by using the final loss value; the expression of the batch average dice loss function is: in the formula: Q is the number of samples in each batch, w, h and l respectively correspond to the length, width and height of the 3D image block, g corresponds to the foreground area in the label image, and e is the prediction result corresponding to the deep learning model based on three-dimensional context information.

2. The deep learning-based lung nodule refined segmentation method according to claim 1, wherein, The pre-processing includes normalization operation and image cropping processing.

3. The deep learning-based lung nodule refined segmentation method according to claim 2, wherein, The normalization operation is specifically: the value of HU in the CT image greater than 400 is defined as 1, and the value of HU less than -1000 is defined as 0.

4. The deep learning-based lung nodule refined segmentation method according to claim 1, wherein, The lung nodule segmentation model takes 3D U-Net network as the basic structure, and multiple different scale outputs are introduced in the decoding part of the 3D U-Net network.

5. The deep learning-based lung nodule refined segmentation method according to claim 1, wherein, The data set used when training the lung nodule segmentation model includes CT images and corresponding segmentation mask images formed based on annotation information. 6.A lung nodule fine segmentation device based on deep learning, characterized in that, comprise: one or more processors; memory; and one or more programs stored in the memory, the one or more programs comprising instructions for performing the lung nodule fine segmentation based on deep learning as claimed in any one of claims 1-5.

7. A computer readable storage medium characterized in that, comprise one or more programs for one or more processors of an electronic device, the one or more programs comprising instructions for performing the lung nodule fine segmentation based on deep learning as claimed in any one of claims 1-5.

Images