Deep learning-based lung disease diagnosis method

Abstract

The invention provides a lung disease diagnosis method based on deep learning. According to the method, focus segmentation and pathological classification tasks of a lung CT image are processed through a single model and limited resources. According to the method, slices under different visual angles are input into a model, compared with conventional convolution, a multi-scale expansion extrusion excitation module in the model pays more attention to channel information, small feature information of a multi-scale focus can be extracted, and semantic information under different receptive fields can be obtained; the model adopts a multi-task joint learning method, and information sharing between segmentation and classification tasks is enhanced, so that the network is trained more flexibly and accurately, and a more accurate prediction result is obtained. According to the method, the marking labor force of doctors is relieved, the CT image of the patient is diagnosed more accurately, the segmentation precision is high, and the model structure is simple and stable.

Description

Technical Field

[0001] The present invention belongs to the field of deep learning technology and is applied to medical image segmentation and classification. It specifically relates to a lung CT image disease diagnosis method based on a multi-scale expansion and compression excitation module and multi-task joint learning. Background Art

[0002] Lung disease is one of the major health problems worldwide, including lung cancer, pneumonia, tuberculosis, chronic obstructive pulmonary disease, etc. These diseases increase the risk of severe illness and death in patients, and bring a huge burden to personal health and public health. Therefore, early and accurate diagnosis and location of lung lesions are crucial for treatment and prognosis; In traditional medical imaging diagnosis, doctors usually rely on CT (computed tomography) or X-rays to evaluate lung lesions. These images often contain a lot of information, and doctors need to spend time and effort to observe and analyze them. Due to the influence of personal experience and other aspects, there may be subjectivity and errors. Therefore, with the help of image processing technology, the appearance, size and position of the lesion area can be automatically identified from the CT image, and automatic screening to identify and segment the lesion area is an urgent problem to be solved. Medical image processing methods based on deep learning have become the mainstream research direction, and the accuracy of medical image classification and lesion segmentation has been continuously improved to replace manual segmentation. More intuitive visualization methods reduce the workload of doctors, avoid subjective judgment, and improve the difficulties brought about by medical standards; Currently, researchers complete the single task of lesion segmentation or pathology classification by using 2D slices or 3D images as model input. Although 3D models can utilize the overall 3D structural information, they are often more expensive to compute and deploy, and the reasoning process is slow, which makes these models less general and difficult to generalize. Among the current networks, there are few networks that can complete both lesion segmentation and pathology classification tasks at the same time. Summary of the invention

[0003] The present invention aims to overcome the above problems of the prior art and proposes a lung disease diagnosis method based on deep learning for accurate lesion segmentation and pathological classification in 2D slices of lung CT images. The main steps of the method are as follows: Step 1, obtain the IEEE8023 dataset and the LUDB dataset; Step 2: Data preprocessing: slice extraction is performed on the sagittal, transverse, and coronal planes of each 3D image in the data set, and the data-enhanced 2D images are saved as training sets, validation sets, and test sets in a ratio of 8:1:1, all of which include segmentation labels and classification labels. Step 3, construct the lung disease diagnosis network MSU-Net; Step 4: Use the three-view training set data to train the lung disease diagnosis model respectively, and output the slices of lesion segmentation and pathology classification; Step 5: Evaluate the segmentation and classification accuracy of the three models on the validation set, adjust the hyperparameters, obtain the average segmentation and classification indicators of each model, and select the best model; Step 6, performing slice extraction operations on the 3D CT images to be diagnosed in the test set according to the cross-section, coronal and sagittal planes and inputting them into the obtained model to obtain the prediction results; Step 7: stack and fill the slices of the prediction results to obtain the 3D CT segmentation results and the prediction classification results as the final diagnosis results. BRIEF DESCRIPTION OF THE DRAWINGS

[0004] Figure 1 is a process flow chart of the present invention; Figure 2 These are slices from three angles of view of the same case; Figure 3 This is the network structure diagram of MSU-Net proposed by the present invention; Figure 4 This is a structural diagram of the multi-scale expansion and extrusion excitation module proposed by the present invention; Figure 5 (a) is the operation structure diagram of the multi-scale expansion and extrusion excitation module proposed in the present invention, and (b) is the expansion convolution parameters with expansion rates of 2, 4, 6, and 8; Figure 6 To utilize the present invention to process Figure 2 Diagnostic results after cross-sectional sectioning. DETAILED DESCRIPTION

[0005] In order to better understand the present invention, the segmentation method of the present invention is described in more detail below in combination with a specific implementation method. In the following description, the detailed description of the existing prior art may dilute the subject content of the present invention, and these descriptions will be ignored here. A lung CT image disease diagnosis method based on a multi-scale expansion and compression excitation module and multi-task joint learning does not need to improve the equipment configuration, but can improve the accuracy. The specific implementation steps are as follows: Step 1, obtain the IEEE8023 dataset and the LUDB dataset. The IEEE8023 dataset contains 3D scanned CT images of 20 patients, and has both pneumonia classification and lesion segmentation labels; the LUDB dataset contains 95 3D scanned CT images for pulmonary nodule lesion segmentation. The two datasets are divided into training set, test set and validation set in a ratio of 8:1:1 and stored in 101; Step 2, perform the same slice extraction operation and data augmentation 102 on the divided training set, test set and validation set. First, slice extraction operations are performed on the 3D image in the direction of the cross section, coronal plane and sagittal plane, including flipping and rotation (horizontal or vertical flipping, random angle rotation), image transformation (such as color transformation or affine transformation) data enhancement, expanding the data set to prevent model overfitting, and extracting slice images such as Figure 2 As shown; then the obtained two-dimensional slice image data is saved as three data sets, X2dLung, Y2dLung and Z2dLung. Each data set contains three new training sets, validation sets and test sets, which are stored in four folders, namely healthy lungs, common pneumonia, novel coronavirus infection and lung nodules, as classification labels; Step 3: Build a lung disease diagnosis network MSU-Net103 based on a multi-scale expansion and compression excitation module and a multi-task learning method, such as Figure 3 The overall network structure is divided into three parts: cross-task feature subnet, lesion segmentation subnet and pathology classification subnet. The cross-task feature subnet and lesion segmentation subnet both contain multi-scale expansion extrusion excitation modules, forming a U-shaped structure similar to the encoder-decoder. The structure diagram is shown in Figure 4 As shown, the model is forced to enhance the network receptive field while paying more attention to key areas and channel information to extract more subtle features; the operation structure is as follows Figure 5 ,Mathematically, the multi-scale expansion and extrusion excitation module is defined as: in Represents input image feature data; Represents a convolution unit with a convolution kernel size of 3×3; Represents a batch normalization operation; express Activation function; surface Show The convolution kernel size is 3×3 and the dilation rate is The convolution unit ； It means that the expansion rate of the dilated convolution is The image features extracted when Indicates points The convolution kernel is used as n The grouped convolutional unit; Represents a serial concatenation operation; represents the global average pooling operation; Represents a convolution unit with a convolution kernel size of 1×1; represents the feature weight extracted by dilated convolution, Feature map representing the output of the multi-scale expansion and squeeze excitation module; The cross-task feature subnet consists of four multi-scale dilation, squeezing and excitation modules, four 2×2 maximum pooling layers and four 3×3 convolutions. After the original image is input into the cross-task feature subnet, it first passes through 4 iterations of a 2×2 maximum pooling layer with a step size of 1, a 3×3 convolution layer and a multi-scale dilation, squeezing and excitation module to obtain multi-level semantic features of sizes 256×256 pixels, 128×128 pixels, 64×64 pixels and 32×32 pixels respectively. The lesion segmentation subnet consists of four multi-scale dilation, squeezing and excitation modules, four bilinear difference upsampling layers and two standard convolution layers. Each upsampling obtains high-level semantic features and jumps with the low-level semantic features obtained by the cross-task feature subnet to restore the details of the lung area, and then outputs them through standard convolution. After the feature map passes through the decoder, it passes through a 1×1 convolution layer and a sigmoid activation layer to output a segmentation result prediction map with the same resolution as the input image. The pathology classification subnet is mapped from the feature map obtained by the cross-task feature subnet to a 1*1*128 feature vector through convolution and global pooling layers. The segmentation prediction results are additionally introduced to improve the classification ability and enhance the interpretability. The predicted feature map of the lesion area is compressed to a size of 24*24 through the maximum pooling, and then concatenated with the feature vector mapped by the feature subnet through the fully connected layer, and finally outputs the prediction results of 4 categories; Step 4, using the saved training sets of the X2dLung, Y2dLung and Z2dLung data sets to train the 2D diagnostic models respectively, to obtain the cross-sectional diagnostic model, the coronal diagnostic model and the sagittal diagnostic model, each model output is the entire lesion area under the corresponding viewing angle slice and the predicted category 104 to which the lesion belongs; Step 5, verify the trained diagnostic models of the three planes on the validation set, obtain the average segmentation index and classification index 105 of each model, and adjust the hyperparameters to obtain the optimal model; The segmentation evaluation index is the Dice Similarity Coefficient (DSC): DSC is used to measure the similarity between the predicted lesion area and the labeled lesion area; DSC is defined as follows: in A Represents the data after model segmentation. B Segmented data representing facts, |A ∩B| yes A and B The intersection between |A| and |B| Respectively A and B The number of elements of , which provides the number of elements in a set; the dice similarity coefficient of the entire lesion area is calculated separately according to this formula; The Kappa coefficient is selected as the classification evaluation index to evaluate the lesion classification model. It is a statistical indicator used to evaluate the consistency of classification or judgment results. It takes into account the accuracy caused by random assignment and measures the consistency between the classification results and random selection. The value is between -1 and 1. The higher the value of the Kappa coefficient, the better the consistency of the lesion classification results; the lower the value, the poorer the consistency of the lesion classification results; the definition is as follows: in, Represents the accuracy of the observed classification results, calculated as the number of samples whose lesion type predictions are consistent with the actual results divided by the total number of samples. Indicates the accuracy of the expected lesion type prediction, which is calculated as the probability that the classifier's prediction result is consistent with the actual result under random conditions. The Kappa coefficient of pathological classification is calculated according to this formula; Step 6, input the data in the three test sets into the corresponding models to obtain the segmentation result slices and classification results 106, such as Figure 6 Shown Figure 2 The predicted result of the cross-sectional slice was that the diagnosis was common pneumonia; Step 7: stack the segmentation result slices of each model to obtain a 3D CT segmentation result, thereby obtaining the lung 3D CT image diagnosis result 107.

[0006] The present invention designs a multi-scale expansion and compression excitation module and introduces a multi-task joint learning method. While enhancing the algorithm segmentation accuracy, it does not put forward higher requirements on the equipment. It successfully segments the lung lesion area and pathological classification. The algorithm of the present invention is simple, highly operable, and stable to reproduce.

[0007] Although the above describes an illustrative specific embodiment of the present invention, it should be clear that the present invention is not limited to the scope of the specific embodiment. For ordinary technicians in this technical field, as long as various changes are within the spirit and scope of the present invention as defined and determined by the attached claims, these changes are obvious, and all inventions and creations using the concept of the present invention are protected.

Claims

1. A lung disease diagnosis method based on deep learning, characterized in that: The network structure adopts a multi-task joint learning method to achieve information sharing between tasks and improve the prediction ability of a single task; a multi-scale expansion and extrusion excitation module structure is designed when extracting image detail features to enhance the model's receptive field and information extraction capabilities. The specific steps include: Step 1, obtain the IEEE8023 dataset and the LUDB dataset; Step 2: Perform slice extraction operations on the sagittal, transverse, and coronal planes of each 3D image in the dataset, and save the data-enhanced 2D images as training sets, validation sets, and test sets in a ratio of 8:1:1, all of which include segmentation labels and classification labels; Step 3: Construct the lung disease diagnosis network MSU-Net. The overall network structure consists of three subnets: cross-task feature subnet, lesion segmentation subnet, and pathology classification subnet. Both the cross-task feature subnet and the lesion segmentation subnet contain multi-scale expansion extrusion excitation modules, which pay more attention to the channel information to extract image detail features; mathematically, the multi-scale expansion extrusion excitation module is defined as ; in Represents input image feature data; Represents a convolution unit with a convolution kernel size of 3×3; Represents a batch normalization operation; express Activation function; express The convolution kernel size is 3×3 and the dilation rate is The convolution unit ； It means that the expansion rate of the dilated convolution is The image features extracted when Indicates points The convolution kernel is used as n The grouped convolutional unit; Represents a serial concatenation operation; represents the global average pooling operation; Represents a convolution unit with a convolution kernel size of 1×1; represents the feature weight extracted by dilated convolution, Feature map representing the output of the multi-scale expansion and squeeze excitation module; The cross-task feature subnet consists of four multi-scale expansion excitation modules, four 2×2 maximum pooling layers, and four 3×3 convolutions; it is used to extract multi-level semantic features. The lesion segmentation subnet consists of four multi-scale expansion excitation modules, four bilinear difference upsampling layers, and two standard convolution layers, which are used to segment lesions. The pathology classification subnet uses the low-level semantic features of the feature subnet and the prediction result feature map of the lesion segmentation subnet for classification; Step 4: Use the three-view training set data to train the lung disease diagnosis model respectively, and output the slices of lesion segmentation and pathology classification; Step 5: Input the data in the validation set into the trained segmentation model, obtain the average segmentation index (Dice similarity coefficient) and classification index (Kappa coefficient) of each model, and adjust the hyperparameters to obtain the optimal model; Step 6: Input the data in the test set into the corresponding model to obtain the predicted segmentation result slices and classification results; Step 7: stack the segmentation result slices of each model to obtain a 3D CT segmentation result, thereby obtaining the 3D CT image diagnosis result of the lung.

2. A lung disease diagnosis method based on deep learning according to claim 1, characterized in that: When building the network, a multi-scale expansion and squeezing excitation module was designed, and combined with a multi-task joint learning method to enhance the network's receptive field and information extraction capabilities, strengthen information sharing between tasks, and obtain more accurate diagnostic results.

Images