Method for segmenting positive lymph nodes in CT images based on multi-task learning

Abstract

The application is suitable for the technical field of CT image processing, and provides a segmentation method for positive lymph nodes in a CT image based on multi-task learning, comprising: preprocessing and enhancing data of a CT image sequence, constructing a multi-task deep learning model, extracting features through an encoder, performing a classification task by using a classification module, and segmenting positive lymph nodes by using a segmentation module combined with a classification result. The application is excellent in classification accuracy (up to 91.1%) and DICE coefficient of segmentation (up to 0.602), and is superior to a separate 2D segmentation model. Compared with a 3D segmentation model, the application maintains similar segmentation performance and is shorter in reasoning time. The application can simultaneously complete the classification and segmentation tasks for positive lymph nodes in a CT image, reduces the work burden of doctors, improves the diagnosis and treatment efficiency of doctors, and fills the blank of simultaneously judging and outlining positive lymph nodes in a head and neck CT image of a nasopharyngeal carcinoma patient.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of CT image processing, and particularly relates to a method for segmenting positive lymph nodes in a CT image based on multi-task learning. BACKGROUND

[0002] Nasopharyngeal carcinoma is a common disease among head and neck malignancies, and its morbidity and mortality rate is high worldwide. In the imaging diagnosis of nasopharyngeal carcinoma patients, there are some specific characteristics, among which lymph node enlargement is the most important sign, usually showing a diameter of more than 1 centimeter. In addition to size, the shape of the lymph node may also appear abnormal, such as presenting an irregular shape or a blurred outline. Furthermore, the change in the internal density of the lymph node is also an important imaging diagnostic indicator, which is often related to pathological changes within the lymph node. Accurate judgment and delineation of positive lymph nodes in head and neck medical images is crucial for subsequent diagnosis and treatment of doctors.

[0003] In actual medical practice, computed tomography (CT) is the most commonly used image type in the diagnosis and treatment of nasopharyngeal carcinoma patients. Head and neck CT scanning can help doctors detect and evaluate lymph node enlargement, tumors, infections, trauma and other lesions to improve the accuracy of diagnosis and treatment effect. However, due to the variable shape and size of head and neck lymph nodes, and the similarity to surrounding tissue structures, the contour boundary is not clear, resulting in a very time-consuming and labor-intensive diagnosis of lymph nodes. Traditionally, the target area judgment and delineation of lymph nodes are performed manually by clinicians. For each head and neck CT sequence, the number of slices is usually around 150, and the clinician first needs to judge whether there is a positive lymph node in each CT slice, and if so, use professional software to delineate its contour. Studies have shown that the average segmentation time for a single head and neck cancer case is 2.7 hours. With the continuous advancement of treatment, this process often needs to be repeated several times. In addition, the accuracy of segmentation is highly dependent on the knowledge, experience and preferences of clinicians, and there is often some deviation between different doctors, and a standard judgment and delineation usually requires the joint efforts of multiple doctors.

[0004] To solve this problem, deep learning has been applied in the delineation and segmentation of positive lymph nodes in head and neck CT. Some studies have used 2D or 3D deep learning models to segment positive lymph nodes, such as 2D U-Net, 3D U-Net, etc. The basic idea of these models is similar: use multi-layer convolutional operations in the encoder part, extract image features from local to global in order, and use pooling operations to reduce feature size at the same time, and then gradually restore the features to the original size through the transposed convolution or interpolation of the decoder part, and finally get the segmentation result. The difference is that 2D models usually use a single CT slice as input, while 3D models need multiple CT slices stacked as input. However, these models also have some problems. 2D models ignore the hidden information between slices, resulting in low segmentation accuracy. While 3D models consider the information between slices, they introduce a large amount of 3D calculation, requiring higher computing resources and longer training and inference time.

[0005] To solve the above problems, the present application proposes a segmentation method for positive lymph nodes in CT images based on multi-task learning. SUMMARY

[0006] The purpose of the present application is to provide a segmentation method for positive lymph nodes in CT images based on multi-task learning, aiming to solve the problems raised in the background art.

[0007] The purpose of the present application is achieved by the following technical solutions:

[0008] The segmentation method for positive lymph nodes in CT images based on multi-task learning comprises the following steps:

[0009] Step 1, preprocessing: pre-processing the acquired CT image sequence and adding data augmentation operation;

[0010] Step 2, construct a multi-task deep learning model, the model includes an encoder, a classification module and a segmentation module;

[0011] Step 3, feature extraction: use the encoder to extract features, the encoder uses a multi-layer convolutional neural network to extract local features of the image and gradually converges, while using a pooling operation to reduce the feature size, and finally obtaining the required features;

[0012] Step 4, classification task: use the classification module to perform the classification task, the classification module takes the features extracted in step 3 as input, and calculates the classification result of each feature;

[0013] Output 0 represents that the classification module believes that the CT slice corresponding to the feature has no positive lymph nodes, and output 1 represents that the classification module believes that the CT slice corresponding to the feature has positive lymph nodes;

[0014] Step 5, segmentation task: a segmentation module is used to perform a segmentation task, the segmentation module takes the features extracted in step 3 as input, obtains the segmentation result of each feature by restoring the feature size, and removes useless segmentation results through the classification result, and finally obtains the segmentation result of the CT slice with only positive lymph nodes.

[0015] Further, the preprocessing operation includes removing all background interference using an additional U-Net model, and then clipping the intensity in the range of [0, 300 HU] to retain the lymph node HU value, and normalizing the lymph node HU value to the range of [0, 1.0].

[0016] Further, the data enhancement operation includes random rotation, random up-down flipping and random left-right flipping.

[0017] Further, the encoder is composed of stacked 4-layer down modules, each down module includes twice convolution, batch normalization, activation function operation, and finally completes the down sampling operation through the maximum pooling layer; after the encoder processing, each CT slice obtains a corresponding feature vector.

[0018] Further, the classification module is composed of multiple 1x1 convolution layers and fully connected layers, each 1x1 convolution layer is used to reduce the dimension of the feature, and finally the feature is flattened and mapped to the output category through the fully connected layer to obtain the classification result.

[0019] Further, the segmentation module is composed of a fusion module and four up modules; the fusion module includes twice SEblock, 1x1 convolution, batch normalization and activation function operation; the SE block suppresses unimportant channels in the data and gives low weight, and encourages important channels in the data and gives high weight through squeezing and exciting operations; each up module includes twice convolution, batch normalization and activation function operation, and finally performs up sampling through transposed convolution; the input of each up module is composed of two parts, one part is the output of the previous layer network, and the other part is the output of the corresponding down module for the i-th slice.

[0020] Compared with the prior art, the beneficial effects of the present application are:

[0021] The application is excellent in classification accuracy (up to 91.1 %) and segmentation DICE coefficient (up to 0.602), and is superior to a single 2D segmentation model, while compared with a 3D segmentation model, the application has shorter inference time while keeping similar segmentation performance. In addition, the method provided by the application can complete the classification and segmentation tasks of positive lymph nodes in head and neck CT images at the same time, reduce the work burden of doctors, improve the diagnosis and treatment efficiency of doctors, fill the blank of simultaneously judging and outlining positive lymph nodes in head and neck CT images of nasopharyngeal carcinoma patients, and provide strong support for clinical diagnosis and treatment. BRIEF DESCRIPTION OF DRAWINGS

[0022] Figure 1 The flowchart of the method of the application.

[0023] Figure 2 The framework flowchart of the method of the application.

[0024] Figure 3 The structural schematic diagram of the down module.

[0025] Figure 4 The structural schematic diagram of the up module.

[0026] Figure 5 The structural schematic diagram of the fusion module. DETAILED DESCRIPTION

[0027] In order to have a clearer understanding of the technical features, objectives and beneficial effects of the application, the technical solutions of the application are described in detail as follows, but it should not be understood as a limitation on the implementable scope of the application.

[0028] The specific implementation of the application is described in detail in combination with specific embodiments.

[0029] One embodiment of the application provides a segmentation method of positive lymph nodes in a CT image based on multi-task learning, a flowchart of the method is shown in Figure 1 , and a framework flowchart is shown in Figure 2 . The method comprises the following steps:

[0030] Step 1, preprocessing: the obtained CT image sequence is preprocessed to adapt to the input of a deep neural network model, and a data enhancement operation is added to improve the generalization ability of the model;

[0031] The original CT image size is 512x512 pixels. Since the black background occupies most of the space, and the base of the CT device will also appear white after imaging, these will affect the segmentation effect of the model. Moreover, due to factors such as the patient's posture and body shape, the patient's head and neck occupy a variable position in the original CT picture, and it is difficult to remove the background part by using a fixed value of the cropping operation. Therefore, first, an additional U-Net model is used to remove all background interference, and then the intensity is cut in the range of [0, 300 HU] to retain the lymph node HU value, and the values are normalized to the range of [0, 1.0], and data enhancement operations such as random rotation, random up-down flip, and random left-right flip are added.

[0032] Step 2, constructing a multi-task deep learning model, the model including an encoder, a classification module and a segmentation module;

[0033] Step 3, feature extraction: using the encoder for feature extraction, the encoder uses a multi-layer convolutional neural network to extract local features of the image and gradually converges, while using a pooling operation to reduce the feature size, and finally obtaining the required features;

[0034] The encoder is composed of 4 stacked down modules (see Figure 3 ), each down module includes twice convolution, batch normalization, activation function operation, and finally completes the down-sampling operation through the maximum pooling layer; after the encoder processing, each CT slice obtains a corresponding feature vector.

[0035] Step 4, classification task: using the classification module to perform the classification task, the classification module takes the features extracted in step 3 as input, and calculates the classification result of each feature; output 0 represents that the classification module considers that the CT slice corresponding to the feature has no positive lymph node, and output 1 represents that the classification module considers that the CT slice corresponding to the feature exists positive lymph node;

[0036] The classification module is composed of multiple 1x1 convolution layers and fully connected layers, each 1x1 convolution layer is used to reduce the dimension of the feature, and finally the feature is flattened and mapped to the output category through the fully connected layer to obtain the classification result.

[0037] Step 5, segmentation task: using the segmentation module to perform the segmentation task, the segmentation module takes the features extracted in step 3 as input, and obtains the segmentation result of each feature by restoring the feature size, and then removes the useless segmentation result through the classification result, finally obtains the segmentation result of the CT slice only with positive lymph nodes;

[0038] The segmentation module is composed of a fusion module (see Figure 5 ) and four up modules (see Figure 4) composition. The fusion module includes twice SE block, 1x1 convolution, batch normalization, activation function operation; the SE block suppresses unimportant channels in data by squeezing and excitation operations, and gives them lower weights, and encourages important channels in data to give them higher weights. The structure of the up module is similar to that of the down module, except that the up module is finally up-sampled by transposed convolution. The input of each up module is composed of two parts, one part is the output of the previous layer network, and the other part is the output of the corresponding down module on the i-th slice, which is consistent with the original U-Net skip connection.

[0039] In this embodiment, the experimental data of this embodiment comes from 92 nasopharyngeal carcinoma patients who visited the radiotherapy department of the First Hospital of Jilin University from January 1, 2020 to December 31, 2023. All patients were guided by doctors to perform computer tomography examination in supine position, and accurate judgment and delineation were performed by experienced clinicians. In order to verify the performance of the model, the data set is divided into 71 training sets and 18 test sets. In the experiment, three different models are compared: the multi-task deep learning model proposed in the application, the 2D segmentation model alone, and the 3D segmentation model alone. These models are compared in execution time and related task indicators, and the specific results are shown in Table 1:

[0040] Table 1 Comparison results

[0041]

[0042] From the data results in Table 1, it can be seen that the classification accuracy of the multi-task deep learning model proposed in the application can reach 91.1%, the segmentation DICE coefficient can reach 0.602, and the average time of the whole process for a single patient is 102s. In contrast, the DICE coefficient of the 2D segmentation model performing only the segmentation task is only 0.537, and the average time for a single patient is 13.2s. Although its execution efficiency is higher than that of the application, its segmentation index is significantly lower than that of the application, and it cannot complete the classification task. Similarly, the DICE coefficient of the 3D segmentation model performing only the segmentation task is 0.610, and the average time for a single patient is 244s. The segmentation index is basically the same as that of the application, but the time consumption is more than twice that of the application, and it also cannot complete the classification task.

[0043] In summary, the application fills the blank of simultaneously judging and delineating positive lymph nodes in the head and neck CT images of nasopharyngeal carcinoma patients. Not only improves the accuracy of segmentation and classification, but also greatly reduces the workload of doctors, providing a certain auxiliary role for doctors.

[0044] The above are only preferred embodiments of the present application, it should be pointed out that, for those skilled in the art, without departing from the concept of the present application, can also make several variations and improvements, these should also be considered as the protection scope of the present application, these will not affect the effect and the practicality of the patent of the present application.

Claims

1. A method for segmenting positive lymph nodes in CT images based on multi-task learning, characterized in that, The method comprises the following steps: Step 1, preprocessing: preprocessing the obtained CT image sequence and adding a data enhancement operation; Step 2, constructing a multi-task deep learning model, the model comprising an encoder, a classification module and a segmentation module; Step 3, feature extraction: using the encoder to extract features, the encoder using a multi-layer convolutional neural network to extract local features of the image and gradually converging, while using a pooling operation to reduce the feature size, and finally obtaining the required features; Step 4, classification task: using the classification module to perform the classification task, the classification module taking the features extracted in step 3 as input, and obtaining the classification result of each feature by calculation; Output 0 represents that the classification module considers that the CT slice corresponding to the feature has no positive lymph node, and output 1 represents that the classification module considers that the CT slice corresponding to the feature has a positive lymph node; Step 5, segmentation task: using the segmentation module to perform the segmentation task, the segmentation module taking the features extracted in step 3 as input, obtaining the segmentation result of each feature by restoring the feature size, and then removing the useless segmentation result through the classification result, and finally obtaining the segmentation result of the CT slice with only positive lymph nodes; The encoder is composed of 4 stacked down modules, each down module comprising twice convolution, batch normalization, activation function operation, and finally completing the down sampling operation through the maximum pooling layer; after the encoder processing, each CT slice obtains a corresponding feature vector; The classification module is composed of multiple 1x1 convolution layers and fully connected layers, each 1x1 convolution layer being used to reduce the dimension of the feature, and finally the feature being flattened and mapped to the output category through the fully connected layer to obtain the classification result; The segmentation module is composed of a fusion module and four up modules; the fusion module comprises twice SE block, 1x1 convolution, batch normalization and activation function operation; the SE block suppresses unimportant channels in data and gives low weight, and encourages important channels in data and gives high weight through squeezing and exciting operations; each up module comprises twice convolution, batch normalization and activation function operation, and finally performs up sampling through transposed convolution; the input of each up module is composed of two parts, one part being the output of the previous layer network, and the other part being the output of the corresponding down module for the i-th slice.

2. The method of claim 1, wherein the method is based on multi-task learning. The preprocessing operation comprises using an additional U-Net model to remove all background interference, then clipping the intensity in the range of [0, 300HU] to retain the lymph node HU value, and normalizing the lymph node HU value to the range of [0, 1.0]. 3.The method of claim 1, wherein, The data enhancement operation comprises random rotation, random up-down flipping and random left-right flipping.

Images