A lung nodule classification method based on multi-task learning

Abstract

The application discloses a lung nodule classification method based on multi-task learning, comprising the following steps: 1, extracting lung nodule CT data and feature level labels; 2, calculating the Spearman rank correlation coefficient between lung nodule features, constructing and training a neural network so that the cosine similarity between the initial labels embedded in the output lung nodule features is approximately equal to the Spearman rank correlation coefficient; 3, designing an adaptive method to obtain a graph adjacency matrix to describe the correlation between lung nodule features; 4, constructing and training a multi-task model with GCN and 3D U-Net as the backbone network, containing an image fusion module and a cross-channel attention module, outputting the malignant degree classification result of the lung nodule, and giving the feature score and segmentation result to provide auxiliary information for the classification result. The application fully utilizes the correlation between lung nodule features to construct a lung nodule classification model, which can output multi-dimensional analysis results for lung nodules, so that the diagnosis result has high interpretability and credibility.

Description

Technical Field

[0001] This invention belongs to the field of medical image processing, specifically relating to a lung nodule classification method based on multi-task learning. Background Technology

[0002] Lung cancer has the highest incidence and mortality rates among cancers. Early screening for lung cancer is beneficial in reducing its incidence and mortality. However, due to limited medical resources, large-scale lung cancer screening relying solely on manual image interpretation is difficult. A significant proportion of early-stage lung cancers present as pulmonary nodules, and artificial intelligence plays a crucial role in improving medical quality and service efficiency, and reducing misdiagnosis and mistreatment. Therefore, designing AI-based pulmonary nodule-assisted diagnostic models has become a key focus in the development of automated lung cancer screening technology. Medically, the malignancy of pulmonary nodules is often diagnosed based on their morphological characteristics on CT images. However, existing assisted diagnostic models ignore the correlations between pulmonary nodule features and cannot provide doctors with explanatory information for the current diagnosis, resulting in low reliability of the model output and limiting the practical application of the models. Summary of the Invention

[0003] To address the problems of existing AI-assisted diagnostic methods failing to utilize the correlations between lung nodule features and exhibiting low accuracy and reliability of diagnostic results, this invention proposes a lung nodule classification method based on multi-task learning. This method fully utilizes the correlations between lung nodule features and employs a multi-task strategy, introducing a lung nodule feature scoring task and a three-dimensional lung nodule segmentation task to assist in classifying benign and malignant nodules.

[0004] The technical solution adopted in this invention is:

[0005] A lung nodule classification method based on multi-task learning includes the following steps:

[0006] S1 collects a dataset of CT images labeled with the feature levels of lung nodules, extracts the CT data of lung nodules according to the labels, and generates a mask.

[0007] The characteristics of the lung nodules include fineness, roundness, margins, lobulation, spiculation, texture, calcification, and malignancy.

[0008] S2 calculates the Spearman rank correlation coefficients between lung nodule features and combines them into a Spearman rank correlation coefficient matrix. ;

[0009] S3 is the Spearman rank correlation coefficient matrix obtained from S2. Design a neural network and loss function to obtain initial label embeddings for eight lung nodule features;

[0010] S4 designs an adaptive graph adjacency matrix acquisition method based on cosine similarity. The method yields a graph adjacency matrix that accurately represents the correlation between lung nodule features;

[0011] S5 constructs a multi-task model for lung nodule classification;

[0012] The multi-task model for lung nodule classification mainly includes a label embedding extraction branch, an image feature extraction branch, a feature fusion module, and a cross-channel attention module; the lung nodule CT data collected in S1 is used as the input of the image feature extraction branch, and the initial label embedding of the lung nodule features obtained in S3 and the graph adjacency matrix obtained in S4 are used as the input of the label embedding extraction branch.

[0013] S6 designs the loss function for a multi-task model of lung nodule classification;

[0014] The lung nodule CT data from step S1, the label embedding of lung nodule features obtained in step S3, and the graph adjacency matrix obtained in step S4 are input into the lung nodule classification multi-task model and trained using five-fold cross-validation. The model outputs lung nodule malignancy classification, lung nodule feature scores, and lung nodule 3D segmentation results.

[0015] Furthermore, in S2, the original lung nodule features and characteristics All level designations are sorted in descending order. For the first level... Characteristics of lung nodules and characteristics The level is marked as and The ranks after sorting are respectively and , Given the number of lung nodules in the training set, then the lung nodule features... and lung nodule characteristics Spearman rank correlation coefficient The calculation formula is shown in equation (1):

[0016]

[0017] Furthermore, the neural network in S3 is a four-layer fully connected neural network, comprising one input layer, two hidden layers, and one output layer, wherein the parameters of each neuron in each layer are 8, 100, 200, and 50.

[0018] Furthermore, the loss function in S3 The expression for is shown in equation (2):

[0019]

[0020] In the formula, and Characteristics of pulmonary nodules and The initial tag embedding, for Transpose to obtain Characteristics of pulmonary nodules and The Spearman rank correlation coefficient is The number of lung nodule features is .

[0021] Furthermore, the specific operation of adaptively obtaining the graph adjacency matrix in S4 is shown in equation (3):

[0022]

[0023] in, and The parameters are for two 1×1 convolution kernels. Represents the convolution operation. Indicates pulmonary nodules The initial label embedding matrix of each feature. They represent pulmonary nodules. Initial label embedding of each feature, d represents the dimension of the initial label embedding matrix; row vectors and column vectors These represent the lung nodule features after 1×1 convolution. and The embedding of tags, To obtain the lung nodule features after two 1×1 convolutions and characteristics The cosine similarity of the embedded labels is used as the corresponding position in the graph adjacency matrix. The value of is shown in equation (4):

[0024]

[0025] Furthermore, in step S5, a four-layer graph convolutional neural network is used as the label embedding extraction branch, and the U-Net encoder is used as the image feature extraction branch. The input to the image feature extraction branch is 48×48×48 mm. 3 CT data blocks of lung nodules.

[0026] Furthermore, in S5, the feature fusion module uses matrix multiplication to fuse the embedded labels with the image features, and finally adds the fused features to the original image features element-wise to obtain the final encoded features; the function of the feature fusion module is shown in equation (5):

[0027] (5)

[0028] in, For the first Layer encoder features It is the first one extracted by the 3D U-Net encoder. Layer image features, The first one extracted from GCN Layer tag embedding, for Transpose to obtain . For a 1×1×1 convolution, Represents the matrix transpose operation. and This represents the deformation operation of the feature map. It is the ReLU activation function. Representative feature transformation operation. Represents the matrix dimension. These are the number of channels, depth, height, and width of the feature, respectively. This represents the number of pulmonary nodule features.

[0029] Furthermore, in S5, the cross-channel attention module aggregates the encoder features and decoder features after the feature fusion module, and its operation is shown in equation (6):

[0030] (6)

[0031] in, The channel weighted first Layer encoder features, the first Layer encoder features Decoder features , For channel weight vectors, This is a global average pooling operation. It is the Sigmoid activation function. and These represent the parameters of the fully connected layer.

[0032] Furthermore, the loss function in S6 CE Loss is used, and Smooth L1 Loss is used for the loss function of each feature scoring task for lung nodules. The overall loss function for the feature scoring task is... To average the loss function for scoring tasks of detail, roundness, edges, lobulation, burrs, texture, and calcification features, The expression is shown in equation (7), where Loss function for 3D segmentation of lung nodules Dice Loss is used;

[0033] (7)

[0034] The loss function of the multi-task model for lung nodule classification is designed by linear weighted summation, and the weights of the loss function for lung nodule malignancy classification, feature scoring and three-dimensional segmentation tasks are adjusted by an adaptive weight adjustment method based on homoscedasticity uncertainty. The specific expression is shown in Equation (8).

[0035] (8)

[0036] in, , and For learnable parameters, Parameters for a multi-task model for classifying lung nodules.

[0037] The beneficial effects of this invention are as follows: This invention extracts the correlations between lung nodule features, including detail, roundness, edge, lobulation, spiculation, texture, calcification, and malignancy, and explicitly applies them to the model training process. This makes the model better at classifying benign and malignant lung nodules. Furthermore, while classifying the malignancy of lung nodules, the model provides scores for detail, roundness, edge, lobulation, spiculation, texture, and calcification features, as well as three-dimensional segmentation results of lung nodules. Combining the information from three-dimensional segmentation and lung nodule feature scores can provide doctors with comprehensive and reliable diagnostic evidence, thus increasing the model's practical application value. Attached Figure Description

[0038] Figure 1 It is a neural network that obtains the initial label embedding of lung nodule features.

[0039] Figure 2 This is a flowchart of the adaptive graph adjacency matrix construction method.

[0040] Figure 3 This is a structural diagram of a multi-task model for classifying lung nodules.

[0041] Figure 4 It is the feature fusion module.

[0042] Figure 5 It is a cross-channel attention module. Detailed Implementation

[0043] The present invention will be further described in detail below with reference to the accompanying drawings and specific implementation examples. The method for implementing a multi-task model for lung nodule classification according to the present invention mainly includes the following steps:

[0044] S1 collects CT image datasets labeled with lung nodule feature levels, extracts lung nodule CT data and lung nodule feature levels based on doctor annotations, and generates a mask; the lung nodule features include detail, roundness, edge, lobulation, spiculation, texture, calcification, and malignancy.

[0045] S1.1 Extract CT data of lung nodules.

[0046] For the 1018 CT images in the LIDC-IDRI lung nodule dataset, after excluding CT images with a slice thickness greater than 2.5 mm, the resolution was first standardized by interpolating all CT image pixels to 1×1×1 mm. 3 CT value limited to Normalize the data to Then, according to the doctor's markings, the lung nodule area was located and a 48×48×48 mm piece was cut. 3 Three-dimensional CT data of lung nodules of varying sizes are used to generate a mask.

[0047] S1.2 Collect lung nodule feature grade annotations. Lung nodules with excessively large differences in feature grade annotations were not considered when screening CT data. The average of the fineness, roundness, margin, lobulation, spiculation, texture, and calcification feature grades annotated by multiple doctors was taken. For malignancy features, if the average doctor-annotated grade was 3, the lung nodule was excluded; if the average grade was greater than 3, it was considered a malignant lung nodule; if it was less than 3, it was considered a benign lung nodule.

[0048] S1.3 If a pixel in a CT image is marked by a doctor, it is considered to be included in the lung nodule region. The pixel value of the lung nodule region is set to 1, and the values ​​of the other pixels are set to 0, thus generating a lung nodule mask.

[0049] S2 calculates the Spearman rank correlation coefficients between lung nodule features and combines them into a Spearman rank correlation coefficient matrix. .

[0050] Assuming the original lung nodule characteristics and characteristics All level designations are sorted in descending order. For the first level... Characteristics of lung nodules and characteristics The level is marked as and The ranks after sorting are respectively and , Given the number of lung nodules in the training set, then the lung nodule features... and lung nodule characteristics Spearman rank correlation coefficient The calculation formula is shown in equation (1):

[0051] (1).

[0052] S3 is the Spearman rank correlation coefficient matrix obtained from S2. Design a neural network and loss function to obtain the initial label embedding of lung nodule features.

[0053] S3.1 Design the loss function for the neural network.

[0054] Given lung nodule characteristics and characteristics The initial tag embedding is and ,right Transpose to obtain The Spearman rank correlation coefficient between the two is... The number of lung nodule features in the training set is The designed loss function As shown in equation (2), the initial tag embedding is guaranteed. and The cosine similarity is close to the Spearman rank correlation coefficient. .

[0055] (2)

[0056] S3.2 Designing Neural Networks.

[0057] use Figure 1 The network shown obtains the initial label embeddings of lung nodule features. Specifically, it is a four-layer fully connected neural network, and the network parameters are shown in Table 1. Finally, it obtains the initial label embeddings reflecting the correlation between lung nodule features.

[0058] Table 1. Parameters of the fully connected network for obtaining initial feature label embeddings.

[0059] Network layer Number of neurons Fully Connected Layer 1 – Input Layer 8 Fully Connected Layer 2 – Hidden Layer 100 Fully Connected Layer 3 – Hidden Layer 200 Fully Connected Layer 4 – Output Layer 50

[0060] S4 designs an adaptive graph adjacency matrix acquisition method based on cosine similarity. The method yields a graph adjacency matrix that accurately represents the correlation of lung nodule features.

[0061] Adaptive graph adjacency matrix acquisition based on cosine similarity The specific expression is shown in equation (3).

[0062] (3)

[0063] in, and The parameters are for two 1×1 convolution kernels. Represents the convolution operation. This represents the initial label embedding matrix of lung nodule features obtained through the fully connected neural network in S3. Let d represent the dimension of the initial label embedding matrix. This classification method extracts features such as detail, roundness, edge, lobulation, spiculation, texture, calcification, and malignancy. Based on the typical initial label embedding vector dimension of 50 in classic models, the dimension of the initial label embedding matrix is ​​set accordingly. The two parameters are independent of each other.

[0064] row vectors and column vectors These represent the lung nodule features after 1×1 convolution. and characteristics The embedding of tags, As shown in equation (4).

[0065] (4)

[0066] Equation (4) represents the calculation of the cosine similarity of the label embeddings of lung nodule features after 1×1 convolution as the graph adjacency matrix. The structure of the adaptive graph adjacency matrix construction method is as follows: Figure 2 As shown.

[0067] S5 is used to construct a multi-task model for lung nodule classification.

[0068] S5.1 In the aforementioned multi-task model for lung nodule classification, a four-layer graph convolutional neural network is used as the label embedding extraction branch, and a U-Net encoder is used as the image feature extraction branch. The input to the label embedding extraction branch is the initial label embedding of the lung nodule labels. And graph adjacency matrix The input for the image feature extraction branch is 48×48×48mm. 3 The CT data block of lung nodules. The specific structure of the multi-task model for lung nodule classification is as follows: Figure 3 As shown, the processing flow is as follows: In the encoding part, the U-Net encoder first extracts the first... Layer image features GCN extracts label embeddings of lung nodule features through graph convolution. Then, the image features and label embeddings are fused through the feature fusion module to obtain the model's first... Layer encoder features In the decoding section, a cross-channel attention module is used to weight the encoder features using information from the decoder features, and the weighted first channel attention module is then used to perform the final decoding. Layer encoder features With the Layer decoder features By splicing, we obtain the first... Layer decoder features Regarding the task output, the model uses a cross-activation module and a two-layer fully connected neural network to process the features. The results of the lung nodule malignancy classification, and scores for fineness, roundness, margin, lobulation, spiculation, texture, and calcification characteristics were obtained. and The convolution of the stitched features yields the three-dimensional segmentation results of lung nodules. Figure 3 The specific meanings of the symbols are shown in Table 2.

[0069] Table 2 Figure 3 Explanation of the meaning of Chinese characters

[0070]

[0071] The structure of the S5.2 feature fusion module is as follows: Figure 4 As shown, equation (5) illustrates the function of the feature fusion module. It uses matrix multiplication to fuse the embedded labels with the image features, and finally, the fused features are element-wise added to the original image features to obtain the final encoded features. Assume... For the first Layer encoder features It is the first one extracted by the 3D U-Net encoder. Layer image features, The first one extracted from GCN Layer tag embedding, for Transpose to obtain .

[0072] (5)

[0073] in, For a 1×1×1 convolution, Represents the matrix transpose operation. and This represents the deformation operation of the feature map. It is the ReLU activation function. Representative feature transformation operation. Represents the matrix dimension. These are the number of channels, depth, height, and width of the feature, respectively. This represents the number of pulmonary nodule features.

[0074] S5.3 Cross-Channel Attention Module, such as Figure 5 As shown, equation (6) illustrates the working principle of the cross-channel attention module. It is assumed that the first... Layer decoder features First, the encoder features and decoder features Global average pooling is performed, followed by a fully connected layer and an activation function to obtain the channel weight vectors of the encoder features and the decoder features, respectively. The two channel weight vectors are then added together to obtain the final channel weight vector, which is then applied to the encoder features. Perform channel-weighted calculation, and finally calculate the channel-weighted result. Layer encoder features With decoder features By splicing, we obtain the first... Layer decoder features .

[0075] (6)

[0076] in, This is a global average pooling operation. It is the Sigmoid activation function. and The parameters representing the fully connected layer, This is the channel weight vector.

[0077] S6 designs a loss function for a multi-task model of lung nodule classification.

[0078] The lung nodule CT data from step S1, the initial label embedding of lung nodule features from step S3, and the graph adjacency matrix from step S4 are input into the lung nodule classification multi-task model and trained using five-fold cross-validation.

[0079] S6.1 adopts a linear weighted summation method to design the loss function of the multi-task model for lung nodule classification, and uses an adaptive weight adjustment method based on homoscedasticity uncertainty to adjust the weights of the loss function for lung nodule malignancy classification, feature scoring and 3D segmentation tasks. , and For learnable parameters, The parameters of the multi-task model for lung nodule classification are given, and the loss function expression is shown in Equation (7).

[0080] (7)

[0081] in, The loss function for the lung nodule malignancy classification task is CE Loss. The loss function for the lung nodule feature scoring task is expressed as shown in equation (8).

[0082] (8)

[0083] in, The loss function for the lung nodule feature scoring task is the average of the Smooth L1 Loss for seven feature scoring tasks: fineness, roundness, edge, lobulation, spiculation, texture, and calcification. The loss function for the 3D segmentation task of lung nodules is Dice Loss.

[0084] S6.2 employs five-fold cross-validation in model training. First, 10% of the samples are randomly selected as the test set. The remaining samples are then divided into five equal parts, with one part selected as the validation set and the remaining part used as the training set. Flipping and rotating the samples by 90 degrees are used in the process. 。 The training set data was augmented using a specific method. The training set data was input into a multi-task model for lung nodule classification, and the model was trained using stochastic gradient descent. Simultaneously, an adaptive moment estimation algorithm was used to optimize the multi-task model. The network batch size was 64, the number of training iterations was 100, and the initial learning rate was 0.01. The learning rate was optimized during training using the ReduceLROnPlateau strategy in PyTorch with a weight reduction of 0.1. The model's convergence was determined based on the decreasing trend of the loss function value on the validation set, as well as whether underfitting or overfitting occurred. Finally, the optimal model parameters from each fold cross-validation were saved.

[0085] To evaluate the effectiveness of the proposed multi-task lung nodule classification model in lung nodule malignancy classification, feature scoring, and 3D segmentation tasks, its performance was compared with that of a 101-layer residual network (ResNet-101), a 121-layer densely connected network (DenseNet-121), an AS-GCVN model, and a Fuse-TSD model in the lung nodule malignancy classification task in terms of accuracy, recall, precision, F1 score, and area under the receiver operating characteristic curve (AUC). The specific results are shown in Table 3.

[0086] Table 3. Model performance on lung nodule malignancy classification task

[0087]

[0088] In the 3D segmentation task of lung nodules, the multi-task model was compared with UNet++, V-Net and NoduleNet in terms of Intersection over Union (IoU) and Dice Similarity Coefficient (DSC). The specific results are shown in Table 4.

[0089] Table 4. Model performance on lung nodule segmentation task

[0090]

[0091] In the lung nodule feature scoring task, the performance of ResNet-101, DenseNet-121, MTMR-Net and MTR models on the mean absolute distance (MAE) index for features such as detail, roundness, edge, lobulation, spiculation, texture and calcification is compared. The specific results are shown in Table 5.

[0092] Table 5. Model performance on lung nodule feature scoring task (mean ± standard deviation)

[0093]

[0094] According to Tables 3, 4, and 5, compared to previously proposed advanced methods for the auxiliary diagnosis of pulmonary nodules, the method of this invention achieves optimal results in all evaluation indicators except for the calcification feature scoring task. Therefore, the model proposed in this invention can more accurately classify benign and malignant pulmonary nodules, scoring features such as fineness, roundness, margin, lobulation, spiculation, texture, and calcification, and can effectively perform three-dimensional segmentation of pulmonary nodules. These conclusions demonstrate the excellent performance of the proposed multi-task pulmonary nodule classification model in pulmonary nodule malignancy classification, feature scoring, and three-dimensional segmentation tasks, and provide the basis for classifying benign and malignant pulmonary nodules, exhibiting high interpretability and reliability.

[0095] The advantages and innovations of this invention are mainly reflected in the following aspects:

[0096] (1) A method for obtaining label embedding of lung nodule features was designed. It can obtain label embedding containing lung nodule feature related information without collecting a corpus related to lung nodules and training a natural language processing model.

[0097] (2) A graph adjacency matrix construction method that is not affected by doctor annotation error is designed. The constructed adjacency matrix can accurately reflect the correlation between lung nodule features, which is beneficial for the label embedding branch to extract label embeddings that conform to the correlation between lung nodule features.

[0098] (3) The designed cross-channel attention module solves the problem that skip connections in U-Net cannot effectively aggregate encoder features and decoder features.

[0099] (4) Make full use of the correlation between lung nodule features to solve the problem that the current research only uses image data to train the model to handle the lung nodule malignancy classification task, resulting in the conclusions lacking interpretability and having low reliability.

[0100] (5) It simultaneously outputs the scores of lung nodule detail, roundness, edge, lobulation sign, spiculation sign, texture, calcification features, malignancy classification, and three-dimensional segmentation results of lung nodules, providing a comprehensive and accurate description of lung nodules, which has high application value.

[0101] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and not to limit it. Although the present invention has been described in detail with reference to preferred embodiments, the scope of protection of the present invention is not limited thereto. Any modifications or equivalent substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in the present invention, without departing from the spirit and scope of the technical solutions of the present invention, should be covered within the scope of protection of the present invention.

Claims

1. A lung nodule classification method based on multi-task learning, characterized in that, Includes the following steps: S1 collects a dataset of CT images labeled with the feature levels of lung nodules, extracts the CT data of lung nodules according to the labels, and generates a mask. The characteristics of the lung nodules include fineness, roundness, margins, lobulation, spiculation, texture, calcification, and malignancy. S2 calculates the Spearman rank correlation coefficients between lung nodule features and combines them into a Spearman rank correlation coefficient matrix. ; S3 is the Spearman rank correlation coefficient matrix obtained from S2. Design a neural network and loss function to obtain initial label embeddings for eight lung nodule features; S4 designs an adaptive graph adjacency matrix acquisition method based on cosine similarity. The method yields a graph adjacency matrix that accurately represents the correlation between lung nodule features; S5 constructs a multi-task model for lung nodule classification; The multi-task model for lung nodule classification includes a label embedding extraction branch, an image feature extraction branch, a feature fusion module, and a cross-channel attention module; the lung nodule CT data collected in S1 is used as the input of the image feature extraction branch, and the initial label embedding of the lung nodule features obtained in S3 and the graph adjacency matrix obtained in S4 are used as the input of the label embedding extraction branch. S6 designs the loss function for a multi-task model of lung nodule classification; The lung nodule CT data from step S1, the label embedding of lung nodule features obtained in step S3, and the graph adjacency matrix obtained in step S4 are input into the lung nodule classification multi-task model and trained using five-fold cross-validation. The model outputs lung nodule malignancy classification, lung nodule feature scores, and lung nodule 3D segmentation results.

2. The lung nodule classification method based on multi-task learning according to claim 1, characterized in that, In S2, the original lung nodule features and characteristics All level designations are sorted in descending order. For the first level... Characteristics of lung nodules and characteristics The level is marked as and The ranks after sorting are respectively and , Given the number of lung nodules in the training set, then the lung nodule features... and lung nodule characteristics Spearman rank correlation coefficient The calculation formula is shown in equation (1):

3. The lung nodule classification method based on multi-task learning according to claim 1, characterized in that, The neural network in S3 is a four-layer fully connected neural network, consisting of one input layer, two hidden layers, and one output layer, with the parameters of each neuron being 8, 100, 200, and 50.

4. The lung nodule classification method based on multi-task learning according to claim 1, characterized in that, The loss function in S3 The expression for is shown in equation (2): （2） In the formula, and Characteristics of pulmonary nodules and The initial tag embedding, for Transpose to obtain Characteristics of pulmonary nodules and The Spearman rank correlation coefficient is The number of lung nodule features is .

5. The lung nodule classification method based on multi-task learning according to claim 1, characterized in that, The specific operation of adaptively obtaining the graph adjacency matrix in S4 is shown in equation (3): in, and The parameters are for two 1×1 convolution kernels. Represents the convolution operation. Indicates pulmonary nodules The initial label embedding matrix of each feature. They represent pulmonary nodules. Initial label embedding of each feature, d represents the dimension of the initial label embedding matrix; row vectors and column vectors These represent the lung nodule features after 1×1 convolution. and The embedding of tags, To obtain the lung nodule features after two 1×1 convolutions and characteristics The cosine similarity of the embedded labels is used as the corresponding position in the graph adjacency matrix. The value of is shown in equation (4): 。 6. The lung nodule classification method based on multi-task learning according to claim 1, characterized in that, In step S5, a four-layer graph convolutional neural network is used as the label embedding extraction branch, and the U-Net encoder is used as the image feature extraction branch. The input of the image feature extraction branch is 48×48×48 mm. 3 CT data blocks of lung nodules.

7. The lung nodule classification method based on multi-task learning according to claim 1, characterized in that, The feature fusion module in S5 uses matrix multiplication to fuse the embedded labels with image features. Finally, the fused features are element-wise added to the original image features to obtain the final encoded features. The fusion module operates as shown in equation (5): in, For the first Layer encoder features It is the first one extracted by the 3D U-Net encoder. Layer image features, The first one extracted from GCN Layer tag embedding, for Transpose to obtain , For a 1×1×1 convolution, Represents the matrix transpose operation. and This represents the deformation operation of the feature map. It is the ReLU activation function. Representative feature transformation operation. Represents the matrix dimension. These are the number of channels, depth, height, and width of the feature, respectively. The number of pulmonary nodule features.

8. The lung nodule classification method based on multi-task learning according to claim 1, characterized in that, The cross-channel attention module in S5 aggregates the encoder features and decoder features after the feature fusion module, and its operation is shown in equation (6): in, The channel weighted first Layer encoder features, the first Layer encoder features Decoder features , This is the channel weight vector. This is a global average pooling operation. It is the Sigmoid activation function. and These represent the parameters of the fully connected layer.

9. The lung nodule classification method based on multi-task learning according to claim 1, characterized in that, The loss function in S6 CE Loss is used, and the loss function for each feature scoring task of lung nodules is Smooth L1 Loss. The overall loss function for the feature scoring task is... To average the loss function for scoring tasks of detail, roundness, edges, lobulation, burrs, texture, and calcification features, The expression is shown in equation (7), where ; Loss function for 3D segmentation of lung nodules Dice Loss is used; The loss function of the multi-task model for lung nodule classification is designed by linear weighted summation, and the weights of the loss function for lung nodule malignancy classification, feature scoring and three-dimensional segmentation tasks are adjusted by an adaptive weight adjustment method based on homoscedasticity uncertainty. The specific expression is shown in Equation (8): in, , and For learnable parameters, Parameters for a multi-task model for classifying lung nodules.

Images