Method for classifying benign and malignant thyroid nodules based on ti-rads multi-task neural network

Abstract

The application belongs to the technical field of deep learning of computer technology, and relates to a TI-RADS multi-task neural network-based thyroid nodule benign and malignant classification method; the technical problem to be solved is to provide an improved TI-RADS multi-task neural network-based thyroid nodule benign and malignant classification method; the technical solution for solving the above technical problem is as follows: image preprocessing; a convolutional neural network is constructed; DenseNets 121 is used as a backbone network, DenseNets 121 comprises four Dense Blocks and three Transition layers, and the Dense Block is composed of Dense Layers; an SGE module is inserted after a batch normalization layer of a bottleneck layer in each Dense Block; the full connection layer of the network has five branches, which correspond to five TI-RADS classification tasks respectively, and each branch comprises three linear layers; the convolutional neural network is trained; the pre-training weight of the first Dense Block is migrated during training, and the network weight is updated and solved through an optimization objective function; and a benign and malignant classification result of the thyroid nodule is output; the application is applied to thyroid nodule benign and malignant classification.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of a thyroid nodule benign and malignant classification method based on a TI-RADS multi-task neural network. BACKGROUND

[0002] A convolutional neural network is a kind of neural network containing convolution, pooling, activation function calculation and having a certain depth structure, and is one of representative algorithms in the field of deep learning. At present, it has been confirmed by a large number of research examples that it has strong performance in target classification, positioning and detection, and has made breakthrough progress in target classification with multi-level feature learning and rich feature expression ability.

[0003] In recent years, one after another classic network model has emerged in the field of image classification. GoogLeNet proposes an Inception module, which widens the network, thereby enhancing the feature extraction capability of the network from another direction; ResNet proposes a residual structure, which effectively alleviates problems such as gradient disappearance and network degradation, so that the neural network can reach a depth of hundreds of layers; DensenNet proposes a dense connection concept, which connects the input of each layer before the network to all layers after the network, thereby achieving efficient feature reuse and also slowing down problems such as gradient disappearance. In the field of thyroid nodule diagnosis based on deep learning, many scholars have applied the above-mentioned models to nodule diagnosis, but the model constructed by them is only a binary classification model of thyroid nodule benign and malignant, which directly gives the nodule as benign or malignant, ignoring the problem of weak explainability, resulting in a decrease in the degree of trust of doctors.

[0004] As described above, for the image classification task, there are various convolutional neural network frameworks, but in the thyroid nodule diagnosis task, the accuracy is still not satisfactory and the explainability is poor, which leads to many limitations in practical application. Generally speaking, the most direct way to improve network performance is to increase the depth and width of the network, but this will bring some defects: too many parameters, easy to overfit; the larger the network, the greater the computational complexity, which is difficult to apply; the deeper the network, the more likely the gradient is to disappear. SUMMARY

[0005] The application overcomes the deficiencies in the prior art and solves the technical problem of providing an improved thyroid nodule benign and malignant classification method based on a TI-RADS multi-task neural network.

[0006] To solve the above technical problems, the technical scheme adopted by the application is as follows: a thyroid nodule benign and malignant classification method based on a TI-RADS multi-task neural network, comprising the following steps:

[0007] S1: image preprocessing: the image preprocessing includes image denoising, obtaining artificial marking area, removing artificial marking of ultrasound image, extracting ROI area, image enhancement;

[0008] S2: constructing convolutional neural network: using DenseNets 121 as backbone network, the DenseNets 121 contains 4 Dense Blocks and 3 Transition layers, the Dense Block is composed of Dense Layer, and each Dense Layer contains batch normalization layer, rectified linear unit layer, 1x1 convolution layer, batch normalization layer, rectified linear unit layer and 3x3 convolution layer in turn;

[0009] SGE modules are inserted after the batch normalization layer of the bottleneck layer in each Dense Block;

[0010] The fully connected layer of the network has five branches, respectively corresponding to five TI-RADS classification tasks, and each branch contains three linear layers;

[0011] S3: training convolutional neural network: the pre-training weight of the first Dense Block is migrated during training, and the network weight is updated and solved through the optimization objective function;

[0012] S4: inputting the thyroid nodule image into the trained convolutional neural network to output the benign and malignant classification results of the thyroid nodule.

[0013] The image denoising in step S1 specifically adopts an adaptive median filter algorithm.

[0014] The artificial marking area is obtained by using YOLO V4 and MSER algorithm in step S1.

[0015] The fast marching algorithm is used to filter out the artificial marking of the ultrasound image in step S1.

[0016] The 4 Dense Blocks in step S2 are respectively composed of 6, 12, 24 and 16 Dense Layers.

[0017] Each branch of the fully connected layer in step S2 contains three linear layers, wherein the number of neurons of the linear layer L1 is 1024, the number of neurons of the linear layer L2 is 512, and the number of neurons of the linear layer L3 is the number of categories corresponding to the classification task.

[0018] The SGE module regulates the importance of sub-features by generating attention factors for each spatial position of each semantic group, and the attention factors are generated by the similarity between global and local feature descriptors in each group.

[0019] The initial 10 cycles are trained in the step S3 when training the neural network, the learning rate is set to 0.0001, the network weight is preheated, then the learning rate is increased to 0.001, and the cosine annealing algorithm is used to update the learning rate.

[0020] The objective function optimized in the step S3 is as follows:

[0021]

[0022] In the above formula: W is the network weight, sigma i is a noise parameter, L i represents the loss function of the subtask, and the calculation formula is as follows:

[0023]

[0024] In the above formula: C is the number of categories, t n represents a real probability distribution, y n represents a predicted probability distribution, and gamma is used to balance the contribution of difficult classification samples and easy classification samples to the loss.

[0025] The present application has the beneficial effects that the present application introduces TI-RADS into the thyroid nodule diagnosis problem, designs a multi-task network structure with multiple branches, and introduces an attention module, so that the network is more focused on learning the nodule position, thereby improving the network performance. BRIEF DESCRIPTION OF DRAWINGS

[0026] The present application will be further described below in conjunction with the drawings:

[0027] Figure 1 The present application is a flowchart of the method. DETAILED DESCRIPTION

[0028] As Figure 1 shown, in order to overcome the problems of poor accuracy and poor interpretability of the existing thyroid diagnosis model, the present application proposes a thyroid multi-task diagnosis neural network based on TI-RADS, which takes TI-RADS as a guide, introduces a multi-task learning mechanism and an attention mechanism, and improves the accuracy of network classification while enhancing the interpretability.

[0029] The technical solution adopted by the present application to solve its technical problems includes the following steps:

[0030] Step 1: image preprocessing:

[0031] The image preprocessing includes five steps of (1) denoising the image using an adaptive median filtering algorithm, (2) obtaining artificial marking areas using a YOLO V4 and MSER algorithm, (3) removing artificial markers of the ultrasound image using a fast marching algorithm, (4) extracting an ROI area, and (5) image enhancement.

[0032] Step 2: Constructing a convolutional neural network:

[0033] DenseNets 121 is used as a backbone network, and DenseNets 121 includes 4 Dense Blocks and 3 transition layers. The 4 Dense Blocks are respectively composed of 6, 12, 24 and 16 Dense Layers, and each Dense Layer includes a batch normalization layer (BN), a rectified linear unit layer (ReLu), a 1x1 convolution layer, a batch normalization layer, a rectified linear unit layer and a 3x3 convolution layer.

[0034] In order to reduce the interference of noise in the data, an SGE module is added. The SGE module is a hybrid attention module, and the SGE regulates the importance of sub-features by generating an attention factor for each spatial position of each semantic group. The attention factor is generated by the similarity between the global and local feature descriptors in each group, so that each group can autonomously enhance its learned expression and suppress possible noise, so that the model pays more attention to the learning of high-order semantic features. The SGE module is inserted after the batch normalization layer of the bottleneck layer in each Dense Block.

[0035] The fully connected layer of the neural network has five branches, respectively corresponding to five TI-RADS classification tasks, and each branch includes three linear layers. The number of neurons of the linear layer L1 is 1024, the number of neurons of the linear layer L2 is 512, and the number of neurons of the linear layer L3 is the number of categories corresponding to the classification task.

[0036] Step 3: Training the convolutional neural network:

[0037] During training, the pre-training weights of the first Dense Block are migrated, and the weights are obtained by training on the ImageNet dataset. The initial 10 cycles are trained, the learning rate is set to 0.0001, the network weights are preheated, then the learning rate is increased to 0.001, and the cosine annealing algorithm is used to update the learning rate.

[0038] The present application updates and solves the network weights by optimizing the following objective function:

[0039]

[0040] wherein, L i represents a loss function of a subtask, a specific form is shown as (2), W is a network weight, and sigma i is a noise parameter.

[0041]

[0042] wherein, C is a category number, t n represents a real probability distribution, y n represents a predicted probability distribution, and gamma is used to balance the contribution of difficult samples and easy samples to the loss.

[0043] The application discloses a thyroid nodule diagnosis multi-task network based on TI-RADS. Since most deep learning-based auxiliary diagnosis methods are only used to simply obtain the benignity and malignancy of a nodule, the classification of nodules with different malignant degrees is ignored, and due to the black box nature of a neural network, doctors cannot know the basis for decision-making of the neural network, so it is difficult to explain the reason for decision-making of the neural network to patients, and the degree of trust in the result of the neural network is low. The application designs a multi-task neural network based on TI-RADS for the diagnosis of thyroid nodules. The method can obtain the clinical ultrasound features of the nodule, calculate the TI-RADS score, divide the malignant degree of the nodule into TR1-TR5, and finally obtain the benignity and malignancy of the nodule. The network is guided by ACR TI-RADS, and the clinical diagnosis process of doctors is combined with the benignity and malignancy discrimination process of the neural network, so that the explainability of the model result is improved, the degree of trust of doctors in the model is deepened, the multi-task learning mode also promotes mutual promotion between different classification tasks of the network, and the effect is better than that of a single-task neural network.

[0044] It should be noted that the connection relationship between the various components and modules adopted by the application is definite and achievable. Except for the special description in the embodiments, the specific connection relationship can bring corresponding technical effects, and based on the premise of not relying on the execution of the corresponding software program, the technical problems proposed by the application are solved. The model, module, specific component type, and connection mode appearing in the application, except for the specific description, belong to the prior art that can be obtained by the person skilled in the art before the application date, such as published patents, published journal papers, or common knowledge, and do not need to be described in detail. The technical scheme provided in the case is clear, complete, and achievable, and the corresponding entity product can be reproduced or obtained according to the technical means.

[0045] It should be noted that the above embodiments are only used to illustrate the technical solutions of the present application, and are not intended to limit the present application; although the present application has been described in detail with reference to the above embodiments, those skilled in the art should understand that the technical solutions recorded in the above embodiments can be modified, or some or all of the technical features can be replaced by equivalents; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the scope of the technical solutions of the embodiments of the present application.

Claims

1. A method for classifying benign and malignant thyroid nodules based on a TI-RADS multitask neural network, characterized in that: Comprising the following steps: S1: image preprocessing: the image preprocessing comprises image denoising, obtaining an artificial marking area, removing an artificial marking of an ultrasound image, extracting an ROI area, and image enhancement; S2: constructing a convolutional neural network: using DenseNets 121 as a backbone network, the DenseNets 121 comprising 4 Dense Blocks and 3 Transition layers, each Dense Block being composed of Dense Layers, and each Dense Layer comprising, in sequence, a batch normalization layer, a rectified linear unit layer, a 1x1 convolution layer, a batch normalization layer, a rectified linear unit layer, and a 3x3 convolution layer; each branch of the fully connected layer in the step S2 comprising 3 linear layers, wherein the number of neurons of the linear layer L1 is 1024, the number of neurons of the linear layer L2 is 512, and the number of neurons of the linear layer L3 is the number of categories corresponding to a classification task; An SGE module is inserted after the batch normalization layer of the bottleneck layer in each Dense Block; The fully connected layer of the network has five branches, respectively corresponding to five TI-RADS classification tasks, and each branch comprises 3 linear layers; S3: training the convolutional neural network: the pre-training weight of the first Dense Block is migrated during training, and the updating and solving of the network weight are realized through the optimization of the objective function; The objective function optimized in the step S3 is as follows: In the formula: W is the network weight, σ i is the noise parameter, L i Loss function of subtask, the calculation formula is as follows: In the above equation: C is the number of classes, t n represents a true probability distribution, y n represents a predicted probability distribution, γ is used to balance the contribution of difficult and easy-to-classify samples to the loss; S4: inputting a thyroid nodule image into the trained convolutional neural network to output a benign or malignant classification result of the thyroid nodule. 2.The method of claim 1, wherein the method is based on a TI-RADS multitask neural network. The image denoising in the step S1 specifically adopts an adaptive median filter algorithm. 3.The method of claim 1, wherein the TI-RADS multitask neural network is based on a ResNet-50 architecture. The artificial marking area is obtained by using YOLO V4 and MSER algorithms in the step S1. 4.The method of claim 1, wherein the TI-RADS multitask neural network is based on a ResNet-50 architecture. The fast marching algorithm is used to filter out the artificial marking of the ultrasound image in the step S1.

5. The method of claim 1, wherein the method is based on a TI-RADS multitask neural network for thyroid nodule benign and malignant classification. The 4 Dense Blocks in the step S2 are respectively composed of 6, 12, 24, and 16 Dense Layers.

6. The method of claim 1, wherein the method is based on a TI-RADS multitask neural network for thyroid nodule benign and malignant classification. The SGE module regulates the importance of sub-features by generating an attention factor for each spatial position of each semantic group, and the attention factor is generated from the similarity between global and local feature descriptors in each group.

7. The method of claim 1, wherein the method is based on a TI-RADS multitask neural network for thyroid nodule benign and malignant classification. In the step S3, the initial 10 cycles are trained when training the neural network, the learning rate is set to 0.0001 to preheat the network weight, then the learning rate is increased to 0.001, and the cosine annealing algorithm is used to update the learning rate.

Images