A computerized tomography-based auxiliary diagnosis system for benign and malignant pulmonary nodules

By building a deep learning neural network with Transformer module, using the three-dimensional spatial information of CT images, the problem of insufficient diagnostic accuracy of benign and malignant lung nodules in the existing technology is solved, and more efficient detection and diagnosis assistance of lung nodules are achieved .

CN113902702BInactive Publication Date: 2024-12-06NINGBO INST OF LIFE & HEALTH IND UNIV OF CHINESE ACAD OF SCI
View PDF 2 Cites 0 Cited by

Patent Information

Application Number
CN202111171232.X
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2021-10-08
Publication Date
2024-12-06
Estimated Expiration
Not applicable · inactive patent

AI Technical Summary

Technical Problem

The existing computed tomography image processing technology is difficult to effectively utilize three-dimensional spatial relationship characteristics, resulting in insufficient auxiliary diagnosis accuracy of benign and malignant lung nodules, and the planar image object detection algorithm is difficult to apply to three-dimensional images.

Method used

Deep learning neural network with Transformer module, including downsampling module, timing channel Transformer module and gate mechanism module, is used to preprocess, label and train CT images, and build a neural network model that can utilize three-dimensional spatial information for Detection of the location and malignancy of the lung nodules.

Benefits of technology

It improves the accuracy of benign and malignant detection of pulmonary nodules, can assist doctors in diagnosis more accurately, reduce doctors' judgment time, and improve work efficiency.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN113902702B_ABST
    Figure CN113902702B_ABST
Patent Text Reader

Abstract

The present invention discloses a pulmonary nodule benign and malignant auxiliary diagnosis system based on computer tomography, comprising the following steps: step 1, collecting CT images of benign and malignant pulmonary nodules, annotating these CT images, and arranging them into a data set; step 2, preprocessing the self-made data set image; step 3, constructing a deep learning neural network with a Transformer module, the neural network including a downsampling module, a temporal channel Transformer module and a gate mechanism module, step 4, neural network initialization, step 5, neural network training, testing and tuning, dividing the data set into a training set, a validation set and a test set; step 6, inputting the CT image to be diagnosed into the trained network, and outputting the diagnosis result. The advantages of the present invention are: strong recognition and positioning capabilities, can improve the detection accuracy of benign and malignant pulmonary nodules, can reduce the judgment time of doctors, assist doctors in decision-making, and improve work efficiency.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The invention relates to the technical field of medical image processing and auxiliary diagnosis, in particular to a computer tomography-based auxiliary diagnosis system for benign and malignant pulmonary nodules. Background Art

[0002] Computed tomography (CT) is a common test for diagnosing lung nodules and plays an important role in the diagnosis of benign and malignant lung nodules. In recent years, with the continuous development of computer technology and deep learning technology, computer vision-assisted diagnosis systems have achieved certain results in the identification of benign and malignant lung nodules.

[0003] As a complete diagnostic process, it is necessary to mark and judge the location and malignancy of lung nodules. Each convolution kernel in the convolutional neural network (CNN) not only has the function of feature extraction, but also retains the extraction of spatial relationship features of the original image. However, for CT images, the spatial relationship features in three-dimensional space are not utilized. The general target detection algorithm of planar images is difficult to be effectively applied to the auxiliary diagnosis of benign and malignant lung nodules; because CT images are three-dimensional images, they can make full use of spatial information and combine the information of the upper and lower layers to provide more useful information for auxiliary diagnosis. Therefore, it is necessary to propose new ideas that can assist in the diagnosis of benign and malignant lung nodules based on the characteristics of CT three-dimensional images. Summary of the invention

[0004] The purpose of the present invention is to make up for the above-mentioned deficiencies and to disclose to the public a computerized tomography-based auxiliary diagnosis system for benign and malignant lung nodules, which has strong identification and positioning capabilities and can improve the detection accuracy of benign and malignant lung nodules.

[0005] The technical solution of the present invention is achieved in this way:

[0006] A computer tomography-based auxiliary diagnosis system for benign and malignant pulmonary nodules, comprising the following steps:

[0007] Step 1: Collect CT images of benign and malignant pulmonary nodules, annotate these CT images, and organize them into a data set;

[0008] Step 2: Preprocess the self-made dataset images;

[0009] Step 3: construct a deep learning neural network with a Transformer module, wherein the neural network includes a downsampling module for sampling and obtaining feature maps of the current layer and adjacent layer images;

[0010] The temporal channel Transformer module, whose encoder encodes the overall spatial channel information of multiple layers, the decoder decodes the partial spatial channel information of the current layer in voxel format, and the spatial decoder decodes the information of each position of the current layer;

[0011] The gate mechanism module uses the gate mechanism to recalibrate the features of the current frame before using the detection head for object detection;

[0012] Step 4. Neural network initialization: design loss function,

[0013]

[0014] Where n is the number of positive anchors, β cls and β loc They correspond to the classification loss L cls and the regression loss L loc The weight of cls ≤1,0≤β loc ≤1, and β cls +β loc =1;

[0015] Step 5: Training, testing and tuning of the neural network, dividing the data set into training set, validation set and test set;

[0016] Step 6: Input the CT image to be diagnosed into the trained network and output the diagnosis result.

[0017] The measures to further optimize this technical solution are:

[0018] As an improvement, in step 1, the annotation of the CT image includes the location of the lung nodules, the benign or malignant nature of the lung nodules, and the degree of malignancy.

[0019] As an improvement, in the step 2, the image preprocessing process is: normalizing the pixel values ​​of the image, x′ is the pixel value after scaling, x is the pixel value before scaling, and x min is the minimum pixel value, x max is the maximum pixel value.

[0020] As an improvement, in the step 2, new samples are obtained by randomly rotating the original image and the target frame therein around the center point by a certain angle, and the new samples are added to the data set.

[0021] As an improvement, in step 4, the classification loss adopts L cls (P t )=-(1-P t ) γ log(P t), P t Represents the probability at time t; regression loss is adopted

[0022]

[0023] Among them, x is the horizontal coordinate of the center point of the lung nodule, y is the vertical coordinate of the center point of the lung nodule, h is the height of the target box, and w is the width of the target box.

[0024] As an improvement, in step 5, 80% of the data set is used as a training set, 10% is used as a validation set, and 10% is used as a test set.

[0025] As an improvement, in step five, the Pytorch framework is used to train the neural network.

[0026] As an improvement, in the step five, the network is made to converge to the optimal value by using the gradient descent method.

[0027] The advantages of the present invention compared with the prior art are:

[0028] The present invention discloses an auxiliary diagnosis system for benign and malignant lung nodules based on computer tomography, which adopts a temporal channel Transformer network model and utilizes the information of adjacent layers and different channels in a three-dimensional image of an electronic computer tomography scan, and can optimize the recognition and positioning capabilities of the network model, thereby improving the detection accuracy of benign and malignant lung nodules, providing auxiliary diagnosis information for doctors, reducing the judgment time of doctors, assisting doctors in decision-making, and improving work efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0029] Figure 1 is a flow chart of the present invention;

[0030] Figure 2 It is a schematic diagram of the network framework structure of the present invention;

[0031] Figure 3 yes Figure 2 Enlarged view of part A in the middle. DETAILED DESCRIPTION

[0032] The present invention adopts a temporal channel Transformer network model to analyze the electronic computed tomography images of lung nodules to determine the location of lung nodules and the corresponding benign or malignant categories; as a computer-aided system, this invention can reduce the doctor's judgment time, assist the doctor in decision-making, and improve work efficiency.

[0033] like Figure 1 As shown, a computer tomography-based auxiliary diagnosis system for benign and malignant pulmonary nodules includes the following steps:

[0034] Step 1: Collect CT images of benign and malignant pulmonary nodules, annotate them, and organize them into a data set. The annotations of CT images include the location of pulmonary nodules, benign and malignant pulmonary nodules, and the degree of malignancy.

[0035] We collected 1,365 computed tomography images of lung nodules from a hospital and annotated them based on electronic medical records. Specifically, for each computed tomography image of a lung nodule, we need to mark the location of the lung nodule and the benign or malignant degree of the lung nodule at that location. We also classify each lung nodule according to its malignancy, which is divided into five levels: benign, 4 to 5 malignant, and 3 based on the doctor's experience. Finally, we organized the 1,365 computed tomography images into a dataset that can be used for deep learning.

[0036] Step 2: Preprocess the images of the self-made dataset.

[0037] For deep learning algorithms, the quantity and quality of data are one of the important factors that affect the model training results and generalization ability. Preprocessing work mainly includes three aspects: data enhancement, sample balance and normalization. In medical image samples, the problem of sample imbalance is particularly prominent. The incidence rates of various diseases are different, and some diseases may have only a few samples. Such unbalanced data is not conducive to model training. In order to solve this problem, the present invention expands the number of training samples through resampling and data enhancement techniques. Specifically, new training samples are obtained by randomly rotating the original image around the center point at a certain angle, and these new training samples are also added to the data set; such as rotating 90°, 180°, 270°, etc. In addition, due to the different machines used to take pictures, the pictures in the samples may have different brightness values ​​or noise values. In order to minimize the errors caused by this, the pixel values ​​of the picture need to be normalized before entering the model. The pixel values ​​of the picture are normalized, x′ is the pixel value after scaling, x is the pixel value before scaling, and x min is the minimum pixel value, x max is the maximum pixel value.

[0038] Step 3: Construct a deep learning neural network with a Transformer module. The neural network mainly includes a downsampling module, a temporal channel Transformer module, and a gate mechanism module. This network model can model the relationship between the spatiotemporal domain and the channel domain, and can use the three-dimensional image obtained by electronic computed tomography for target detection. The downsampling module samples the classic convolutional neural network model initialized with a pre-trained model to obtain the feature map of the current layer and the adjacent layer images. As a special design of the Transformer, the information encoded in the encoder of the temporal channel Transformer is different from the information encoded in the decoder, that is, the encoder encodes the overall spatial channel information of multiple layers, and the decoder decodes the partial spatial channel information of the current layer in voxel format. Specifically, the overall spatial channel encoder of the Transformer is designed to encode the information of different channels and layers by utilizing the correlation between features from different channels and layers; in addition, the spatial decoder of the Transformer will decode the information of each position in the current layer. Before using the detection head for target detection, a gate mechanism is needed to recalibrate the features of the current frame, and filter out information irrelevant to the object by repeatedly refining the representation of the target layer and the downsampling process; finally, the detection head is connected to output the detection results to locate the location of the lung nodules and the category of malignancy.

[0039] like Figure 2 As shown in Figure 2, C represents a weight-sharing convolutional layer, which is used to change the number of channels; Z represents the fused feature map, which is the input of the Transformer; x i Represents the feature map of the current image; S′ i Represents x i After deformation (using the reshape function), it is used as the input of the decoding module in the Transformer; d represents the output feature map of the Transformer module. Figure 3 As shown, the black box in the figure is the target box.

[0040] Step 4: Initialize the neural network: For deep learning neural networks, the loss function determines the training direction and performance of the network. For the detection and positioning task of CT images of lung nodules, the present invention designs the following loss function, which consists of two parts: classification loss and regression loss. The classification loss function is to enable the network to have classification capabilities. Here, the classic loss function designed for class imbalance and classification difficulty difference is adopted: Focal loss, whose formula is: L cls (P t )=-(1-P t ) γ log(P t ), where Pt represents the probability at time t; in addition to generating the object category, we also need to obtain the location coordinates (x, y) of the center point of the lung nodule, as well as the height (h) and width (w) of the target box. The regression loss is as follows: A smooth L1 loss is used to enable the network to acquire the ability to locate. The target box is a rectangular block marking the location of the lung nodule in the image. The rectangular block completely surrounds the lung nodule area, and the rectangular block can be uniquely represented by the center point coordinates (x, y) and height (h) and width (w).

[0041] The total loss function is expressed as a linear combination of these two loss functions. Where n represents the number of positive anchors, β cls and β loc They correspond to the weights of classification loss and regression loss respectively.

[0042]

[0043] Where 0≤β cls ≤1,0≤β loc ≤1, and β cls +β loc =1.

[0044] Step 5: Training, testing and tuning of neural networks. Divide the data set into training set, validation set and test set. Before network training, we use random initialization to assign a small value to the network parameters. In addition, we divide the data set into 80% as training set, 10% as validation set and 10% as test set. Use Pytorch framework for model deployment and training. After completing the design and initialization of the time series channel Transformer network structure, input the training set into the network for model training. During the training process, according to the loss changes of the training set and validation set, the gradient descent method is used to make the network converge to the optimal value, and then save the model hyperparameters that make the validation set result the best. Then, all the data are used as the training set to input the previously saved hyperparameter network model for training, and the obtained model is saved for the detection of new samples.

[0045] Step 6: Input the CT image to be diagnosed into the trained network and output the diagnosis result. The system outputs whether the patient has benign or malignant lung nodules. If yes, it gives the confidence rate and the corresponding location. The output structure is provided for the doctor’s reference, which can reduce the doctor’s judgment time, assist the doctor in decision-making, and improve work efficiency.

[0046] The best embodiment of the present invention has been described, and various changes or modifications may be made by those skilled in the art without departing from the scope of the present invention.

Claims

1. A computer tomography-based auxiliary diagnosis system for benign and malignant pulmonary nodules, characterized by: The following steps are involved: Step 1: Collect CT images of benign and malignant pulmonary nodules, annotate these CT images, and organize them into a data set; Step 2: Preprocess the self-made dataset images; Step 3: Build a deep learning neural network with a Transformer module. The neural network includes The downsampling module is used to sample and obtain the feature maps of the current layer and the adjacent layer images; The temporal channel Transformer module, whose encoder encodes the overall spatial channel information of multiple layers, the decoder decodes the partial spatial channel information of the current layer in voxel format, and the spatial decoder decodes the information of each position of the current layer; The gate mechanism module uses the gate mechanism to recalibrate the features of the current frame before using the detection head for object detection; Step 4. Neural network initialization: design loss function, Where n is the number of positive anchors, β cls and β loc They correspond to the classification loss L cls and regression loss L loc The weight of cls ≤1,0≤β loc ≤1, and β cls +β loc =1; Step 5: Training, testing and tuning of the neural network, dividing the data set into training set, validation set and test set; Step 6: Input the CT image to be diagnosed into the trained network and output the diagnosis result.

2. The computer tomography-based auxiliary diagnosis system for benign and malignant pulmonary nodules according to claim 1 is characterized by: In the step 1, the annotation of the CT image includes the location of the lung nodules, the benign or malignant nature of the lung nodules, and the degree of malignancy.

3. The computer tomography-based auxiliary diagnosis system for benign and malignant pulmonary nodules according to claim 1 is characterized by: In the step 2, the image preprocessing process is: normalizing the pixel values ​​of the image, x′ is the pixel value after scaling, x is the pixel value before scaling, and x min is the minimum pixel value, x max is the maximum pixel value.

4. The computer tomography-based auxiliary diagnosis system for benign and malignant pulmonary nodules according to claim 1, characterized in that: In the step 2, new samples are obtained by randomly rotating the original image and the target frame therein around the center point by a certain angle, and the new samples are added to the data set.

5. The computer tomography-based auxiliary diagnosis system for benign and malignant pulmonary nodules according to claim 1 is characterized by: In step 4, the classification loss is L cls (P t )=-(1-P t ) γ log(P t ), P t Represents the probability at time t; regression loss is adopted Among them, x is the horizontal coordinate of the center point of the lung nodule, y is the vertical coordinate of the center point of the lung nodule, h is the height of the target box, and w is the width of the target box.

6. The computer tomography-based auxiliary diagnosis system for benign and malignant pulmonary nodules according to claim 1, characterized in that: In step 5, 80% of the data set is used as a training set, 10% as a validation set, and 10% as a test set.

7. The computer tomography-based auxiliary diagnosis system for benign and malignant pulmonary nodules according to claim 1 is characterized by: In the step 5, the Pytorch framework is used to train the neural network.

8. The computer tomography-based auxiliary diagnosis system for benign and malignant pulmonary nodules according to claim 1, characterized in that: In the step 5, the network is converged to the optimal value by the gradient descent method.

Citation Information

Patent Citations

  • Pulmonary nodule benign and malignant classification method based on combination of adversarial generative network and self-coding

    CN110516747A

  • Medical diagnosis and treatment system

    CN111292821A