Auxiliary judgment method, device and medium for nasopharyngeal carcinoma radiotherapy brain damage image

Abstract

The application discloses a nasopharyngeal carcinoma radioactive brain damage image auxiliary judgment method, equipment and medium, the method is: collecting MRI image and preprocessing, segmenting the temporal lobe region, using the temporal lobe region and corresponding lesion annotation training level prediction model of MRI image, the probability that a single temporal lobe image region exists radioactive brain damage lesion is predicted;With the probability that each subject all layers contain lesions output by the level prediction model as input, whether the subject exists radioactive brain damage as output, training patient level prediction model;Finally, the trained level prediction model and patient level prediction model are used to obtain the incidence probability of radioactive brain damage of a single nasopharyngeal carcinoma subject, when multiple modes coexist, the average of multiple mode prediction result outputs is taken as the final prediction result. The application can judge whether the corresponding nasopharyngeal carcinoma brain MRI image exists radioactive brain damage according to the input, provide reference for clinical doctors to diagnose, and reduce the missed diagnosis of nasopharyngeal carcinoma radioactive brain damage.

Description

TECHNICAL FIELD

[0001] The application belongs to the field of medical information technology, and particularly relates to a nasopharyngeal carcinoma radioactive brain damage image auxiliary judgment method, device and medium. BACKGROUND

[0002] Nasopharyngeal carcinoma is an epithelial malignant tumor originating from the nasopharyngeal mucosa. Since the nasopharyngeal cavity is adjacent to the skull base and the invasive growth of nasopharyngeal carcinoma, it is difficult to avoid the temporal lobe, especially the bilateral anterior temporal lobe and the inferior temporal lobe, even if intensity modulated radiation therapy and volume modulated arc therapy are used, and radioactive temporal lobe damage is prone to occur. The late symptoms of radioactive temporal lobe damage are irreversible, manifested as memory loss, cognitive impairment, etc., and severe temporal lobe necrosis may occur. However, in clinical work, due to the reasons of doctor's attention, lesion size and location, there are cases of missed diagnosis or delayed diagnosis of nasopharyngeal carcinoma radioactive brain damage, which affects the prognosis and quality of life of patients. In recent years, deep learning has shown good prospects in the field of medical image analysis such as disease diagnosis and lesion segmentation. Therefore, a nasopharyngeal carcinoma radioactive brain damage diagnosis model based on deep learning can be built to assist doctors in clinical decision-making. SUMMARY

[0003] The application provides a nasopharyngeal carcinoma radioactive brain damage image auxiliary judgment method, device and medium, which can judge whether there is radioactive brain damage corresponding to the input nasopharyngeal carcinoma brain MRI image, provide a reference for clinical doctors in diagnosis, and reduce the missed diagnosis of nasopharyngeal carcinoma radioactive brain damage.

[0004] To achieve the above technical purposes, the application adopts the following technical solutions:

[0005] A nasopharyngeal carcinoma radioactive brain damage image auxiliary judgment method comprises the following steps:

[0006] Step 1, collecting the brain MRI image of a nasopharyngeal carcinoma patient and pre-processing the same;

[0007] Step 2, segmenting the temporal lobe region in the MRI image to obtain a temporal lobe MRI image;

[0008] Step 3, taking the temporal lobe MRI image layer as input and whether the lesion is contained as output, training a first deep learning network to obtain a nasopharyngeal carcinoma radioactive brain damage lesion identification model;

[0009] Step 4, inputting each layer of the temporal lobe MRI image into the nasopharyngeal carcinoma radioactive brain damage lesion identification model to output the probability of existence of a radioactive brain damage lesion corresponding to each layer;

[0010] Step 5, taking the probability of radioactive brain injury lesions of all layers of the temporal lobe MRI image of each patient as input and whether the patient has radioactive brain injury necrosis as output, training a second deep learning network to obtain a nasopharyngeal carcinoma radioactive brain injury diagnosis model;

[0011] Step 6, for the newly obtained brain MRI image of the nasopharyngeal carcinoma patient, pre-processing according to step 1, obtaining the segmented temporal lobe MRI image according to step 2, and using the radioactive brain injury lesion identification model to obtain the probability of nasopharyngeal carcinoma lesions in each temporal lobe layer according to step 4, and then inputting the radioactive brain injury lesion probability of all temporal lobe layers into the nasopharyngeal carcinoma radioactive brain injury diagnosis model to obtain the probability of each patient having radioactive brain injury.

[0012] In a more preferred technical solution, the pre-processing of step 1 includes image quality control and bias field correction.

[0013] In a more preferred technical solution, the first deep learning network adopts an improved EfficientNet network; specifically, an attention mechanism is added after the last convolutional layer of the original EfficientNet network, and the top layer full connection layer is modified into four full connection layers with neuron numbers of 256, 128, 64 and 32, respectively, and the activation function is ReLU, and finally a neuron sigmoid classification layer; the first deep learning network is trained using a binary classification cross-entropy loss function.

[0014] In a more preferred technical solution, L2 regularization and Dropout are added to each full connection layer of the improved EfficientNet network.

[0015] In a more preferred technical solution, the second deep learning network adopts a multilayer perceptron with neuron numbers of 30, 16 and 8, respectively, and the activation function is ReLU, and finally a neuron sigmoid classification layer; the second deep learning network is trained using a class cross-entropy loss function.

[0016] In a more preferred technical solution, the brain MRI image includes T1WI, T2WI and CE-T1WI three modal data, and three different modal data are used to train three lesion identification models and three diagnosis models respectively.

[0017] In a more preferred technical solution, the average of the radioactive brain injury probabilities output by the diagnosis models corresponding to the three modalities is taken as the final diagnosis probability of radioactive brain injury.

[0018] An electronic device comprises a memory and a processor, the memory stores a computer program, and the computer program is executed by the processor to enable the processor to implement the auxiliary identification method of nasopharyngeal carcinoma radioactive brain damage image according to any one of the above.

[0019] A computer readable storage medium stores a computer program, and the computer program is executed by a processor to implement the auxiliary identification method of nasopharyngeal carcinoma radioactive brain damage image according to any one of the above.

[0020] Advantages

[0021] The application uses the method of artificial intelligence to extract deep convolutional features from temporal lobe MRI images, realizes nasopharyngeal carcinoma temporal lobe radioactive brain damage lesion detection and diagnosis, and adds attention mechanism to improve lesion positioning efficiency and force the model to focus on global features of the lesion, which can further improve the diagnosis performance of nasopharyngeal carcinoma temporal lobe radioactive brain damage. The deep learning model can be integrated into the daily workflow of radiologists. According to the research, the deep learning model reduces the missed detection rate of nasopharyngeal carcinoma radioactive brain damage to 9.8%, especially improves the detection of small solid enhanced nodules with a diameter of less than or equal to 10 mm. BRIEF DESCRIPTION OF DRAWINGS

[0022] Figure 1 The technical roadmap of the method described in the embodiments of the present application. DETAILED DESCRIPTION

[0023] The embodiments of the present application are described in detail below. The embodiments are based on the technical solutions of the present application and give detailed implementation methods and specific operation processes, which further explain and describe the technical solutions of the present application. EMBODIMENTS

[0024] The embodiment provides an auxiliary identification method for nasopharyngeal carcinoma radioactive brain damage image, as shown in Figure 1 The method comprises the following steps:

[0025] Step 1, collect the brain MRI image of the nasopharyngeal carcinoma patient, and pretreat it.

[0026] The MRI images of nasopharyngeal carcinoma patients collected in this embodiment include CE-T1WI, T1WI and T2WI three modal data. The data is collected from Xiangya Hospital of Central South University, Chenzhou First Hospital, First Affiliated Hospital of Nanhua University, Second Affiliated Hospital of Nanhua University, etc. There are 3842 cases of nasopharyngeal carcinoma patients with radiation brain injury. The diagnostic criteria for nasopharyngeal carcinoma radiation brain injury are as follows: (1) white matter lesions (finger-shaped lesions with increased signal intensity on T2WI); (2) enhanced lesions (small nodules or large necrotic lesions on CE-T1WI); (3) cysts (circular or oval lesions with signal intensity similar to cerebrospinal fluid on T2WI); (4) hemorrhage (T1WI shows high signal, and T2WI shows low signal of hemosiderin deposition). When any of these manifestations occurs, it can be diagnosed as nasopharyngeal carcinoma radiation brain injury.

[0027] During MR scanning, the bias field causes the magnetic field strength to be non-uniform, which causes the MR intensity value to change in images obtained from the same scanner, the same patient, and even the same tissue. In this embodiment, the N4BiasFiledCorrection function in the Python package SimpleITK is used to solve this problem, and ANTS is used to register multi-modal image data.

[0028] Step 2, segment all temporal lobe regions in the brain MRI image. The segmentation method is: import the collected image data into ITKSNAP, draw the temporal lobe boundary, and export the original image data image and the segmentation image as “.nii” format.

[0029] Multiply the original image with the corresponding manually segmented temporal lobe mask to obtain a region containing only the temporal lobe, and crop it to the minimum horizontal bounding rectangle to include the minimum horizontal rectangle of the region, and adjust the size to 380x380 square pixels of temporal lobe MRI image.

[0030] Since the images collected in this embodiment are thick layer images, and the scanning range is inconsistent (some are head and neck, and some are only head), manual segmentation is used. For the collected thin layer images, automatic segmentation can also be achieved by registration (registration to template, and then use template to segment the region).

[0031] Step 3, take the temporal lobe MRI image layer as input, and whether it contains lesions as output, train the first deep learning network, and obtain the lesion recognition model of nasopharyngeal carcinoma radiation brain injury;

[0032] Among them, the method for the radiologists to label whether the temporal lobe MRI image layer contains lesions is: the expert browses the original images of each modality, labels the radiation brain injury lesions in the scanning transverse layer, and takes the skull base as the starting layer.

[0033] In step 3, the dataset is divided into a training set, a validation set, and a test set on a patient-by-patient basis. The training set and the validation set data come from Xiangya Hospital of Central South University, and the remaining hospital data constitutes the test set.

[0034] In step 4, the temporal lobe MRI image layers are input into the nasopharyngeal carcinoma radiation brain injury lesion recognition model, and the probability of the presence of a radiation brain injury lesion in each layer is output.

[0035] In step 5, the probability of a radiation brain injury lesion in all layers of the temporal lobe MRI image of each patient is input, and whether the patient has a radiation brain injury necrosis is output. A second deep learning network is trained to obtain a nasopharyngeal carcinoma radiation brain injury diagnosis model.

[0036] Steps 3-5 above can first use all training data to train the lesion recognition model in step 3, then use the trained lesion recognition model to obtain the lesion probability contained in each layer in step 4, and then use the training data to train the nasopharyngeal carcinoma radiation brain injury diagnosis model in step 5. The programming language used to train the nasopharyngeal carcinoma radiation brain injury lesion recognition model and the nasopharyngeal carcinoma radiation brain injury diagnosis model is Python, and the framework used is Keras. The training graphics card is Quadro GV100 with 16GB of memory. During training, image augmentation is performed on the training samples, such as random axis flipping, translation, random cropping, etc., to enrich the training sample set.

[0037] The nasopharyngeal carcinoma radiation brain injury lesion detection model uses an improved EfficientNet-B4 network, with the input being the preprocessed temporal lobe MRI image layer and the output being whether the layer contains a radiation brain injury lesion. This network balances all dimensions of network width, depth, and resolution, providing better performance. In addition, to force the network to learn more comprehensive nasopharyngeal carcinoma radiation brain injury lesion features, an Attention-based Dropout Layer (ADL) attention mechanism is added after the last convolutional layer of the base network EfficientNet-B4 (without the last fully connected layer). This mechanism is applied to each feature map without additional network parameters, guiding the model to learn the complete lesion area. Finally, four fully connected layers and a Sigmod layer are added after the ADL layer for classification. The number of neurons in the four fully connected layers is 256, 128, 64, and 32, respectively. L2 regularization constraints and Dropout layers are added to each fully connected layer to reduce the risk of network overfitting. The loss function used by this network is the binary cross-entropy loss function.

[0038]

[0039] wherein, is the number of slices, is the gold standard corresponding to the slice, that is, whether the layer contains a nasopharyngeal carcinoma radioactive brain injury lesion, is the probability of containing a lesion output by the detection model.

[0040] The network structure of the nasopharyngeal carcinoma radioactive brain injury diagnosis model is a multilayer perceptron, the input is the probability output of the nasopharyngeal carcinoma radioactive brain injury detection model of each patient single temporal lobe layer, and the output is whether a single patient is diagnosed as nasopharyngeal carcinoma radioactive brain injury. The number of neurons in each layer is 32, 16, 8, and 1, respectively, the activation function of the last layer is Sigmoid, and the activation functions of the remaining layers are ReLU. The loss function used in network training is the binary cross-entropy loss function.

[0041]

[0042] wherein, is the number of patients, is the gold standard corresponding to the patient, that is, whether the patient is diagnosed as nasopharyngeal carcinoma radioactive brain injury, is the probability of disease output by the diagnosis model.

[0043] Step 6, for the newly obtained brain MRI image of the nasopharyngeal carcinoma patient, the pre-processing is carried out according to step 1, the segmented temporal lobe MRI image is obtained according to step 2, and the probability of the existence of nasopharyngeal carcinoma lesion in each temporal lobe layer is obtained by using the radioactive brain injury lesion identification model according to step 4, then the radioactive brain injury lesion probabilities of all temporal lobe layers are input into the nasopharyngeal carcinoma radioactive brain injury diagnosis model to obtain the probability of the existence of radioactive brain injury of each patient.

[0044] Since the MRI image in the embodiment includes CE-T1WI, T1WI and T2WI three kinds of modal data, in order to improve the diagnosis accuracy, three kinds of different modal data are used to correspond to three lesion identification models and three diagnosis models obtained by training according to steps 3-5. Since the lesion identification model in the embodiment adopts a three-channel input structure, when the model only uses one kind of modal data, a single slice three-channel stacking method is used to input the three channels of the lesion identification model. Further, when judging the newly obtained brain MRI image of the nasopharyngeal carcinoma patient, the average value of the radioactive brain injury probabilities output by the three modal corresponding diagnosis models is taken as the probability of the final diagnosis of radioactive brain injury.

[0045] Through the nasopharyngeal carcinoma radioactive brain injury detection and diagnosis method based on deep learning, the brain MRI image can obtain: (1) the probability of containing a radioactive brain injury lesion in a single temporal lobe layer and the corresponding position; (2) the probability of whether a single patient has a radioactive brain injury.

[0046] Embodiment 2

[0047] The embodiment provides an electronic device, comprising a memory and a processor, the memory stores a computer program, and the computer program is executed by the processor to enable the processor to implement the method in the embodiment 1.

[0048] Embodiment 3

[0049] The embodiment provides a computer readable storage medium, which stores a computer program, and the computer program is executed by a processor to implement the method in the embodiment 1.

[0050] The above embodiments are the preferred embodiments of the present application, and those skilled in the art can make various modifications or improvements on the basis of the above embodiments without departing from the general concept of the present application, and these modifications or improvements shall also belong to the scope of protection of the present application.

Claims

1. A method for assisting in the determination of a radiotherapy brain lesion image of nasopharyngeal carcinoma, characterized in that, The method comprises the following steps: Step 1: Collecting brain MRI images of nasopharyngeal carcinoma patients and preprocessing the images; Step 2: Segmenting the temporal lobe region in the MRI images to obtain temporal lobe MRI images; Step 3: Training a first deep learning network by taking the temporal lobe MRI image layers as input and whether the lesions are included as output to obtain a nasopharyngeal carcinoma radioactive brain injury lesion identification model; The first deep learning network adopts an improved EfficientNet network; specifically, an ADL attention mechanism is added after the last convolutional layer of the original EfficientNet network; Step 4: Inputting the temporal lobe MRI image layers into the nasopharyngeal carcinoma radioactive brain injury lesion identification model to output the probability of the existence of radioactive brain injury lesions in each layer; Step 5: Training a second deep learning network by taking the radioactive brain injury lesion probabilities of all layers of the temporal lobe MRI images of each patient as input and whether the patient has radioactive brain injury necrosis as output to obtain a nasopharyngeal carcinoma radioactive brain injury diagnosis model; Step 6: For newly obtained brain MRI images of nasopharyngeal carcinoma patients, preprocessing is performed according to step 1, the segmented temporal lobe MRI images are obtained according to step 2, and the probability of the existence of nasopharyngeal carcinoma lesions in each temporal lobe layer is obtained using the radioactive brain injury lesion identification model according to step 4, and then the radioactive brain injury lesion probabilities of all temporal lobe layers are input into the nasopharyngeal carcinoma radioactive brain injury diagnosis model to obtain the probability of the existence of radioactive brain injury for each patient.

2. The method of claim 1, wherein, The preprocessing of step 1 includes image quality control and bias field correction.

3. The method of claim 1, wherein, The improved EfficientNet network modifies the top full connection layer of the original EfficientNet to four full connection layers with neuron numbers of 256, 128, 64 and 32, respectively, and the activation function is ReLU, and finally a neuron sigmoid classification layer; the first deep learning network is trained using a binary cross-entropy loss function.

4. The method of claim 3, wherein, L2 regularization and Dropout are added to each full connection layer of the improved EfficientNet network.

5. The method of claim 1, wherein, The second deep learning network adopts a multilayer perceptron with neuron numbers of 30, 16 and 8, respectively, and the activation function is ReLU, and finally a neuron sigmoid classification layer; the second deep learning network is trained using a class cross-entropy loss function.

6. The method of claim 1, wherein, The brain MRI images include T1WI, T2WI and CE-T1WI three modal data, and three lesion identification models and three diagnosis models are obtained by training using three different modal data respectively.

7. The method of claim 6, wherein, The average value of the radioactive brain injury probabilities output by the diagnosis models corresponding to the three modalities is taken as the final probability of diagnosis as radioactive brain injury.

8. An electronic device comprising a memory and a processor, said memory having stored therein a computer program, characterized in that, The computer program is executed by the processor to implement the method of any one of claims 1-7.

9. A computer readable storage medium having stored thereon a computer program, characterized in that, The computer program is executed by the processor to implement the method of any one of claims 1-7.

Images