An intracranial hematoma classification method based on CNN and improved VIT

Abstract

The application discloses an intracranial hematoma classification method based on a CNN and an improved VIT, belongs to the technical field of medical image classification, and aims at solving the problem of low classification accuracy in existing medical image classification methods.The method comprises the following steps: preparing a data set, manually classifying CT image data sets of intracranial hematoma according to five types of EDH, IPH, IVH, SAH and SDH; data set pre-processing, performing skull removal operation and data expansion on the classified five types of hematoma images; constructing a network model, constructing a network combining a residual neural network ResNet50 and a feature iteration adjacent relationship Vision Transformer model; training the network model, inputting the images obtained through pre-processing in step 2 into the network model constructed in step 3 for training; selecting a minimum loss function and an optimal evaluation index; fine-tuning the model, training and fine-tuning the model with the intracranial hematoma data set to obtain stable and usable model parameters; and saving the model.The method improves the classification accuracy, saves the diagnosis time and avoids the diagnosis errors of different doctors.

Description

TECHNICAL FIELD

[0001] The application relates to an intracranial hematoma classification method based on a CNN and an improved VIT, and belongs to the technical field of medical image classification. BACKGROUND

[0002] Medical image classification is one of important applications of pattern recognition, and the main purpose is to obtain medical pictures from terminals, perform data processing and feature extraction, analyze and recognize the images, and finally complete classification. In recent years, nuclear magnetic resonance imaging CT has achieved rapid progress in the examination of intracranial hematoma and has attracted considerable clinical attention. When deep learning technology is applied to the diagnosis of intracranial hematoma, CT images are often used as input. The present technology is aimed at CT images of intracranial hematoma, and based on deep learning, a convolutional neural network and an improved Vision Transformer model are used for classification.

[0003] Referring to an article entitled "A deep learning algorithm for automatic detection and classification of acute intracranial hemorrhages in head CT scans" published by Wang Xiyue et al. in the journal "NeuroImage: Clinical", a deep learning method combining a convolutional neural network and two sequence models is proposed. In the first stage, a 2D CNN classifier is trained using labeled intracranial hematoma data to obtain a feature map; in the second stage, the feature map in the first stage is used as input, a GRU unit is used, and a bidirectional RNN is applied for accurate estimation of each hematoma type; in the third stage, the outputs of the first and second stages are simultaneously used as input, another RNN model and a GRU unit are used to realize the stacking generalization integration idea, and the recognition and classification of intracranial hematoma are realized. The method cannot extract global features and adjacent features of the image, and the classification accuracy is low, which is not sufficient to be applied to actual tasks in clinical medicine. SUMMARY

[0004] In order to solve the problem of low classification accuracy in the existing medical image classification method, the application provides an intracranial hematoma classification method based on a convolutional neural network (CNN) and an improved Vision Transformer (VIT), which improves the classification accuracy, saves the diagnosis time of doctors, and avoids the diagnosis errors of different doctors. The technical problem solving scheme of the application is:

[0005] An intracranial hematoma classification method based on a CNN and an improved VIT comprises the following steps:

[0006] Step 1, preparing the data set: the CT image data set of intracranial hematoma is manually classified according to five types of epidural hematoma (EDH), intraparenchymal hemorrhage (IPH), intraventricular hemorrhage (IVH), subarachnoid hemorrhage (SAH) and subdural hematoma (SDH);

[0007] Step 2, data set preprocessing: skull removal operation and data augmentation are performed on the classified five types of hematoma images;

[0008] Step 3, building a network model: a network combining residual neural network ResNet50 and feature iterative adjacent relationship of Vision Transformer model is constructed;

[0009] Step 4, training the network model: the images obtained by preprocessing in step 2 are input into the network model constructed in step 3 for training;

[0010] Step 5, selecting the minimum loss function and optimal evaluation index: by minimizing the loss function of the output image and the label, until the training times reach the set threshold or the value of the loss function reaches the set range, the model parameters can be considered to have been trained, and the model parameters are saved. At the same time, the optimal evaluation index is selected to measure the accuracy of the algorithm and evaluate the performance of the system;

[0011] Step 6, fine-tuning the model: the model is trained and fine-tuned with the intracranial hematoma data set to obtain stable and usable model parameters, further improving the classification ability of the model;

[0012] Step 7, save the model: the final determined model parameters are solidified, and then when intracranial hematoma classification operation is needed, the image is directly input into the network to obtain the final segmentation image.

[0013] In step 2, the CT image window is set to 100 and the window width is set to 55, so that the intracranial hemorrhage lesion area is more obvious. The processed DICOM image is converted into a RGB three-channel jpg image with a size of 512x560 and a depth of 24 bits. The data skull removal operation uses Opencv to analyze the pixel features and extract the closed skull region, and fills the skull and skull outside the region with black color consistent with the image background.

[0014] The convolutional neural network used in the step 3 training adopts a ResNet50 convolutional neural network, which contains 49 convolutional layers and one fully connected layer, wherein there are 16 residual blocks, and each residual block is composed of three convolutional layers; the feature map extracted in the ResNet50 convolutional neural network is divided into 16 patches to generate a 16*16 adjacency relationship matrix and a 1*16 feature matrix, the adjacency relationship matrix is multiplied with the feature matrix to obtain an adjacency relationship matrix, and after being multiplied with a diagonal matrix and standardized, a feature matrix with iterated adjacency relationship is obtained, and then the Vision Transformer model composed of position coding, multi-head attention module, multilayer perception and classifier is passed through, that is, the overall network model is constructed.

[0015] The loss function selected in the training process in the step 5 is a cross-entropy loss function.

[0016] The data set used in the step 1 and the data set used in the step 6 fine-tuning model are both public CQ500 data sets.

[0017] The present application can directly distinguish the type of blood clots in the input image by constructing an intracranial hematoma classification network based on a convolutional neural network and an improved Vision Transformer, and the method has the following two beneficial effects:

[0018] 1. The skull of the preprocessed image is removed from the five types of classified hematoma images, which removes the interference of the skull on the lesion area features and improves the positioning accuracy of the hematoma area in feature extraction.

[0019] 2. The convolutional neural network and the Vision Transformer are combined to consider the extraction of local and global features of the image, and the adjacency relationship of the node features is iterated on the basis of the Vision Transformer, which solves the problem of ignoring the feature relationship between each adjacent patch in the Vision Transformer, improves the feature extraction accuracy and increases the classification accuracy. BRIEF DESCRIPTION OF DRAWINGS

[0020] Figure 1 The present application is a flow chart of a method for classifying intracranial hematoma based on CNN and improved VIT.

[0021] Figure 2 The present application is a schematic diagram of the skull removal operation in the step 2.

[0022] Figure 3 The network model structure constructed in the step 3 of the present application.

[0023] Figure 4An operation step of iteratively critical features in step 4 of the present application. DETAILED DESCRIPTION

[0024] The application will be further described in detail below with reference to the accompanying drawings.

[0025] The intracranial hematoma classification method based on CNN and improved VIT comprises the following steps:

[0026] Step 1, prepare the data set. Adjust the window width (WW) and window level (WL) of the brain CT image through ITK software to increase the visibility of the subtle abnormal area, and batch format conversion to obtain a large number of image files, then select the intracranial slices, delete the slice images containing only the skull, and then through expert diagnosis, classify the processed images into different intracranial hematoma, epidural hematoma (EDH), subdural hematoma (SDH), intraventricular hemorrhage (IVH), brain parenchymal hemorrhage (IPH) and subarachnoid hemorrhage (SAH), a total of five types of hematoma, and make labels.

[0027] Step 2, according to the pretreatment. Set the CT image window to 100 and the window width to 55, so that the intracranial hemorrhage lesion area is more obvious, then we convert the processed DICOM image into a jpg image with a size of 512*560 and a depth of 24 bits RGB three channels, find the closed contour of the skull through Opencv, generate two masks of the skull region and the intracranial region, fill the skull region and the region outside the skull with black color, and complete the skull removal operation. The skull-removed image is enhanced and expanded, and 6 times expansion is obtained through horizontal and vertical flipping, random rotation and other ways, and at the same time the image is denoised and contrast enhanced, so that the hematoma area and the intracranial tissue are more distinct. Finally, the processed data is divided into training set, test set and validation set, and the quantity ratio is 7:2:1.

[0028] Step 3, build network model. Use the prepared data set to train ResNet50 network, extract the feature map of the 49th layer of the model as the input of the next model, divide the obtained feature map into 16 patches with feature vectors and number them 1-16, generate an adjacency matrix, a degree matrix and a feature vector matrix according to the adjacency relationship between each patch, multiply the three matrices to obtain a feature vector matrix with iterated adjacency relationship, and output the classification result through the Vision Transformer module composed of position coding, multi-head attention module, multilayer perception and classifier.

[0029] Step 4, train the network model. The data set preprocessed in step 2 is input into the network model constructed in step 3 for training.

[0030] Step 5, select the minimum loss function and the optimal evaluation index. By minimizing the loss function of the network output image and the label, until the training times reach the set threshold or the value of the loss function reaches the set range, it can be considered that the model parameters have been trained, and the model parameters are saved. At the same time, select the optimal evaluation index to measure the accuracy of the algorithm and evaluate the performance of the system;

[0031] Step 6, fine-tune the model. Train and fine-tune the model with the intracranial hematoma dataset to obtain stable and usable model parameters, and further improve the classification ability of the model. Finally, the model classifies the image better;

[0032] Step 7, save the model. The final determined model parameters are solidified, and then when intracranial hematoma classification operation is needed, the image is directly input into the network to obtain the final segmentation image.

[0033] Embodiment:

[0034] As shown in Figure 1 , an intracranial hematoma classification method based on CNN and improved VIT, which specifically includes the following steps:

[0035] Step 1, prepare the dataset. The data uses the CQ500 dataset. The dataset contains 500 patient brain CT images, and each file is preprocessed in step 1 to obtain 4,027 CT image pictures.

[0036] Step 2, data preprocessing. Set the CT image window to 100 and the window width to 55 to make the intracranial hemorrhage lesion area more obvious. Convert the processed DICOM image to a 512x560 size, 24-bit depth RGB three-channel jpg image. Find the closed contour of the skull through Opencv, generate two masks of the skull region and the intracranial region, fill the skull region and the region outside the skull with black, and complete the skull removal operation, as shown in Figure 2 . Perform image enhancement and expansion on the skull-removed image, get 6 times expansion through horizontal and vertical flipping, random rotation, etc., and perform denoising and contrast enhancement operations on the image to make the hematoma area and intracranial tissue more distinct. Finally, the processed data is divided into training set, test set and validation set in the ratio of 7:2:1.

[0037] Step 3, build the network model. As shown in Figure 3 , the preprocessed image is processed through the ResNet50 residual network to obtain a feature image, and the 40th layer feature with obvious feature extraction effect is extracted. The obtained feature map is divided into 16 patchs with feature vectors and numbered 1-16. As shown in Figure 4As shown, a 16x16 adjacency matrix A, a 16x16 degree matrix D and a 1x16 eigenvector matrix X are generated according to the adjacency relationship between 16 patches The adjacency matrix A is added to a unit matrix to obtain a new adjacency matrix to ensure that the characteristics of the self are not lost. The degree matrix D is taken The adjacency matrix is weighted and averaged to prevent gradient explosion, giving greater weight to nodes with low degrees to reduce the influence of high-degree nodes. The feature matrix is iterated to obtain a new 1x16 feature matrix of the iterated adjacency relationship The formula is as follows:

[0038]

[0039] The matrix is generated by position coding, and then output to the Vision Transformer module composed of the multi-head attention module, multi-layer perception and classifier. After training, the classification result is output.

[0040] Step 4, training the network model. In step 4, the model training is set to 200 training times, and the number of pictures input to the network each time is about 8-16. The upper limit of the number of pictures input to the network each time is mainly determined by the performance of the computer graphics processor. Generally, the number of pictures input to the network each time is in the range of 8-16, which can make the network training more stable and the training result better. The learning rate of the training process is set to 0.0001, which can ensure the rapid fitting of the network. The network parameter optimizer selects the Adam optimizer. Its advantages mainly lie in simplicity, high efficiency, low memory demand, and the update of parameters is not affected by the scaling and transformation of gradients, making the parameters more stable. The threshold value of the loss function is set to about 0.0005, and less than 0.0005 can be considered as the completion of the training of the entire network.

[0041] Step 5, select the minimum loss function and the optimal evaluation index. In step 5, the loss function is calculated by the output of the network and the label, and the better fusion effect is achieved by minimizing the loss function. The loss function selects the cross-entropy loss function. The formula of the cross-entropy loss function is as follows:

[0042]

[0043] Where C represents the cost, x represents the sample, y represents the actual value, a represents the output value, and n represents the total number of samples.

[0044] Step 6, fine-tune the model. The image dataset is used to fine-tune the parameters of the entire network model, and the dataset is the CQ500 dataset.​

[0045] Step 7, save the model. In step 7, after the network training is completed, all parameters in the network need to be saved, and then the CT image to be classified is input into the network to obtain the segmented image. The network has no requirements for the size of the input image, and any size can be used.

[0046] Among them, the residual network, position coding, multi-head attention module, multilayer perception and the like are algorithms known to those skilled in the art, and the specific procedures and methods can be consulted in the corresponding textbooks or technical documents.

[0047] The intracranial hematoma classification network based on the convolutional neural network and the improved Vision Transformer can directly distinguish the hematoma type of the input image, and no longer needs to go through other intermediate steps, thereby avoiding the process of artificial inspection and diagnosis. The reliability of the method is further verified by comparing the data indicators of the existing deep learning method combining a convolutional neural network with two sequence models with the method. The comparison of related indicators of the prior art and the method proposed in the application is shown in Table 1:

[0048] Table 1: Comparison of related indicators of the prior art and the method proposed in the application

[0049]

[0050] As can be seen from the table, the method proposed in the application has higher accuracy, AUC and precision than the existing method, and these indicators further illustrate that the method proposed in the application has higher classification accuracy and classification performance.

Claims

1. A method for classifying intracranial hematomas based on CNN and improved VIT, characterized in that, It includes the following steps: Step 1, Prepare the dataset: Manually classify the CT image dataset of intracranial hematoma into five types: EDH, IPH, IVH, SAH, and SDH; Step 2, Dataset Preprocessing: Perform craniotomy and data augmentation on the five types of hematoma images that have been classified; Step 3, Construct the network model: Construct a network that combines the residual neural network ResNet50 and the Vision Transformer model with iterative adjacency relationships for features; Step 4, train the network model: input the preprocessed image obtained in step 2 into the network model constructed in step 3 for training; Step 5: Select the minimum loss function and the optimal evaluation metric: Minimize the loss function between the network output image and the label until the number of training iterations reaches a set threshold or the value of the loss function reaches a set range. The model parameters are then considered to have been trained and saved. At the same time, select the optimal evaluation metric to measure the accuracy of the algorithm and evaluate the performance of the system. Step 6, fine-tune the model: train and fine-tune the model using the intracranial hematoma dataset to obtain stable and usable model parameters, and further improve the model's classification ability; Step 7, Save the model: Solidify the finalized model parameters. When intracranial hematoma classification is needed, simply input the image into the network to obtain the final segmented image. In step 3, the convolutional neural network used for training is the ResNet50 convolutional neural network, which contains 49 convolutional layers and one fully connected layer, including 16 residual blocks, each consisting of three convolutional layers. The feature maps extracted from the ResNet50 convolutional neural network are divided into 16 patches, generating a 16×16 adjacency matrix and a 1×16 feature matrix. The adjacency matrix is ​​multiplied by the feature matrix to obtain an adjacency matrix. After multiplication with the diagonal matrix and standardization, the final iterative adjacency feature matrix is ​​obtained. Then, it is processed by the Vision Transformer model, which consists of position encoding, a multi-head attention module, a multilayer perceptron, and a classifier, to construct the overall network model.

2. The intracranial hematoma classification method based on CNN and improved VIT according to claim 1, characterized in that, In step 2, the CT image window is set to 100 and the window width is set to 55 to make the intracranial hemorrhage lesion area more obvious. The processed DICOM image is converted into a 512×560 RGB three-channel JPG image with a depth of 24 bits. The data deskulling operation uses OpenCV to analyze pixel features, extracts closed skull regions, and fills the skull and the area outside the skull with black that matches the image background.

3. The intracranial hematoma classification method based on CNN and improved VIT according to claim 1, characterized in that, In step 5, the cross-entropy loss function is selected as the loss function during training.

4. The intracranial hematoma classification method based on CNN and improved VIT according to claim 1, characterized in that, The datasets used in step 1 and the datasets used in step 6 for fine-tuning the model are both publicly available CQ500 datasets.

Images