A classification method for CT images of cerebral hemorrhage based on image sequence analysis

Abstract

The present invention discloses a CT image classification method for cerebral hemorrhage based on image sequence analysis. It belongs to the field of medical image analysis technology. The operation process includes the acquisition of brain CT image data, image processing of data, construction of a model and implementation of a classification method. The specific operation steps are: obtaining brain CT scan results to obtain raw brain CT image data; image processing of raw brain CT image data to form a three-channel image sequence as input of a subsequent model; constructing a model including a channel attention module and a spatial attention module to extract features of the three-channel image sequence; training the model to optimize parameters, identify positive cerebral hemorrhage and distinguish cerebral hemorrhage subtypes, and realize the classification of CT images of cerebral hemorrhage. This model can make full use of CT scan image information and improve attention to key information in the image sequence. The judgment process of the overall method conforms to the diagnosis process of doctors in real scenes, which helps to improve the classification effect of cerebral hemorrhage.

Description

Technical Field

[0001] The invention belongs to the technical field of medical image analysis and relates to a cerebral hemorrhage CT image classification method based on image sequence analysis. Background Art

[0002] With the continuous development of computer-related technologies, artificial intelligence methods are increasingly used in various fields, and medical image analysis-related work based on machine learning and deep learning methods can improve the diagnostic efficiency of clinically related diseases and reduce the workload of doctors. In the clinical diagnosis of cerebral hemorrhage, it is often based on the patient's brain computer tomography (CT) image results. This type of medical imaging performs tomographic scans of the brain at predetermined intervals and obtains multiple image layers. The scanning time is fast and the image has sufficient clarity. In clinical CT examinations, non-contrast enhanced plain scans are generally preferred. Cerebral hemorrhage often occurs suddenly, causing a strong image to the patient himself. , which itself has a high mortality rate, so rapid diagnosis is of great significance; at the same time, a large number of brain CT images need to be examined, so in the entire important and repetitive work, the radiologist responsible for reading the films is required to have a considerable degree of professional knowledge, rich experience, and a certain recognition speed and work efficiency; artificial intelligence technology is gradually playing a role in the intelligent recognition of CT images and the rapid diagnosis of cerebral hemorrhage diseases. Many of these technologies have good performance, high accuracy and fast diagnosis speed, and are expected to assist doctors in their diagnosis and treatment work in practical applications, so as to achieve the purpose of rapid screening of cerebral hemorrhage and start symptomatic treatment as soon as possible.

[0003] At present, the work related to the diagnosis of cerebral hemorrhage based on brain CT images can be mainly divided into two categories: 1) identifying and segmenting the details and morphology of the patient's cerebral hemorrhage area, and 2) screening and subtype analysis of cerebral hemorrhage for the subjects to be diagnosed. It is mainly determined by relevant classification methods whether cerebral hemorrhage occurs in the image results, and the specific type of hemorrhage is determined according to the state and characteristics of the bleeding. In the work of cerebral hemorrhage identification and segmentation under artificial intelligence technology, the training and verification of the model generally requires image-level or even pixel-level labels, which are often difficult to obtain. On the one hand, it is due to the large workload; on the other hand, the steps themselves are relatively cumbersome and rigorous. The labeling task requires doctors with certain experience, which will consume a lot of their time and energy. At the same time, the use of cerebral hemorrhage recognition and segmentation models can enable doctors to quickly obtain bleeding location information, but it is not intuitive enough to judge the type of bleeding. Therefore, a model that directly diagnoses the bleeding situation and bleeding type is often required. For this purpose, the classification model is more able to meet this demand. In the task of classifying cerebral hemorrhage, the specific objectives of the task are divided into two categories: the presence or absence of bleeding and the multi-classification of bleeding subtypes. Furthermore, in terms of label setting, part of the work is to mark the images of each scan layer of a single scanned object, and part of the work is to mark the images of each scan layer of a single scanned object. The former uses a single image to train the model and generates a diagnosis result for an independent image after converting the scanned image into an image representation suitable for artificial intelligence technology; the latter considers multiple scanned surfaces of the entire scan, and uses image sequences to train the model to obtain a diagnosis for the entire scan. It avoids discarding contextual image information in a scan sample by fusing image features in the sequence, and is more in line with the diagnosis method of clinical experts; in addition, when converting the DICOM standard format of CT scan medical images to ordinary image representation, most methods currently fix a specific window position and width that displays the entire brain state, and this single fixed value has no definite standard, and also loses some information in other windows; some methods obtain the image display at this time by setting multiple windows and fusing features in the model, but in fact the importance of windows is different and needs to be given different degrees of attention; in summary, in order to improve the efficiency of clinical diagnosis and reduce the workload of doctors, it is hoped that artificial intelligence-assisted diagnosis methods can provide direct help for the diagnosis of cerebral hemorrhage, and a CT image classification method for cerebral hemorrhage based on image sequence analysis is proposed by using more comprehensive and appropriate intra-layer and inter-layer information in the scan layer. Summary of the invention

[0004] Purpose of the invention: The purpose of the present invention is to provide a method for classifying CT images of cerebral hemorrhage based on image sequence analysis, which is of great significance for clinical diagnosis of cerebral hemorrhage by artificial intelligence. Specifically, the present invention solves the above problems to a certain extent. The method improves the utilization of image information by multi-window and attention to the whole scanning layer sequence, introduces the channel attention and spatial attention mechanism convolutional long short term memory (CLSTM) network to extract and distinguish the features in the multi-channel image sequence, and trains an end-to-end cerebral hemorrhage classification model based on the reasonable global feature representation extracted from the scanning sample, intelligently diagnoses the occurrence of cerebral hemorrhage and distinguishes the subtypes of cerebral hemorrhage, so as to provide help for the screening of cerebral hemorrhage in clinical practice.

[0005] Technical solution: The present invention discloses a method for classifying CT images of cerebral hemorrhage based on image sequence analysis. First, CT images of cerebral hemorrhage are classified. Specifically, the method comprises:

[0006] Determine whether cerebral hemorrhage has occurred from the brain CT scan, that is, judge whether the brain CT scan result is positive or negative for cerebral hemorrhage; for samples with positive cerebral hemorrhage, identify the subtype of cerebral hemorrhage, that is, judge whether the cerebral hemorrhage shown in the brain CT scan result is intraventricular hemorrhage, cerebral parenchymal hemorrhage, subarachnoid hemorrhage, epidural hemorrhage or subdural hemorrhage.

[0007] Furthermore, a brain hemorrhage CT image classification method based on image sequence analysis, the operation process of which includes the acquisition of brain CT image data, image processing of the data, construction of a model and implementation of a classification method;

[0008] Specifically: obtain the original brain CT image data from the brain CT scan results; perform imaging processing on the original brain CT image data to form a three-channel image sequence as the input of the subsequent model; construct a model including a channel attention module and a spatial attention module to extract the features of the three-channel image sequence; train the model to optimize parameters, identify positive cerebral hemorrhage and distinguish cerebral hemorrhage subtypes, and realize the classification of cerebral hemorrhage CT images.

[0009] Further, the operation steps are as follows:

[0010] (1) Processing of brain CT image data:

[0011] (1.1) obtaining brain CT scan results, storing the obtained brain CT scan results in a standard manner of DICOM medical image format, and using the stored brain CT scan results as brain CT image raw data;

[0012] (1.2) Determine the coarse-grained category label of each sample in the stored raw brain CT image data.

[0013] According to whether the high-density shadow representing the occurrence of cerebral hemorrhage is shown in the image, the original data of the brain CT image are marked as positive for cerebral hemorrhage and negative for cerebral hemorrhage, that is, ICH positive and ICH negative;

[0014] (1.3) Determine the fine-grained category labels of each sample of the marked ICH-positive brain CT image raw data;

[0015] (1.4) Perform image processing on all raw brain CT image data in DICOM format, and extract each sample in the raw brain CT image data into a three-channel image sequence;

[0016] (2) Model construction:

[0017] (2.1) Using the channel attention module, weights are assigned to the features extracted from the three-channel image sequence, and then the weighted features are fused;

[0018] (2.2) Using the CLSTM network with a spatial attention module, we generate spatial region attention for the fused features, extract the spatiotemporal features of the three-channel image sequence, and classify them;

[0019] (3) Implementation of classification method:

[0020] The three-channel image sequence obtained after image processing is used as the input of the constructed model, and the parameters in the model are trained using the determined coarse-grained and fine-grained category labels to obtain the final model for cerebral hemorrhage CT image classification.

[0021] Furthermore, in step (1.3), the fine-grained category label of each sample of the ICH positive data is determined specifically as follows: based on the position and shape of the high-density shadows shown in the image, the ICH positive brain CT image raw data is labeled as five subtypes of cerebral hemorrhage, including intraventricular hemorrhage, cerebral parenchymal hemorrhage, subarachnoid hemorrhage, epidural hemorrhage and subdural hemorrhage, namely IVH, IPH, SAH, EDH and SDH.

[0022] Furthermore, in step (1.4), the imaging processing of all DICOM format brain CT image raw data is specifically: setting different window positions and window widths, selecting the whole brain window, blood window and skull window respectively, converting the information within the corresponding HU value range into a common image representation, so that each sample in the brain CT image raw data is extracted into a three-channel image sequence, wherein the channel obtained after processing is in the image PNG format, and the sequence length is the number of tomographic scan layers in the sample brain CT image raw data.

[0023] Beneficial effects: Compared with the prior art, the present invention is characterized in that the method of the present invention splits and differentiates CT image information, and improves the diagnostic ability of the model by obtaining high-level feature representations of channel key information, spatial key information, and timeline context information. Overall, the method comprehensively simulates the way doctors view image results, while paying more attention to key CT window channels and key areas in the image, and comprehensively pays attention to each scanning layer, capturing the spatial dimension information contained in the medical image data and the contextual relationship of each scan on the timeline, and finally obtaining a reasonable diagnosis result. In practical applications, after dividing the training set, according to the label information, the corresponding models are trained for whether bleeding occurs and whether bleeding can be diagnosed as five subtypes, and the models will be used in their respective diagnostic tasks. Specifically for the sample to be diagnosed, the coarse-grained diagnosis model of bleeding is first used to obtain the result of whether there is cerebral hemorrhage in the image. If so, five fine-grained models are used to perform subdivision diagnosis of cerebral hemorrhage subtypes, and the output results of each model are combined to obtain the final brain CT image classification result for cerebral hemorrhage disease. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] Figure 1 It is an operation flow chart of the present invention;

[0025] Figure 2 It is a schematic diagram of the operation of imaging processing of brain CT image data in the present invention;

[0026] Figure 3 It is a schematic diagram of the operation of extracting three-channel image sequence features by using the channel attention module in the model construction of the present invention;

[0027] Figure 4 It is a schematic diagram of the operation of using the spatial attention module to extract features and complete classification in the model construction of the present invention. DETAILED DESCRIPTION

[0028] The present invention will be further described below in conjunction with the accompanying drawings and embodiments.

[0029] As shown in the figure, the method for classifying CT images of cerebral hemorrhage based on image sequence analysis of the present invention first classifies CT images of cerebral hemorrhage; specifically:

[0030] Determine whether cerebral hemorrhage has occurred from the brain CT scan, that is, judge whether the brain CT scan result is positive or negative for cerebral hemorrhage; for samples with positive cerebral hemorrhage, identify the subtype of cerebral hemorrhage, that is, judge whether the cerebral hemorrhage shown in the brain CT scan result is intraventricular hemorrhage, cerebral parenchymal hemorrhage, subarachnoid hemorrhage, epidural hemorrhage or subdural hemorrhage.

[0031] Furthermore, a brain hemorrhage CT image classification method based on image sequence analysis, the operation process of which includes the acquisition of brain CT image data, image processing of the data, construction of a model and implementation of a classification method;

[0032] Specifically: obtain the original brain CT image data from the brain CT scan results; perform imaging processing on the original brain CT image data to form a three-channel image sequence as the input of the subsequent model; construct a model including a channel attention module and a spatial attention module to extract the features of the three-channel image sequence; train the model to optimize parameters, identify positive cerebral hemorrhage and distinguish cerebral hemorrhage subtypes, and realize the classification of cerebral hemorrhage CT images.

[0033] Specifically, in the brain CT image data processing part, the acquired brain CT scan is stored as brain CT imaging data based on the DICOM medical image format standard, and a category label is determined for each object sample; according to the coarse-grained category, it is divided into intracerebral hemorrhage positive and intracerebral hemorrhage negative, i.e., ICH (Intracerebral hemorrhage) positive and ICH negative, depending on whether cerebral hemorrhage can be observed in the brain CT image data;

[0034] There are five specific subtypes of intracerebral hemorrhage: intraventricular hemorrhage, intracerebral parenchymal hemorrhage, subarachnoid hemorrhage, epidural hemorrhage, and subdural hemorrhage.

[0035] According to the fine-grained bleeding subtype, ICH-positive samples were marked as IVH (Intraventricular hemorrhage), IPH (Intraparenchymal hemorrhage), SAH (Subarachnoid hemorrhage), EDH (Epidural hematoma), and SDH (Subdural hematoma).

[0036] Then, in order to make the data recognizable by the model method, the brain CT medical imaging data in DICOM format was processed into images according to the clinical reading method;

[0037] In the medical field, different tissues have different densities, resulting in different absorption of X-rays. This degree is measured using Hounsfield units (HU). By setting a pair of window level (WL) and window width (WW), a window is defined. The lower limit of the window is W. min and upper bound W max They are:

[0038]

[0039]

[0040] Tissues and lesions with HU values ​​above this range will appear as white shadows in the grayscale image, while those below this range will appear as black shadows. HU values ​​within this window range will correspond to gray levels in the range of 0 to 255. The closer to 255, the brighter it appears. The corresponding method is:

[0041]

[0042] Where, I is the indicator function;

[0043] In this way, the tissue structures within the range will be distinguished by different grayscales, and the grayscale image corresponding to the window can be obtained; the whole brain window (WL=50, WW=100), hematoma window (WL=70, WW=50) and skull window (WL=500, WW=3000) are selected respectively, and the information within the corresponding HU value range is converted into a common image representation, such as Figure 2 As shown, a three-channel medical image is formed for a single sample data, where the length of the image sequence is L i is equal to the number of tomographic slices contained in the CT scan sample of the subject;

[0044] On the other hand, the distances between tomographic scans may be different in the samples. In order to ensure that the diagnostic task of the model is not disturbed as much as possible, the resampling technology is used to fix the spacing on the z-axis and sample all data into a consistent standard interval. For the sampled image sequences of unequal lengths, since the framework of the network is fixed, the sequence needs to be cropped to a fixed length, that is, n windows of length L are randomly intercepted on the three channels of the sequence at the same time, and n L-length subsequences are obtained on each sample. The labels of the subsequences are consistent with the source sequence. Finally, N three-channel image sequence samples of length L are obtained from all diagnostic objects for training the channel and spatial dual attention CLSTM network model.

[0045] In the construction of the model, the main components of the model mainly include: 1) Using the channel attention module, the corresponding weights are assigned to the features extracted from the processed three-channel image sequence, and the features are fused based on different attention levels. The specific structure is as follows Figure 3 As shown;

[0046] 2) Using the CLSTM network with a spatial attention module, the spatial region attention is generated for the aforementioned fused feature map, and the spatiotemporal features of the sequence are extracted for classification. The specific structure is as follows: Figure 4 As shown;

[0047] Among them, after data preprocessing, a total of N labeled subsequences are obtained, denoted as X n represents a subsequence;

[0048] Where n∈{1,2,…,N}; the fixed-length subsequence is used as the image sequence input of the model, and the length of each subsequence is L, denoted as {X n :X n,1 ,X n,2 ,…,X n,L , n=1,2,…,N}, where each image of the subsequence consists of three channel images obtained under the whole brain window, hematoma window and skull window,

[0049] Therefore, a single image contained in a subsequence can be represented in the form of three channels:

[0050] First, each channel of each image in the sequence is input into a convolution with batch normalization (BN) and Relu activation, where the convolution kernel size is 2×2 and the step size is 2, and the feature maps of the three channels are obtained respectively.

[0051] Then, channel attention is used to assign channel-dimensional weights to the feature maps of a single scanned face. The specific implementation modules include parallel global max pooling (GMP) and global average pooling (GAP), multi-layer perceptron (MLP) and a sigmoid activation function; the input feature map is compressed into two one-dimensional vectors in the spatial dimension after passing through GMP and GAP, and the two vectors are input into the multi-layer perceptron activated by Relu to fuse the obtained results;

[0052] Finally, after sigmoid activation, the channel weight vector is obtained, whose vector length is equal to the number of input channels. The whole process is expressed as:

[0053] CW n,t = CAtt(f n,t )

[0054] =Sigmoid(MLP(GMP(f n,t ))+MLP(GAP(f n,t )))

[0055] The attention weights of each channel are superimposed on their respective features, and the obtained weighted feature maps are superimposed pixel by pixel to obtain the feature representation of the current scanned surface that fuses the three channels:

[0056]

[0057] Will be composed of L F n,t The composed feature image sequence is input into the CLSTM network that can capture spatiotemporal features; the network contains a CLSTM layer with spatial attention, a normal CLSTM layer, a global average pooling layer, a fully connected (FC) layer and a sigmoid activation function;

[0058] Specifically, in the CLSTM layer, for the subsequence sample X n , input the current two-dimensional feature map F into the network at the tth time step n,t , the hidden state obtained by CLSTM calculation of the first layer The calculation method is:

[0059]

[0060]

[0061]

[0062]

[0063]

[0064]

[0065] In the formula, represents the input gate, represents the forget gate, represents the output gate, represents a new memory cell, represents the final updated memory unit, ⊙ represents the Hadamard product operation between matrices, and Conv represents the convolution operation;

[0066] in, represents the weight parameter to be trained in the convolution operation of the corresponding gate, b * Represents the bias inside the gate; the CLSTM layer that incorporates the spatial attention mechanism is based on the initial input F at the time step n,t and the current hidden state obtained after passing through the general CLSTM layer Get the spatial weight; F n,t and The features are combined into a two-channel feature by splicing and cascading, and a hidden layer with a convolution kernel of size 1×1×2 and a step size of 1 is used for convolution operation. After sigmoid activation, the spatial attention weight SW corresponding to the feature map is obtained. n,t , the calculation process is expressed as:

[0067]

[0068] In the formula, Represents the concatenation of two features in dimension, and b are the parameters that need to be trained in the convolution, and the obtained attention weight SW n,t With F n,t have the same size, the SW n,t and F n,t Superposition with Hadamard product can obtain features with spatial distinction attention Input to the general CLSTM layer of the next layer to calculate the hidden state

[0069] Input the two-dimensional feature map containing channel attention in the sequence in sequence according to the time step, and use the model to model the attention weight and spatiotemporal features to get the last time step F n,L The hidden state of It contains the overall information of the image sequence; therefore, as the high-level information extracted from the entire CT scan sequence, it is very important for The GAP operation is performed, and then the result is input into the FC layer. After the sigmoid activation function, the classification result of each sequence is obtained, which is expressed as the probability of belonging to the class.

[0070] The above are only preferred embodiments of the present invention. The protection scope of the present invention is not limited to the above embodiments. All technical solutions under the concept of the present invention belong to the protection scope of the present invention. It should be pointed out that for ordinary technicians in this technical field, some improvements and modifications without departing from the principle of the present invention should be regarded as the protection scope of the present invention.

Claims

1. A method for classifying CT images of cerebral hemorrhage based on image sequence analysis, characterized in that: First, the CT images of cerebral hemorrhage are classified; specifically: Determine whether cerebral hemorrhage has occurred from the brain CT scan, that is, determine whether the brain CT scan result is positive for cerebral hemorrhage or negative for cerebral hemorrhage; for samples with positive cerebral hemorrhage, identify the subtype of cerebral hemorrhage, that is, determine whether the cerebral hemorrhage shown in the brain CT scan result is intraventricular hemorrhage, cerebral parenchymal hemorrhage, subarachnoid hemorrhage, epidural hemorrhage or subdural hemorrhage; The operation process includes the acquisition of brain CT image data, image processing of data, construction of models and implementation of classification methods; specifically: obtaining brain CT scan results to obtain raw brain CT image data; image processing of raw brain CT image data to form a three-channel image sequence as input for subsequent models; constructing a model including a channel attention module and a spatial attention module to extract features of the three-channel image sequence; training the model to optimize parameters, identify positive cerebral hemorrhage and distinguish cerebral hemorrhage subtypes, and realize the classification of cerebral hemorrhage CT images; The specific operation steps are as follows: (1) Processing of brain CT image data: (1.1) Obtain brain CT scan results, store the obtained brain CT scan results in a standard manner in the DICOM medical image format, and use the stored brain CT scan results as raw brain CT image data; (1.2) Determine the coarse-grained category label of each sample in the stored raw brain CT image data. According to whether the high-density shadow representing the occurrence of cerebral hemorrhage is shown in the image, the original data of the brain CT image are marked as positive for cerebral hemorrhage and negative for cerebral hemorrhage, that is, ICH positive and ICH negative; (1.3) Determine the fine-grained category labels of each sample of the marked ICH-positive brain CT image raw data; (1.4) Perform image processing on all raw brain CT image data in DICOM format, and extract each sample in the raw brain CT image data into a three-channel image sequence; (2) Model construction: (2.1) Use the channel attention module to assign weights to the features extracted from the three-channel image sequence, and then fuse the weighted features; (2.2) Using the CLSTM network with a spatial attention module, we generate spatial region attention for the fused features, extract the spatiotemporal features of the three-channel image sequence and classify them; (3) Implementation of classification method: The three-channel image sequence obtained after image processing is used as the input of the constructed model, and the parameters in the model are trained using the determined coarse-grained and fine-grained category labels to obtain the final model for cerebral hemorrhage CT image classification.

2. The method for classifying CT images of cerebral hemorrhage based on image sequence analysis according to claim 1, characterized in that: In step (1.3), the fine-grained category label of each sample of the ICH positive data is determined as follows: based on the position and morphological characteristics of the high-density shadows shown in the image, the ICH positive brain CT image raw data are labeled as five subtypes of cerebral hemorrhage, including: intraventricular hemorrhage, cerebral parenchymal hemorrhage, subarachnoid hemorrhage, epidural hemorrhage and subdural hemorrhage, namely, IVH, IPH, SAH, EDH and SDH.

3. The method for classifying CT images of cerebral hemorrhage based on image sequence analysis according to claim 1, characterized in that: In step (1.4), the imaging processing of all DICOM format brain CT image raw data is specifically: setting different window positions and window widths, selecting the whole brain window, blood window and skull window respectively, converting the information within the corresponding HU value range into a common image representation, so that each sample in the brain CT image raw data is extracted into a three-channel image sequence, wherein the channel obtained after processing is in the image PNG format, and the sequence length is the number of tomographic scan layers in the sample brain CT image raw data.

Images