Prediction method for volume change of brain hematoma

Abstract

The invention relates to the field of brain hematoma analysis, and discloses a brain hematoma volume change prediction method, which comprises the following steps of collecting and sorting basic data information of an acute cerebral hemorrhage patient; extracting initial vector features of the text and the image; the text features and the image features are sent into a bidirectional multi-mode attention block, and dependence between the text and the image is captured; the text features and the image features are connected, and then self-attention calculation is carried out; according to the scheme, for an acute cerebral hemorrhage patient, timely evaluation can be carried out according to historical medical data of the patient, so that judgment can be quickly and effectively made within limited time, and the accuracy of the acute cerebral hemorrhage patient is improved. Doctors are assisted in treatment, prognosis is improved, and finally the life quality of postoperative patients is improved.

Description

Technical Field

[0001] The invention relates to the field of brain hematoma analysis, and in particular to a method for predicting brain hematoma volume changes. Background Art

[0002] Spontaneous cerebral hemorrhage refers to bleeding in the brain parenchyma caused by non-traumatic factors, mostly caused by hypertension. The disease progresses rapidly and varies greatly. The changes in hematoma volume and the uncertainty of lesion outcome are also large. It is often impossible to observe the changes in the patient's condition in time during clinical work. Early and rapid identification of high-risk patients with hematoma volume expansion can take appropriate intervention measures to improve prognosis. Clinicians use the human brain to assess the patient's condition and use the naked eye to interpret medical images to discover, describe, monitor the disease, and formulate emergency plans. However, this assessment is often based on knowledge reserves and work experience, with strong subjective dependence, and individual differences cannot be avoided. Young frontline doctors may misdiagnose, mislead, delay the disease, and affect treatment when facing critical cases. However, in intelligent medical diagnosis based on machine learning, how to effectively interpret clinical information and medical images is still worth studying. At present, medical analysis and auxiliary diagnosis based on simple image information or text information has been widely used, but the analysis and diagnosis method combined with the fusion of image and text modalities has better flexibility and applicability. For this reason, we propose a prediction method for changes in brain hematoma volume. Summary of the invention

[0003] 1. Technical issues to be resolved

[0004] In view of the shortcomings of the prior art, the present invention provides a method for predicting changes in cerebral hematoma volume. Multimodal data integration is performed based on the medical images, medical history, past history and laboratory test data of existing patients with acute cerebral hemorrhage, and a label of the hematoma volume change is given to each group of data. A prediction model is trained in a supervised manner. During the prediction, the patient's relevant data information can be input, so that high-risk patients with hematoma volume expansion can be quickly and effectively identified, and corresponding intervention measures can be taken to improve the prognosis.

[0005] (II) Technical solution

[0006] To achieve the above-mentioned purpose, the present invention provides the following technical solution: a method for predicting the volume change of cerebral hematoma, comprising the following steps:

[0007] Step 1: Collect and organize basic data information of patients with acute cerebral hemorrhage;

[0008] Step 2: Extract preliminary vector features of text and images;

[0009] Step 3: Feed the text features and image features into the bidirectional multimodal attention block to capture the dependency between text and image;

[0010] Step 4: Connect text features and image features, and then calculate self-attention;

[0011] Step 5: Perform classification training on the feature data using the fully connected classification layer, calculate the error with the classification label, and obtain the prediction model through cross entropy back propagation;

[0012] Step 6: Input medical data to predict the changes in hematoma volume in patients with acute cerebral hemorrhage. For high-risk patients with increased hematoma volume, appropriate intervention measures can be taken to improve prognosis.

[0013] Preferably, the text features in the second step are obtained by encoding the patient's structured and unstructured data information through BERT to obtain a text vector.

[0014] Preferably, the data features in the second step are extracted through a pre-trained model for CT and MRI brain hemorrhage segmentation, and the final feature vector is taken as the feature of the medical image.

[0015] Preferably, the specific contents of the third step are as follows:

[0016] 1) The input text vector and image vector undergo a linear mapping to obtain two sets of weight vectors K, Q, V with the same feature length as the input vector;

[0017] 2) Multiply Q and K, calculate the similarity, and get the weight distribution;

[0018] 3) The weight distribution is normalized by softmax;

[0019] 4) The weight distribution is multiplied by V and weighted summed; through the self-attention mechanism, the unpredictable

[0020] The relationship between the same position and since;

[0021]

[0022] Q I With K I , V I Calculate the attention to capture the dependencies between images, Q I With K T , V T Computational attention captures the dependencies between images and text;

[0023] Q T With K T , V T Calculate the attention to capture the dependencies between texts, Q T With K I , V IComputational attention captures the dependencies between text and images.

[0024] Preferably, the λ is a natural number, and in the first bidirectional multimodal attention layer X I and X T Take the average and send it to the second-layer bidirectional multimodal attention block. The second-layer bidirectional multimodal attention layer X I and X T After taking the average, they are concatenated and sent to the self-attention block.

[0025] (III) Beneficial effects

[0026] Compared with the prior art, the present invention provides a method for predicting the volume change of cerebral hematoma, which has the following beneficial effects:

[0027] 1. The prediction method of brain hematoma volume change. In order to accurately evaluate the changes in hematoma volume in patients with acute cerebral hemorrhage, clinicians usually need to consider multimodal information such as the patient's chief complaint, medical imaging, and laboratory test results. However, this evaluation is often based on knowledge reserves and work experience, with strong subjective dependence, and individual differences are inevitable. Young frontline doctors may misdiagnose, mislead, delay the condition, and use imaging treatment when facing critical cases. For patients with acute cerebral hemorrhage, this solution can conduct timely evaluation based on the patient's historical medical data, so as to make quick and effective judgments within a limited time, assist doctors in treatment, improve prognosis, and ultimately improve the quality of life of patients after surgery.

[0028] 2. Compared with the existing prediction and evaluation methods using other statistical learning methods, the prediction method of brain hematoma volume change can only perform statistical analysis on the information of a certain modality and cannot perform multi-modal evaluation, which may not be comprehensive. This solution is more comprehensive.

[0029] 3. The prediction method of the volume change of brain hematoma, BRIEF DESCRIPTION OF THE DRAWINGS

[0030] Figure 1 A schematic diagram for collating and collecting basic data information of patients with acute cerebral hemorrhage;

[0031] Figure 2 Schematic diagram of multimodal model training;

[0032] Figure 3 It is a schematic diagram of the process of the present invention. DETAILED DESCRIPTION

[0033] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0034] See also Figure 1-3 , a method for predicting the volume change of brain hematoma, comprising the following contents:

[0035] First, the basic data of patients with acute cerebral hemorrhage were collected: 1. Demographic information of patients such as age, gender, smoking and drinking; 2. Past medical history and medication history, personal history, and physical signs at admission; 3. Whether the patient has diabetes, hypertension, atrial fibrillation, heart failure, myocardial infarction, cerebrovascular disease, chronic obstructive pulmonary disease and other past medical history information; 4. The first laboratory test of the patient after admission (blood cell analysis, liver function, kidney function, ions, random blood sugar, blood coagulation, etc.); 5. Clinical examination data after admission such as swallowing dysfunction and National Institutes of Health Neurological Deficit Score (NIHSS). 6. Patients' head CT imaging manifestations (including hematoma volume, bleeding site, whether it has broken into the ventricle, midline shift), chest CT, head MR, cerebrovascular CTA and other information.

[0036] Encode the patient's structured and unstructured data information to complete text vectorization. Bert is a pre-trained language model based on the Transformer architecture. This tool can be used to encode structured and unstructured data directly. The chief complaint is obtained through Bert to obtain a word vector combination with a maximum length of 40. The laboratory examination is obtained through Bert to obtain a word vector combination with a maximum length of 92. The demographic information is obtained through Liner to obtain a one-hot vector with a length of 10, and finally a 142-dimensional text vector is obtained. For medical imaging data, feature extraction is performed through a pre-trained model for CT and MRI brain hemorrhage segmentation, and the final feature vector is taken as the feature of the medical image.

[0037] After extracting the preliminary vector features of the text and image, the bidirectional multimodal attention block shown below is used to calculate the attention between different modalities. 1) The input text vector and image vector are linearly mapped to obtain two sets of weight vectors K, Q, V with the same length as the input vector feature; 2) Q is multiplied by K to calculate the similarity and obtain the weight distribution; 3) The weight distribution is normalized by softmax; 4) The weight distribution is multiplied by V and weighted summed; After the self-attention mechanism, the relationship and relationship between different positions in the input sequence can be captured. Here Q I With K I , VI Calculate the attention to capture the dependencies between images, Q I With K T , V T Computational attention captures the dependencies between images and text; Q T With K T , V T Calculate the attention to capture the dependencies between texts, Q T With K I , V I Computational attention captures the dependencies between text and images.

[0038]

[0039] In this scheme, λ is taken as 1, and X I and X T Take the average and send it to the second-layer bidirectional multimodal attention block. The second-layer bidirectional multimodal attention layer X I and X T After taking the average, they are concatenated and sent to the self-attention block.

[0040] Multimodal model training:

[0041] 1) Collect and organize the data of patients with acute cerebral hemorrhage, and mark the changes of hematoma according to the patients, with different levels of change ranging from 0 to 10;

[0042] 2) Extract structured and unstructured text feature vectors based on BERT;

[0043] 3) Extract feature vectors of medical images based on pre-trained medical image segmentation networks;

[0044] 4) Feeding text features and image features into a bidirectional multimodal attention block to capture the dependency between text and image;

[0045] 5) After the bidirectional multimodal attention block, the text features and image features are connected, and then the self-attention is calculated;

[0046] 6) Finally, the feature data is classified and trained through a fully connected classification layer, and the error is calculated with the classification label, and the prediction model is obtained through cross entropy back propagation.

[0047] After the model is trained, medical images, clinical data, chief complaints and other information can be input to quickly predict the changes in hematoma volume in patients with acute cerebral hemorrhage. For high-risk patients with increased hematoma volume, appropriate intervention measures can be taken to improve prognosis.

[0048] Although embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions and variations may be made to the embodiments without departing from the principles and spirit of the present invention, and that the scope of the present invention is defined by the appended claims and their equivalents.

Claims

1. A method for predicting changes in brain hematoma volume, characterized in that: The following steps are involved: Step 1: Collect and organize basic data information of patients with acute cerebral hemorrhage; Step 2: Extract preliminary vector features of text and images; Step 3: Feed the text features and image features into the bidirectional multimodal attention block to capture the dependency between text and image; Step 4: Connect text features and image features, and then calculate self-attention; Step 5: Perform classification training on the feature data using the fully connected classification layer, calculate the error with the classification label, and obtain the prediction model through cross entropy back propagation; Step 6: Input medical data to predict the changes in hematoma volume in patients with acute cerebral hemorrhage. For high-risk patients with increased hematoma volume, appropriate intervention measures can be taken to improve prognosis.

2. The method for predicting the volume change of cerebral hematoma according to claim 1, characterized in that: The text features in the second step are encoded by BERT on the patient's structured and unstructured data information to obtain text vectors.

3. The method for predicting the volume change of cerebral hematoma according to claim 1, characterized in that: The data features in the second step are extracted through a pre-trained model for CT and MRI brain hemorrhage segmentation, and the final feature vector is taken as the feature of the medical image.

4. The method for predicting the volume change of cerebral hematoma according to claim 1, characterized in that: The specific contents of the third step are as follows: S1: The input text vector and image vector undergo a linear mapping to obtain two sets of weight vectors K, Q, V with the same feature length as the input vector; S2: Multiply Q and K, calculate the similarity, and get the weight distribution; S3: weight distribution is normalized by softmax; S4: The weight distribution is multiplied by V and weighted summed; through the self-attention mechanism, the relationship and relationship between different positions in the input sequence can be captured; Q I With K I , V I Calculate the attention to capture the dependencies between images, Q I With K T , V T Computational attention captures the dependencies between images and text; Q T With K T , V T Calculate the attention to capture the dependencies between texts, Q T With K I , V I Computational attention captures the dependencies between text and images.

5. The method for predicting the volume change of cerebral hematoma according to claim 4, characterized in that: The λ is a natural number, and in the first bidirectional multimodal attention layer X I and X T Take the average and send it to the second-layer bidirectional multimodal attention block. The second-layer bidirectional multimodal attention layer X I and X T After taking the average, they are concatenated and sent to the self-attention block.

Images