Method for establishing post-hemorrhagic functional prediction model based on multi-knowledge fusion model

Abstract

The application relates to a cerebral hemorrhage post-function prediction model establishment method, in particular to a cerebral hemorrhage post-function prediction model establishment method based on a multi-knowledge fusion model. The method is used for predicting various function grades of a cerebral hemorrhage patient after 48 hours of operation. The prediction model establishment method realizes the prediction of the function rehabilitation grade by building a multi-knowledge fusion network model. The method comprises the following steps: data preprocessing, multi-knowledge fusion network model construction, model training and prediction.

Description

Technical Field

[0001] The present invention relates to a method for establishing a function prediction model after cerebral hemorrhage, and in particular to a method for establishing a function prediction model after cerebral hemorrhage based on a multi-knowledge fusion model. Background Art

[0002] Recovery after intracerebral hemorrhage surgery needs to be predicted in order to study the relationship between surgical plans and patient recovery. Common functions that need to be assessed include: language impairment, hearing impairment, visual impairment, cognitive impairment, and physical impairment.

[0003] Among them, the levels of language disability are: Level 1 refers to those who can only make simple pronunciations and have completely lost the ability to speak; Level 2 refers to those who have a certain pronunciation ability and the speech clarity is between 10% and 30%; Level 3 refers to those who have the pronunciation ability and the speech clarity is between 31% and 50%.

[0004] The degree of cognitive impairment was assessed based on the Mini-Mental State Examination (MMSE): normal (27-30 points); cognitive impairment (<27 points); mild (21-26 points); moderate (10-20 points); and severe (0-9 points).

[0005] Limb disability level: Level 1 includes lower limb paraplegia, hemiplegia or complete loss of limb function on one side; Level 2 includes bilateral lower limb paraplegia or severe functional impairment of both limbs; Level 3 includes bilateral lower leg amputation or limb loss, etc.

[0006] Hearing impairment levels: 26-40dBHL is mild, 41-60dBHL is moderate, 61-80dBHL is severe, and ≥81dBHL is profound. Visual impairment levels: Level 1 refers to no light perception, or a visual acuity within 0.02, and a visual field radius of less than 5 degrees; Level 2 refers to a visual acuity of 0.02-0.05, or a visual field radius of less than 10 degrees; Level 3 refers to 0.02-0.1; and Level 4 refers to 0.1-0.4.

[0007] It can be seen from this that the prediction of the patient's body function condition is a multi-task prediction.

[0008] Different areas of the brain are responsible for different functions (such as Figure 1 The frontal lobe is responsible for thinking and planning, and is associated with individual needs and emotions. The parietal lobe responds to sensations of pain, touch, taste, temperature, and pressure, and is also associated with mathematics and logic. The temporal lobe processes auditory information and is associated with memory and emotion. The occipital lobe processes visual information. The cerebellum is a key regulatory center for movement, with numerous afferent and efferent connections. This indicates that imaging changes in different brain regions before and after surgery are closely related to the patient's functional recovery.

[0009] Therefore, in order to integrate more prior knowledge to better assist multi-task prediction, the application provides a brain hemorrhage post-function prediction model establishment method based on a multi-knowledge fusion model. SUMMARY

[0010] The application is aimed at the defects in the prior art, and provides a brain hemorrhage post-function prediction model establishment method based on a multi-knowledge fusion model. The prediction model establishment method is used for predicting various function levels of a brain hemorrhage patient 48 hours after surgery. The prediction model establishment method realizes the prediction of the function rehabilitation level by building a multi-knowledge fusion network model.

[0011] To achieve the above object, the application adopts the following technical scheme, comprising:

[0012] Data preprocessing.

[0013] Multi-knowledge fusion network model construction.

[0014] Model training and prediction.

[0015] Further, the data preprocessing is performed on brain images, including preoperative NCCT and postoperative NCCT, and comprises the following steps:

[0016] S1, a rigid registration algorithm is used to align the postoperative NCCT image and the preoperative NCCT image.

[0017] S2, based on a skull stripping algorithm, the preoperative NCCT is used to obtain a brain tissue mask.

[0018] S3, based on a non-rigid registration algorithm and a template of five brain partitions, the preoperative NCCT is used to obtain five partition masks corresponding to the five brain partitions.

[0019] S4, the skull-removed preoperative NCCT and postoperative NCCT are normalized in window width and window level [0, 200] Hu.

[0020] Further, the multi-knowledge fusion network model construction comprises taking the processed preoperative and postoperative NCCT and the corresponding partition masks as inputs of the network model, and outputting the impairment levels corresponding to five tasks; the network model comprises three modules: a preoperative and postoperative difference feature extraction module, a prior knowledge fusion module, and a multi-task prediction module.

[0021] Further, the preoperative and postoperative difference feature extraction module includes basic feature extraction on preoperative and postoperative NCCT images respectively, and is realized through 3*3*3 convolution with 2 steps [2, 2, 2] and 1 step [2, 2, 1], and a relu activation layer and a batchnorm regular layer are used after each convolution; after obtaining X1 and X2 features, the feature difference of the two is extracted through a DIFF layer, as a shared feature, that is, X = X2-X1, DX = (X-min(X)) / (max(X)-min(X)). Wherein, X2 is the feature after postoperative processing, X1 is the feature after preoperative processing, X represents the difference value of the feature, and DX represents the difference feature.

[0022] Further, the prior knowledge fusion module includes downsampling the brain partition mask to the size (64, 64, 6) of the preoperative and postoperative difference feature, and then point multiplying with DX respectively to obtain the preoperative and postoperative difference features corresponding to different brain regions; according to the functional characteristics, the order is frontal lobe, parietal lobe, cerebellum, temporal lobe and occipital lobe; (Because individual functions require interaction of different regions, such as cerebellum and parietal lobe both have certain influence on limb movement.) Through BiLSTM, the features of different regions are interacted to obtain the gated features with introduced prior knowledge.

[0023] Further, the multi-task prediction module includes based on the gated features, through 1 convolution block (including convolution with step [2, 2, 2], relu, batchnorm), 1 full connection block (including full connection, dropout), 1 softmax layer, the obstacle level prediction of different functions is carried out.

[0024] Further, the training and prediction of the model include: in the training process of the model, the model is optimized by using Adam; the learning rate is 0.01, and is reduced to 0.1 times of the original after every 50 rounds; 5 loss functions adopt cross entropy loss, and the corresponding weight is the current category number / total category number; the output result of each task in the test stage is the obstacle level corresponding to the maximum probability.

[0025] Compared with the prior art, the present application has the following advantages.

[0026] The present application can predict the recovery of patients 48 hours after cerebral hemorrhage surgery and output 5 obstacle levels. That is, by building a multi-knowledge fusion network model, the functional rehabilitation level is predicted. BRIEF DESCRIPTION OF DRAWINGS

[0027] The present application will be further described below in combination with the drawings and specific embodiments. The protection scope of the present application is not limited to the following content.

[0028] Figure 1 It is a brain function area schematic diagram.

[0029] Figure 2 is a multi-knowledge fusion prediction model schematic diagram. DETAILED DESCRIPTION

[0030] The method for establishing a post-hemorrhagic functional prediction model based on a multi-knowledge fusion model comprises data preprocessing, multi-knowledge fusion network model construction Figure 2 ), model training and prediction.

[0031] 1.

Data preprocessing

[0032] (1) Based on the rigid registration algorithm, the postoperative NCCT image is aligned with the preoperative NCCT.

[0033] (2) Based on the skull stripping algorithm, the brain tissue mask is obtained using the preoperative NCCT.

[0034] (3) Based on the non-rigid registration algorithm and the known brain 5-partition template, the corresponding 5-partition mask is obtained using the preoperative NCCT.

[0035] (4) The de-skulled preoperative and postoperative NCCT is normalized by window width and window level [0, 200] Hu.

[0036] 2.

Multi-knowledge fusion network model construction

[0037] (1) Preoperative and postoperative difference feature extraction module.

[0038] (2) Prior knowledge fusion module.

[0039] (3) Multi-task prediction module.

[0040] Among them, (1) respectively extracts the basic features of the preoperative and postoperative NCCT images, mainly through 3*3*3 convolution with 2 steps [2, 2, 2] and 1 step [2, 2, 1], and uses relu activation layer and batchnorm regularization layer after each convolution. After obtaining X1 and X2 features, the feature difference between the two is extracted as shared features, that is, X = X2-X1, DX = (X-min(X)) / (max(X)-min(X)).

[0041] (2) The brain region mask is sampled to the above feature size (64, 64, 6) and multiplied by DX to obtain the preoperative and postoperative difference features corresponding to different brain regions. According to the functional characteristics, the sequence is frontal lobe, parietal lobe, cerebellum, temporal lobe, and occipital lobe. Due to the need for interaction of different regions for individual functions, such as the cerebellum and parietal lobe both having certain influence on limb movement. Therefore, the features of different regions are interacted through BiLSTM to obtain the prior knowledge introduced gated features.

[0042] (3) Based on the extracted features, a convolution block (including convolution with a step of [2, 2, 2], relu, batchnorm), a full connection block (including full connection, dropout), and a softmax layer are used to predict the levels of different functions.

[0043] 3.

Model training and prediction

[0044] It can be understood that the above specific description of the present application is only used to illustrate the present application and is not limited to the technical solutions described in the embodiments of the present application. Those skilled in the art should understand that the present application can still be modified or replaced equivalently to achieve the same technical effect; as long as the use needs are met, it is within the protection scope of the present application.

Claims

1. A method for establishing a post-hemorrhagic functional prediction model based on a multi-knowledge fusion model, characterized in that, The application relates to a method for predicting the level of brain injury, and belongs to the field of medical image processing. The method comprises the following steps: Data preprocessing; Multi-knowledge fusion network model construction; Model training and prediction. The multi-knowledge fusion network model construction comprises the following steps: taking the preoperative and postoperative NCCT and the corresponding brain partition mask after processing as the input of the network model, and outputting the obstacle levels corresponding to five tasks; the network model comprises three modules: a preoperative and postoperative difference feature extraction module, a priori knowledge fusion module and a multi-task prediction module. The priori knowledge fusion module comprises the following steps: down-sampling the brain partition mask to the size of the preoperative and postoperative difference feature (64, 64, 6), and then respectively multiplying the preoperative and postoperative difference feature with DX to obtain the preoperative and postoperative difference feature corresponding to different brain regions; the sequence is frontal lobe, parietal lobe, cerebellum, temporal lobe and occipital lobe according to the functional characteristics; and information interaction of the features of different regions is carried out through BiLSTM to obtain the gated feature with the introduction of priori knowledge. DX=(X-min(X)) / (max(X)-min(X)), X=X2-X1; wherein X2 is the feature after processing postoperatively, X1 is the feature after processing preoperatively, X represents the difference value of the feature, and DX represents the difference feature.

2. The method of claim 1, wherein the method is characterized by: The multi-task prediction module carries out obstacle level prediction of different functions based on the gated feature. The data preprocessing is carried out on the brain image, and the brain image comprises preoperative NCCT and postoperative NCCT; the data preprocessing comprises the following steps: S1, a rigid registration algorithm is used to align the postoperative NCCT image with the preoperative NCCT image; S2, a skull stripping algorithm is used to obtain a brain tissue mask based on the preoperative NCCT; S3, a non-rigid registration algorithm and a template of five brain partitions are used to obtain the corresponding five partition masks based on the preoperative NCCT; 3. The method of claim 1, wherein the method is characterized by: S4, the preoperative NCCT without skull and the postoperative NCCT are subjected to window width and window level [0, 200] Hu normalization.

4. The method of claim 1, wherein the method is characterized by: The preoperative and postoperative difference feature extraction module comprises the following steps: basic feature extraction is carried out on the preoperative and postoperative NCCT images respectively, 3*3*3 convolution is realized through two step lengths [2, 2, 2] and one step length [2, 2, 1], and a relu activation layer and a batchnorm regular layer are used after each convolution; after the X1 and X2 features are obtained, DIFF layer is used to extract the feature difference of the two features as shared features.

5. The method of claim 1, wherein the method is characterized by: The multi-task prediction module carries out obstacle level prediction of different functions through one convolution block, one full connection block and one softmax layer. The model training and prediction comprise the following steps: in the training process of the model, the model is optimized by using Adam; the learning rate is 0.01, and is reduced to 0.1 times of the original value after every 50 rounds; five loss functions adopt cross-entropy loss, and the corresponding weights are the current category number / total category number; and the maximum probability corresponding to the obstacle level in the output result of each task is taken as the output result in the test stage.

Images