Method for judging severity of first diagnosis AIS patient based on head MRI-DWI image
Through a multimodal fusion processing method based on head MRI-DWI images, the infarction core area information of AIS patients was extracted and classified in combination with clinical information, which solved the problem of time-consuming and low consistency in the existing technology, and achieved the correctness of the evaluation Rapid and accurate judgment of the severity of AIS patients and improved treatment plans.
Patent Information
- Application Number
- CN202410945904.5
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-07-15
- Publication Date
- 2024-11-08
AI Technical Summary
The assessment of the severity of patients with acute ischemic stroke (AIS) in the prior art is problematic, especially for patients with coma, muscle relaxation or speech disorders, which leads to the efficiency of the treatment plan formulation. Low.
Using the first diagnosis method based on head MRI-DWI images, the infarction core area information is extracted through the first processing model, and the clinical information is input into the second processing model for multimodal fusion classification, so as to achieve rapid and accurate severity of AIS patients. judge. The method includes an infarction zone extraction module, a responsibility zone calculation module and a multimodal fusion module, and uses artificial intelligence technology to automatically hierarchize.
It achieves rapid and accurate judgment of the severity of AIS patients, is suitable for patients with confusion and aphasia, solves the problem of time-consuming and low consistency of NIHSS scores, and improves the efficiency of formulating treatment plans.
Smart Images

Figure CN118918066A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field of medical image processing, and in particular to a method for judging the severity of AIS patients diagnosed for the first time based on head MRI-DWI images. Background Art
[0002] Acute ischemic stroke (AIS) is a common cardiovascular and cerebrovascular disease, ranking second among all cardiovascular and cerebrovascular deaths. It is a major chronic non-communicable disease that affects human life and health, and is the third leading cause of death in the world. According to statistics on stroke patients, ischemic stroke accounts for about 69.6%-77.8% of stroke in my country, and the total number of patients each year shows a rising trend. In the past 20 years, the incidence of stroke in my country has increased by 8.7% per year. AIS has the characteristics of high morbidity, high mortality, and high disability rate, which has brought a heavy health and economic burden to society. Clinically, neurologists need to closely observe and test severe AIS patients, identify severe signs such as impaired consciousness, increased intracranial pressure, and organ failure as early as possible, so as to intervene in time and avoid a sharp deterioration of the patient's condition. Therefore, the rapid and accurate judgment of the severity of AIS patients is of great significance for the formulation of treatment plans and the improvement of treatment efficiency.
[0003] At present, the most classic method and basis for grading the severity of AIS patients is the National Institute of Health Stroke Scale (NIHSS) score. The NIHSS score contains 11 items, each of which represents a different type of functional impairment after stroke. These items reflect the patient's performance in terms of movement, sensation, language, cognition, and visual field. Each item is assigned 0 to 5 points (0 points means no abnormality) according to the patient's performance. The NIHSS score ranges from 0 points (no symptoms) to 42 points (complete disability). Although the NIHSS score is a widely used tool, it may become difficult to evaluate patients who are unable to cooperate due to coma, muscle relaxation, or language barriers. In addition, the consistency of the evaluation results may be affected due to differences in subjective judgment and interpretation by medical staff. Summary of the invention
[0004] In view of the above problems existing in the prior art, a method for judging the severity of AIS patients diagnosed for the first time based on head MRI-DWI images is now provided.
[0005] The specific technical solutions are as follows:
[0006] A method for judging the severity of AIS patients diagnosed for the first time based on head MRI-DWI images, including:
[0007] Step S1: using the first processing model to extract the infarct core area information in the head MRI-DWI image of the patient to be identified, and collecting the clinical information of the patient to be identified;
[0008] The infarct core area information includes volume information of the infarct core area associated with the NIHSS score and location information of the infarct core area;
[0009] Step S2: using a second processing model to fuse the infarct core area information and the clinical information, and classifying them to obtain the severity corresponding to the patient to be identified.
[0010] On the other hand, in step S1, the first processing model includes:
[0011] An infarct region extraction module, wherein the infarct region extraction module identifies the head MRI-DWI image to obtain a pre-identified infarct core region;
[0012] A responsibility area calculation module, which performs voxel-by-voxel calculation according to the head MRI-DWI image to determine the correlation between each voxel and the NIHSS score and draw a responsibility area map;
[0013] A cross-linking module, wherein the cross-linking module is respectively connected to the infarct area extraction module and the responsibility area calculation module, and the cross-linking module superimposes the pre-identified infarct core area and the responsibility area map to obtain the infarct core area associated with the NIHSS score, and extracts the infarct core area information.
[0014] On the other hand, the infarct area extraction module includes:
[0015] A coding and decoding module, wherein the coding and decoding module processes the input head MRI-DWI image through a plurality of encoders and a plurality of decoders arranged in sequence to obtain segmented regions;
[0016] A template registration module is connected to the encoding and decoding module, and the template registration module uses a standard brain template to register the segmented area to obtain the pre-identified infarct core area.
[0017] On the other hand, the responsibility area calculation module includes:
[0018] A voxel-by-voxel processing module, which processes each voxel of the input head MRI-DWI image separately to determine a correlation score between the voxel and the infarction symptom;
[0019] A multivariate correlation processing module, the multivariate correlation processing module is connected to the voxel-by-voxel processing module, and the multivariate correlation module obtains a plurality of associated voxels associated with the infarction symptom according to the correlation score processing;
[0020] A responsibility area generation module, wherein the responsibility area generation module is connected to the multivariate correlation processing module, and the responsibility area processing module generates the responsibility area map according to the associated voxels.
[0021] On the other hand, in step S2, the second processing model includes:
[0022] An image input module, wherein the image input module extracts an image of a region to be fused from the head MRI-DWI image according to the infarct core area information;
[0023] A multi-level feature fusion module, which extracts image features on multiple receptive fields from the image in the area to be fused in turn through multiple convolutional layers, and performs feature fusion to obtain multi-level fused image features;
[0024] An embedding module, wherein the embedding module constructs an embedding vector for the input clinical information;
[0025] A multimodal fusion module, wherein the multimodal fusion module is connected to the multi-level feature fusion module and the embedding module respectively, and the multimodal fusion module fuses the embedding vector and the multi-level fusion image feature to obtain a multimodal feature;
[0026] A classification module, wherein the classification module is connected to the multimodal fusion module, and the classification module performs classification according to the multimodal features to obtain the severity.
[0027] On the other hand, the multi-level feature fusion module includes:
[0028] A plurality of sequentially connected downsampling layers, wherein the plurality of downsampling layers sequentially perform downsampling processing on the input image of the area to be fused;
[0029] A feature fusion module, wherein the feature fusion module extracts the image features from the output image of each level of the downsampling layer and fuses them to obtain the multi-level fused image features;
[0030] A plurality of upsampling layers are connected in sequence, and the plurality of upsampling layers are connected in sequence and perform upsampling processing on the upsampled image;
[0031] The upsampling layer of the first level is connected to the downsampling layer of the last level and the output image of the downsampling layer is obtained as the upsampling image;
[0032] The upsampling layer at the last level outputs the upsampled image as a model segmentation image;
[0033] The classification module also receives the model segmented image and classifies it together with the multimodal features to obtain the severity.
[0034] In another aspect, the embedded module comprises:
[0035] A one-hot encoding module, wherein the one-hot encoding module encodes the input clinical information to obtain an encoding vector;
[0036] A normalization module, wherein the normalization module is connected to the one-hot encoding module, and the normalization module performs normalization processing on the encoding vector to obtain the embedding vector.
[0037] On the other hand, the multimodal fusion module includes:
[0038] A feature splicing module, wherein the feature splicing module splices the embedding vector and the multi-level fusion image feature to obtain a splicing feature;
[0039] a first fusion module, the first fusion module is connected to the feature splicing module, and the first fusion module fuses the splicing feature and the multi-level fusion image feature and performs normalization processing to obtain a first fusion feature;
[0040] A second fusion module, the second fusion module is connected to the feature splicing module, and the second fusion module fuses the splicing feature and the embedding vector and performs normalization processing to obtain a second fused feature;
[0041] An output module, wherein the output module is connected to the first fusion module and the second fusion module respectively, and the output module fuses the first fusion feature and the second fusion feature to obtain the multimodal feature for output.
[0042] The above technical solution has the following advantages or beneficial effects:
[0043] In order to solve the problem of poor effect of NIHSS scoring for patients in the prior art, in this scheme, the volume information and location information of the infarct core area of AIS patients related to the NIHSS score are first extracted through the first processing model, and then the clinical information is input into the multimodal second processing model for classification, thereby achieving rapid and accurate judgment of the severity of AIS patients who are first diagnosed, and solving the problem of time-consuming and low consistency of manual scoring. BRIEF DESCRIPTION OF THE DRAWINGS
[0044] The embodiments of the present invention will be described more fully with reference to the attached drawings, which are provided for illustration and description only and are not intended to limit the scope of the present invention.
[0045] Figure 1 It is an overall schematic diagram of an embodiment of the present invention;
[0046] Figure 2 This is a schematic diagram of a first processing model in an embodiment of the present invention;
[0047] Figure 3 Schematic diagram of an infarct area extraction module in an embodiment of the present invention;
[0048] Figure 4 This is a schematic diagram of a responsibility area calculation module in an embodiment of the present invention;
[0049] Figure 5 This is a schematic diagram of a second processing model in an embodiment of the present invention;
[0050] Figure 6 Schematic diagram of a multi-level feature fusion module in an embodiment of the present invention;
[0051] Figure 7 Schematic diagram of a multimodal fusion module in an embodiment of the present invention; DETAILED DESCRIPTION
[0052] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.
[0053] It should be noted that, in the absence of conflict, the embodiments of the present invention and the features in the embodiments may be combined with each other.
[0054] Some of the blocks shown in the accompanying drawings are functional entities that do not necessarily correspond to physically or logically independent entities. These functional entities can be implemented in software form, or in one or more hardware modules or integrated circuits, or in different networks and / or processor devices and / or microcontroller devices.
[0055] The present invention will be further described below in conjunction with the accompanying drawings and specific embodiments, but they are not intended to limit the present invention.
[0056] The present invention comprises:
[0057] A method for judging the severity of AIS patients for the first time based on head MRI-DWI images, such as Figure 1As shown, including:
[0058] Step S1: using the first processing model to extract the infarct core area information from the head MRI-DWI image of the patient to be identified, and collecting the clinical information of the patient to be identified;
[0059] The infarct core area information includes the volume information of the infarct core area associated with the NIHSS score and the location information of the infarct core area;
[0060] Step S2: The second processing model is used to fuse the infarct core area information and the clinical information, and classify them to obtain the severity corresponding to the patient to be identified.
[0061] Specifically, in view of the problems existing in the prior art of NIHSS scoring of patients, in this embodiment, the first processing model is introduced to collect the infarct core area information in the head MRI-DWI image of the patient to be identified, thereby realizing the effective extraction of the volume information and position information of the infarct core area associated with the NIHSS score. Subsequently, the multimodal second processing model is used to fuse the infarct core area information and clinical information, and classify the severity of the patient to be identified, realizing the joint discrimination process based on clinical information and MRI-DWI image information, realizing the automatic classification of the severity of AIS patients, thereby realizing the rapid and accurate judgment of the severity of the newly diagnosed AIS patients, solving the problem of time-consuming and labor-intensive NIHSS scoring and poor consistency, and being applicable to patients with unconsciousness and aphasia.
[0062] During implementation, the above method is mainly configured as a software embodiment in a computer device, such as a doctor's workstation, an in-hospital electronic information system, etc. The first processing model and the second processing model are processing models implemented based on artificial intelligence technology, and the first processing model and the second processing model are trained in advance with relevant data sets so that the first processing model and the second processing model can realize corresponding functions.
[0063] The computer device is also connected to an external magnetic resonance imaging system to obtain head MRI-DWI images by performing head magnetic resonance imaging on the patient, and retrieve the patient's clinical information, including general information, medical history information, and laboratory tests, through the electronic medical record system.
[0064] Among them, general information includes: patient's sex (Sex), age (Age); medical history information includes: history of smoking (History of smoking), history of drinking (History of alcohol), history of diabetes (History of diabetes), history of myocardial infarction (History of myocardial infarction), history of coronary atherosclerosis (History of coronary atherosclerosis), history of atrial fibrillation (History of atrial fibrillation), history of hypertension (History of hypertension) and history of stroke (History of stroke); laboratory tests include: prothrombin time (Prothrombin time, PT), fibrinogen (Fibrinogen), D-dimer (D-dimer), serum troponin I (Serum troponin I), blood sugar (Blood sugar), blood lipids (Blood lipids), plasma brain natriuretic peptide (Brainnatriuretic peptide, BNP).
[0065] Severity is grouped based on NIHSS scores, e.g., mild (0-7) / moderate-severe (8-42).
[0066] In one embodiment, Figure 2 As shown, in step S1, the first processing model includes:
[0067] Infarct region extraction module A1, the infarct region extraction module A1 identifies the head MRI-DWI image to obtain the pre-identified infarct core area;
[0068] Responsibility region calculation module A2, which performs voxel-by-voxel calculations based on the head MRI-DWI images to determine the correlation between each voxel and the NIHSS score and draw a responsibility region map;
[0069] Cross-linking module A3, cross-linking module A3 connects the infarct area extraction module and the responsibility area calculation module respectively, and cross-linking module A3 superimposes the pre-identified infarct core area and the responsibility area map to obtain the infarct core area associated with the NIHSS score, and extracts the infarct core area information.
[0070] Specifically, in order to achieve a better recognition effect, in this embodiment, the vector regression-multivariate lesion-symptom mapping (Support Vector Regression-Lesion-Symptom Mapping, SVR-LSM) technology is introduced in the first processing model for processing. Among them, for the head MRI-DWI image, the infarction area extraction module A1 and the responsibility area calculation module A2 are used for determination. Among them, the extraction of the pre-identified infarction core area is completed in the infarction area extraction module A1, and the volume of the infarction core area is calculated according to the number of voxels. In the responsibility area calculation module A2, the head MRI-DWI image is calculated voxel by voxel to determine the correlation between each voxel and the NIHSS score, and the responsibility area map is drawn according to the voxels that are significantly correlated with the NIHSS score. Finally, the cross-linking module A3 superimposes the pre-identified infarction core area and the responsibility area map to obtain the infarction core area actually associated with the NIHSS score, and extracts the infarction core area information, including volume information and position information, which is obtained by the variable Core eloquent ,β-Value Eloquent Indicates. eloquent represents the volume of the infarct core area in the overlapping area, β-Value Eloquent Represents the correlation weight between the voxels in the overlapping area and the NIHSS score.
[0071] In one embodiment, Figure 3 As shown, the infarct area extraction module A1 includes:
[0072] The encoding and decoding module A11 processes the input head MRI-DWI image by using a plurality of encoders and a plurality of decoders arranged in sequence to obtain segmented regions;
[0073] The template registration module A12 is connected to the encoding and decoding module A11. The template registration module A12 registers the segmented area with a standard brain template to obtain a pre-identified infarct core area.
[0074] Specifically, in order to effectively measure the volume of the overall infarct core area, in this embodiment, a codec module A11 is configured in the infarct area extraction module A1, and the codec module A11 processes the input head MRI-DWI image through a plurality of encoders and a plurality of decoders arranged in sequence, including a series of downsampling processes and subsequent upsampling processes, thereby segmenting the head MRI-DWI image and obtaining the segmented area. Subsequently, in order to make the segmented area obtained by the model segmentation conform to the actual head image, the standard brain template is also used for registration to obtain the pre-identified infarct core area.
[0075] The registration process includes: first, turn the sample and template to make their orientation consistent, then perform 4 rounds of registration, the first two rounds are rigid transformations, and the last two rounds are affine transformations. After each round of registration, the Dice Coefficient (DC) is calculated, and the registration process ends early when DC>0.94.
[0076] The standard brain template can use the open source public MRI T1WI average brain image as the standard brain template (http: / / www.bmap.ucla.edu / portfolio / atlases / ICBM_Template / ), and align the MRI-DWI images and core area MASK in all data sets to the standard brain template.
[0077] In the implementation process, the above model was trained in advance using ITK-SNAP open-sourcesoftware (Version 3.8.0, http: / / www.itksnap.org) to manually delineate the infarct core area of the MRI-DWI image. The MRI-DWI image was the first MRI-DWI image of AIS patients within 24 hours of onset, and the scanning parameters were: TR = 4600ms, TE = 89ms, slice thickness 5mm, scanning without spacing, field of view 24cm×24cm, matrix 256×256, echo gap 0.75ms, b value 1000s / mm 2 . It was completed by three junior radiologists, and two senior radiologists reviewed the annotation results. For cases with questionable lesion annotation, the two senior radiologists jointly discussed and decided. In some embodiments, the infarct core area can also be delineated by a pre-trained automatic segmentation model. The DCM image sequences that match the manual delineation results MASK are merged into NII files, and then these NII images are de-skulled using FSL software (Version: 6.0.6.5) to serve as the actual training data set.
[0078] In one embodiment, Figure 4 As shown, the responsibility area calculation module A2 includes:
[0079] A voxel-by-voxel processing module A21, which processes each voxel of the input head MRI-DWI image separately to determine a correlation score between the voxel and the infarction symptom;
[0080] A multivariate correlation processing module A22, the multivariate correlation processing module A22 is connected to the voxel-by-voxel processing module A21, and the multivariate correlation module A22 obtains a plurality of associated voxels associated with the infarction symptom according to the correlation score processing;
[0081] Responsibility area generation module A23, the responsibility area generation module A23 is connected to the multivariate correlation processing module A22, and the responsibility area processing module A23 generates a responsibility area map according to the associated voxels.
[0082] Specifically, in order to achieve effective extraction of voxels that are significantly associated with the severity of AIS, in this embodiment, each voxel of the input head MRI-DWI image is first processed separately by the voxel-by-voxel processing module A21 to determine the correlation score between the voxel and the infarction symptom. Then, after performing regression analysis on most cases of acute ischemic stroke, the inventor believes that when high-correlation voxels appear, there are usually at least 10 patients with diseased voxels involved. Therefore, the multivariate correlation module A22 obtains multiple associated voxels associated with infarction symptoms according to the correlation score processing, including obtaining significant values using 1000 permutations of related measurements, and the significance threshold of P<0.005 is defined as the responsibility area. Finally, the responsibility area processing module A23 generates a responsibility area map according to the screened associated voxels.
[0083] In one embodiment, Figure 5 As shown, in step S2, the second processing model includes:
[0084] An image input module B1, which extracts the image of the area to be fused from the head MRI-DWI image according to the infarct core area information;
[0085] Multi-level feature fusion module B2: The multi-level feature fusion module B2 extracts image features on multiple receptive fields from the fusion area image in turn through multiple convolutional layers, and performs feature fusion to obtain multi-level fused image features;
[0086] Embedding module B3, embedding module B3 constructs an embedding vector for the input clinical information;
[0087] A multimodal fusion module B4, the multimodal fusion module B4 is connected to the multi-level feature fusion module B2 and the embedding module B3 respectively, and the multimodal fusion module B4 fuses the embedding vector and the multi-level fusion image feature to obtain a multimodal feature;
[0088] Classification module B5: Classification module B5 is connected to multimodal fusion module B4, and classification module B4 classifies according to multimodal features to obtain severity.
[0089] Specifically, in order to achieve better correlation prediction between image and table data, in this embodiment, for the feature information on the image side, the head MRI-DWI image is first intercepted according to the infarct core area information through the image input module B1, so as to obtain the image of the area to be fused corresponding to the image to be fused. Subsequently, for the image of the area to be fused, the multi-level feature fusion module B2 extracts image features on multiple receptive fields of the image of the area to be fused in turn through multiple convolutional layers, and performs feature fusion to obtain multi-level fused image features, thereby achieving a better extraction effect of image features.
[0090] For the relevant data on the clinical information side, the embedding module B3 converts it into an embedding vector, and then inputs it into the multimodal fusion module B4, where the embedding vector and the multi-level fusion image features are fused to obtain the multimodal features.
[0091] Finally, the classification module B5 performs classification according to the multimodal features to finally obtain the severity.
[0092] In one embodiment, Figure 5 As shown, the multi-level feature fusion module B2 includes:
[0093] A plurality of sequentially connected downsampling layers B21, wherein the plurality of downsampling layers B21 sequentially downsample the input image of the region to be fused;
[0094] A feature fusion module B22, which extracts image features from the output images of each downsampling layer and fuses them to obtain multi-level fused image features;
[0095] A plurality of upsampling layers B23 are connected in sequence, and a plurality of upsampling layers B23 are connected in sequence and perform upsampling processing on the upsampled image;
[0096] The first-level upsampling layer B23 is connected to the last-level downsampling layer B21 and obtains the output image of the downsampling layer B21 as the upsampling image;
[0097] The last level upsampling layer B23 outputs the upsampled image as the model segmentation image;
[0098] The classification module B5 also receives the model segmentation image and classifies it together with the multimodal features to obtain the severity.
[0099] Specifically, in order to achieve effective extraction of image features on different receptive fields, in this embodiment, a downsampling layer B21 and an upsampling layer B23 connected in sequence are set to form a codec structure. Through the convolution operation of the downsampling layer B21 at different levels, effective extraction of image features on different receptive fields is achieved, and then the features are sent to the feature fusion module B22 for feature fusion processing.
[0100] Among them, the feature fusion module uses a series of flattening, linear activation, and batch normalization operations to unify the image features on different receptive fields to the same scale, and then concatenates and normalizes the features to form multi-level fused image features.
[0101] In addition, in order to achieve a more accurate diagnosis process, the present embodiment continues to process the image features through the upsampling layer B23, and finally obtains and outputs the model segmentation image. The classification module B5 also receives the model segmentation image and classifies it together with the multimodal features to obtain the severity, thereby retaining the original image information.
[0102] In one embodiment, Figure 6 As shown, the embedded module B3 includes:
[0103] One-hot encoding module B31, the one-hot encoding module B31 encodes the input clinical information to obtain an encoding vector;
[0104] Normalization module B32, the normalization module B32 is connected to the one-hot encoding module B31, and the normalization module B32 normalizes the encoding vector to obtain an embedded vector.
[0105] Specifically, in order to achieve better processing effects on different types of tabular data, in this embodiment, for each field of the input clinical information, the One-hot encoding module B31 is used to perform One-hot encoding on the field data to obtain a coding vector, and then the normalization module B32 normalizes the coding vector to obtain an embedded vector within a specific range.
[0106] In one embodiment, Figure 7 As shown, the multimodal fusion module B4 includes:
[0107] A feature splicing module B41, which splices the embedded vector and the multi-level fusion image feature to obtain a spliced feature;
[0108] A first fusion module B42, the first fusion module B42 is connected to the feature stitching module B41, and the first fusion module B42 fuses the stitching feature and the multi-level fusion image feature and performs normalization processing to obtain a first fusion feature;
[0109] A second fusion module B43, the second fusion module B43 is connected to the feature splicing module B41, and the second fusion module B43 fuses the splicing feature and the embedding vector and normalizes them to obtain a second fusion feature;
[0110] The output module B44 is connected to the first fusion module B42 and the second fusion module B43 respectively. The output module B44 fuses the first fusion feature and the second fusion feature to obtain a multimodal feature for output.
[0111] Specifically, in order to achieve a better feature fusion effect, in this embodiment, the embedded vector and the multi-level fusion image feature are firstly spliced by the feature splicing module B41 to obtain the splicing feature, and then, for the embedded vector and the multi-level fusion image feature, the splicing feature and the multi-level fusion image feature are respectively fused and normalized by the first fusion module B42 to obtain the first fusion feature, and the splicing feature and the embedded vector are fused and normalized by the second fusion module B43 to obtain the second fusion feature. Finally, the first fusion feature and the second fusion feature are fused by the output module B44 to obtain the multimodal feature for output, so as to achieve better retention of the original information and facilitate subsequent classification processing.
[0112] Those skilled in the art will appreciate that various aspects of the present invention, or possible implementations of various aspects, can be specifically implemented as systems, methods or computer program products. Therefore, various aspects of the present invention, or possible implementations of various aspects, can adopt the form of complete hardware embodiments, complete software embodiments (including firmware, resident software, etc.), or combined software and hardware embodiments, all collectively referred to as "circuit", "module" or "system" here. In addition, various aspects of the present invention, or possible implementations of various aspects, can adopt the form of computer program products, which refer to computer-readable program codes stored in computer-readable media.
[0113] The computer readable medium may be a computer readable signal medium or a computer readable storage medium. The computer readable storage medium includes but is not limited to electronic, magnetic, optical, electromagnetic, infrared or semiconductor systems, devices or apparatuses, or any suitable combination of the foregoing, such as random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable read-only memory (CD-ROM).
[0114] The processor in the computer reads the computer-readable program code stored in the computer-readable medium, so that the processor can execute the functional actions specified in each step or the combination of steps in the flowchart; and generate a device for implementing the functional actions specified in each block or the combination of blocks in the block diagram.
[0115] It should be understood that the processor in the computer can be understood as one or more application-specific integrated circuits (ASICs), DSPs, programmable logic devices (PLDs), complex programmable logic devices (CPLDs), field programmable gate arrays (FPGAs), general-purpose processors, controllers, microcontrollers (MCUs), microprocessors, or other electronic components implemented to execute the aforementioned computer-readable program codes.
[0116] Computer readable program code can be executed completely on the local computer of the user, partially on the local computer of the user, as a separate software package, partially on the local computer of the user and partially on the remote computer, or executed completely on the remote computer or server.It should also be noted that in some alternative embodiments, the functions noted in each step in the flow chart or each block in the block diagram may not occur in the order noted in the figure.For example, depending on the function involved, two steps shown in succession or two blocks may actually be executed roughly simultaneously, or these blocks may sometimes be executed in reverse order.
[0117] The above are only preferred embodiments of the present invention, and are not intended to limit the implementation methods and protection scope of the present invention. Those skilled in the art should be aware that all solutions obtained by equivalent substitutions and obvious changes made using the description and illustrations of the present invention should be included in the protection scope of the present invention.
Claims
1. A method for judging the severity of AIS patients for the first time based on head MRI-DWI images, characterized in that: include: Step S1: using the first processing model to extract the infarct core area information in the head MRI-DWI image of the patient to be identified, and collecting the clinical information of the patient to be identified; The infarct core area information includes volume information of the infarct core area associated with the NIHSS score and location information of the infarct core area; Step S2: using a second processing model to fuse the infarct core area information and the clinical information, and classifying them to obtain the severity corresponding to the patient to be identified.
2. The method for determining the severity of AIS patients for the first time according to claim 1, characterized in that: In step S1, the first processing model includes: An infarct region extraction module, wherein the infarct region extraction module identifies the head MRI-DWI image to obtain a pre-identified infarct core region; A responsibility area calculation module, which performs voxel-by-voxel calculation according to the head MRI-DWI image to determine the correlation between each voxel and the NIHSS score and draw a responsibility area map; A cross-linking module, wherein the cross-linking module is respectively connected to the infarct area extraction module and the responsibility area calculation module, and the cross-linking module superimposes the pre-identified infarct core area and the responsibility area map to obtain the infarct core area associated with the NIHSS score, and extracts the infarct core area information.
3. The method for determining the severity of AIS patients for the first time according to claim 2, characterized in that: The infarct area extraction module comprises: A coding and decoding module, wherein the coding and decoding module processes the input head MRI-DWI image through a plurality of encoders and a plurality of decoders arranged in sequence to obtain segmented regions; A template registration module is connected to the encoding and decoding module, and the template registration module uses a standard brain template to register the segmented area to obtain the pre-identified infarct core area.
4. The method for determining the severity of AIS patients for the first time according to claim 2, characterized in that: The responsibility area calculation module includes: A voxel-by-voxel processing module, which processes each voxel of the input head MRI-DWI image separately to determine a correlation score between the voxel and the infarction symptom; A multivariate correlation processing module, the multivariate correlation processing module is connected to the voxel-by-voxel processing module, and the multivariate correlation module obtains a plurality of associated voxels associated with the infarction symptom according to the correlation score processing; A responsibility area generation module, wherein the responsibility area generation module is connected to the multivariate correlation processing module, and the responsibility area processing module generates the responsibility area map according to the associated voxels.
5. The method for determining the severity of AIS patients for the first time according to claim 1, characterized in that: In step S2, the second processing model includes: An image input module, wherein the image input module extracts an image of a region to be fused from the head MRI-DWI image according to the infarct core area information; A multi-level feature fusion module, which extracts image features on multiple receptive fields from the image in the area to be fused in turn through multiple convolutional layers, and performs feature fusion to obtain multi-level fused image features; An embedding module, wherein the embedding module constructs an embedding vector for the input clinical information; A multimodal fusion module, wherein the multimodal fusion module is connected to the multi-level feature fusion module and the embedding module respectively, and the multimodal fusion module fuses the embedding vector and the multi-level fusion image feature to obtain a multimodal feature; A classification module, wherein the classification module is connected to the multimodal fusion module, and the classification module performs classification according to the multimodal features to obtain the severity.
6. The method for determining the severity of AIS patients for the first time according to claim 5, characterized in that: The multi-level feature fusion module includes: A plurality of sequentially connected downsampling layers, wherein the plurality of downsampling layers sequentially perform downsampling processing on the input image of the area to be fused; A feature fusion module, wherein the feature fusion module extracts the image features from the output image of each level of the downsampling layer and fuses them to obtain the multi-level fused image features; A plurality of upsampling layers are connected in sequence, and the plurality of upsampling layers are connected in sequence and perform upsampling processing on the upsampled image; The upsampling layer of the first level is connected to the downsampling layer of the last level and the output image of the downsampling layer is obtained as the upsampling image; The upsampling layer at the last level outputs the upsampled image as a model segmentation image; The classification module also receives the model segmented image and classifies it together with the multimodal features to obtain the severity.
7. The method for determining the severity of AIS patients for the first time according to claim 5, characterized in that: The embedded module comprises: A one-hot encoding module, wherein the one-hot encoding module encodes the input clinical information to obtain an encoding vector; A normalization module, wherein the normalization module is connected to the one-hot encoding module, and the normalization module performs normalization processing on the encoding vector to obtain the embedding vector.
8. The method for determining the severity of AIS patients for the first time according to claim 5, characterized in that: The multimodal fusion module comprises: A feature splicing module, wherein the feature splicing module splices the embedding vector and the multi-level fusion image feature to obtain a splicing feature; A first fusion module, the first fusion module is connected to the feature splicing module, The first fusion module fuses the splicing feature and the multi-level fusion image feature and normalizes them to obtain a first fusion feature; A second fusion module, the second fusion module is connected to the feature splicing module, and the second fusion module fuses the splicing feature and the embedding vector and performs normalization processing to obtain a second fused feature; An output module, wherein the output module is connected to the first fusion module and the second fusion module respectively, and the output module fuses the first fusion feature and the second fusion feature to obtain the multimodal feature for output.
