Post-infarction brain injury risk prediction method and system based on multi-modal data fusion

Abstract

This application provides a method and system for predicting brain injury risk after myocardial infarction based on multimodal data fusion, belonging to the field of medical information processing technology. The method includes: firstly, acquiring cardiac images, brain images, and microwave echo signals of brain tissue from a patient; then generating waveform data based on the phase and amplitude differences of the echo signals; next, using cardiac feature data, brain feature data, and waveform data as nodes, establishing connections between nodes according to a pre-set clinical knowledge base to form a heterogeneous graph; further, processing the graph using a graph neural network to obtain associated features; finally, inputting the associated features into a multi-task learning model to output the probability of future brain injury risk and the degree of impairment of self-regulation function. This application improves the accuracy and comprehensiveness of predicting brain injury risk after myocardial infarction.

Description

Technical Field

[0001] This application relates to the field of medical information processing technology, and in particular to a method and system for predicting brain injury risk after myocardial infarction based on multimodal data fusion. Background Technology

[0002] Risk prediction of brain injury after myocardial infarction is of great significance for the long-term rehabilitation management of patients. Timely and accurate identification of high-risk groups helps to implement early intervention, thereby improving prognosis. Such prediction methods have shown broad application prospects in clinical monitoring and personalized treatment plan development.

[0003] In the existing technology, there are already methods for predicting the risk of heart-brain association using medical imaging data of a single or few modalities. For example, some schemes assess cardiac function by analyzing cardiac magnetic resonance imaging and infer the risk of possible brain complications based on this; other schemes focus on using brain magnetic resonance imaging features or routine physiological signals to independently assess the risk of brain injury.

[0004] However, these existing methods analyze cardiac and brain state information as isolated factors, lacking a structured integration method that reflects their intrinsic physiological relationship when constructing predictive models. This limits the depth of understanding of the mechanisms of cardiac-brain synergistic injury, affecting the comprehensiveness and accuracy of risk prediction. Therefore, existing technologies suffer from insufficient comprehensiveness and accuracy in predicting brain injury risk after myocardial infarction. Summary of the Invention

[0005] This application provides a method and system for predicting the risk of brain injury after myocardial infarction based on multimodal data fusion, in order to solve the problems of low accuracy and poor comprehensiveness in the prediction of the risk of brain injury after myocardial infarction in the prior art.

[0006] To address the aforementioned technical problems, in a first aspect, this application provides a method for predicting brain injury risk after myocardial infarction based on multimodal data fusion, comprising:

[0007] Acquire multimodal data of patients with myocardial infarction, including cardiac magnetic resonance imaging, brain magnetic resonance imaging, and microwave echo signals of brain tissue;

[0008] Based on the phase difference and amplitude difference between multiple channels in the microwave echo signal, waveform data is generated;

[0009] The first feature data in the cardiac magnetic resonance image, the second feature data in the brain magnetic resonance image, and the waveform data are used as nodes, and connection edges are established between the corresponding nodes according to a preset clinical medical knowledge base to form a heterogeneous graph.

[0010] The heterogeneous graph is processed using a graph neural network to obtain associated features;

[0011] The associated features are input into a multi-task learning model for prediction, and the results of the probability of future brain injury after myocardial infarction and the degree of impairment of self-regulation function are output.

[0012] Optionally, the step of processing the heterogeneous graph using a graph neural network to obtain associated features includes:

[0013] Obtain the initial feature vector of each node in the heterogeneous graph;

[0014] The structural information of the heterogeneous graph and the initial feature vectors of all nodes are input into the graph neural network. Multi-layer message passing is performed through the graph neural network. In each layer, for each current node in the heterogeneous graph, aggregation and update operations are performed to obtain the updated feature value vector.

[0015] After at least two layers of message passing in the graph neural network, the updated feature vectors of all nodes are extracted from the last layer of the graph neural network, and associated features are generated based on the updated feature vectors of all nodes.

[0016] Optionally, the step of performing multi-layer message passing through the graph neural network, in each layer, for each current node in the heterogeneous graph, performs aggregation and update operations to obtain an updated feature value vector, including:

[0017] Based on the structural information of the heterogeneous graph and combined with the preset coupling mapping relationship between cardiovascular and cerebrovascular systems, all adjacent nodes that have a direct connection relationship with the current node are determined.

[0018] The initial feature vectors of all adjacent nodes are weighted and summed using the first aggregation function preset in the graph neural network to generate an intermediate feature vector.

[0019] The intermediate feature vector is merged with the feature value vector of the current node itself to form a merged vector;

[0020] The merged vector is processed using the preset update rules in the graph neural network to output the updated feature value vector.

[0021] Optionally, the step of inputting the associated features into a multi-task learning model for prediction, and outputting the probability of future brain injury after myocardial infarction and the degree of impairment of self-regulation function, includes:

[0022] The associated features are input into a multi-task learning model, and the associated features are processed through the shared parameter layer of the multi-task learning model to obtain a shared feature representation.

[0023] The shared feature representation is processed through the first task branch of the multi-task learning model to obtain the probability of future brain injury, and the shared feature representation is processed through the second task branch of the multi-task learning model to obtain the degree of impairment of self-regulation function.

[0024] Optionally, the step of processing the shared feature representation through the first task branch of the multi-task learning model to obtain the probability of future brain injury includes:

[0025] The shared feature representation is input into the first neural network layer of the first task branch. The first neural network layer performs linear transformation and nonlinear activation operations on the shared feature representation to obtain the first hidden layer features.

[0026] The first hidden layer features are input into the second neural network layer of the first task branch. The second neural network layer performs dimensionality reduction processing on the first hidden layer features and outputs a scalar value.

[0027] Based on the scalar value, the probability of future brain injury is obtained using a preset probability transformation function.

[0028] Optionally, generating waveform data based on the phase difference and amplitude difference between multiple channels in the microwave echo signal includes:

[0029] The microwave echo signal is coherently accumulated to obtain accumulated signals from multiple channels;

[0030] Calculate the amplitude difference and phase difference between the accumulated signals from multiple channels;

[0031] The amplitude difference and phase difference are spatiotemporally integrated to generate a comprehensive difference sequence;

[0032] The comprehensive difference sequence is discretized and sampled to form waveform data.

[0033] Optionally, the step of using the first feature data from the cardiac magnetic resonance image, the second feature data from the brain magnetic resonance image, and the waveform data as nodes, and establishing connection edges between corresponding nodes according to a preset clinical medical knowledge base to form a heterogeneous graph, includes:

[0034] Based on the cardiac magnetic resonance imaging, brain magnetic resonance imaging, and waveform data, a first node corresponding to cardiac image features, a second node corresponding to brain image features, and a third node corresponding to waveform features are extracted and constructed respectively.

[0035] Based on the connection rules defined in the clinical medical knowledge base, undirected edges, directed edges, or bidirectional edges representing the correlation of features are established within various types of nodes and between nodes of different categories.

[0036] Integrate all types of nodes and all types of connection edges to obtain heterogeneous edges.

[0037] Secondly, this application provides a system for predicting the risk of brain injury after myocardial infarction based on multimodal data fusion, comprising:

[0038] The acquisition module is used to acquire multimodal data of patients with myocardial infarction, including cardiac magnetic resonance imaging, brain magnetic resonance imaging, and microwave echo signals of brain tissue.

[0039] The generation module is used to generate waveform data based on the phase difference and amplitude difference between multiple channels in the microwave echo signal;

[0040] A module is established to use the first feature data in the cardiac magnetic resonance image, the second feature data in the brain magnetic resonance image, and the waveform data as nodes, and to establish connection edges between the corresponding nodes according to a preset clinical medical knowledge base to form a heterogeneous graph.

[0041] The processing module is used to process the heterogeneous graph using a graph neural network to obtain associated features;

[0042] The input module is used to input the associated features into a multi-task learning model for prediction, and output the probability of future brain injury after myocardial infarction and the degree of impairment of self-regulation function.

[0043] Thirdly, this application provides an electronic device, comprising:

[0044] Memory, used to store computer programs;

[0045] A processor, configured to execute the computer program to implement the steps of the method for predicting brain injury risk after myocardial infarction based on multimodal data fusion as described in the first aspect above.

[0046] Fourthly, this application provides a computer-readable storage medium storing a computer program that, when executed by a processor, can implement the steps of the method for predicting brain injury risk after myocardial infarction based on multimodal data fusion as described in the first aspect above.

[0047] The method for predicting brain injury risk after myocardial infarction based on multimodal data fusion provided in this application has the following beneficial effects:

[0048] First, by acquiring diverse data covering cardiac and brain structures and blood flow dynamics, a comprehensive information foundation is provided for subsequent analysis. Then, waveform data generated based on microwave echo signals can more directly reflect the real-time hemodynamic state of brain tissue. Next, different types of data are used as nodes, and connecting edges are constructed based on medical knowledge to form a heterogeneous graph, allowing the physiological relationship between the heart and brain to be presented in a structured manner. Then, a graph neural network is used to process this heterogeneous graph, effectively extracting deep features characterizing the propagation path of cardiocerebrovascular injury from complex connections. Finally, joint prediction is performed using a multi-task learning model, enabling the model to assess the risk of brain injury while also evaluating the state of cerebral blood flow regulation function, thereby improving the comprehensiveness and clinical applicability of risk assessment.

[0049] Furthermore, through the iterative processing of the graph neural network described above, this method enables node features to propagate and fuse along connection paths constructed based on medical knowledge, thereby more deeply exploring the complex correlation between cardiac injury features and brain abnormalities, and enhancing the expressive power and interpretability of the extracted features.

[0050] These or other aspects of this application will become more apparent in the following description of the embodiments. Attached Figure Description

[0051] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0052] Figure 1 A flowchart illustrating a method for predicting brain injury risk after myocardial infarction based on multimodal data fusion, provided in this application embodiment;

[0053] Figure 2 A schematic diagram illustrating a specific implementation of a method for predicting brain injury risk after myocardial infarction based on multimodal data fusion, provided in an embodiment of this application;

[0054] Figure 3 This is a schematic diagram of the structure of a brain injury risk prediction system based on multimodal data fusion provided in an embodiment of this application. Detailed Implementation

[0055] In existing technical solutions, information on the state of the heart and brain is often analyzed in isolation in prediction models. The root cause is the lack of an effective data structure and modeling mechanism that can reflect the intrinsic physiological relationship between the heart and brain. This isolated approach limits the depth of the model's exploration of the heart-brain synergistic injury mechanism, resulting in insufficient interpretation of complex pathological relationships in the prediction results and affecting the comprehensiveness and accuracy of risk assessment.

[0056] To address the aforementioned issues, this application provides a method for predicting brain injury risk after myocardial infarction based on multimodal data fusion. This method first acquires multimodal data including cardiac structure, brain structure, and brain blood flow dynamics, and then constructs these heterogeneous data into a graph structure with clear physiological meaning based on clinical medical knowledge. Next, a graph neural network is used to model this structure, enabling features from different sources to interact and fuse along predefined physiological correlation paths. Finally, a multi-task learning model is used to simultaneously output brain injury risk and functional status assessments. This approach combines prior medical knowledge with a deep learning model to achieve a structured representation and in-depth mining of the complex relationship between the heart and brain, thereby effectively improving the comprehensiveness and accuracy of risk prediction and solving the technical problem of insufficient integration of multi-source heterogeneous information in existing technologies.

[0057] To enable those skilled in the art to better understand the present application, the present application will be further described in detail below with reference to the accompanying drawings and specific embodiments. Obviously, the described embodiments are merely some embodiments of the present application, and not all embodiments. Based on the embodiments in this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0058] The core of this application is to provide a method for predicting brain injury risk after myocardial infarction based on multimodal data fusion. A flowchart of one specific implementation is shown below. Figure 1 As shown, the method includes:

[0059] Step 101: Acquire multimodal data of myocardial infarction patients, including cardiac magnetic resonance imaging, brain magnetic resonance imaging, and microwave echo signals of brain tissue.

[0060] In step 101, cardiac magnetic resonance imaging is a series of image data obtained by scanning the patient's heart using magnetic resonance imaging technology. This data can clearly show the structure, movement, and presence of damaged areas of the heart muscle.

[0061] Brain magnetic resonance imaging (MRI) is image data obtained by scanning a patient's brain using the same technology. It is mainly used to observe the structural morphology of brain tissue, such as whether there are abnormal lesions or microbleeds in the white matter area.

[0062] Microwave echo signals from brain tissue refer to electromagnetic waves of a specific frequency emitted to the patient's head through a non-contact microwave detection device, and the reflected signals generated by blood flow and cerebrospinal fluid movement within the brain tissue caused by heartbeat. These signals carry real-time information on changes in the hemodynamic state within the brain.

[0063] In this embodiment of the application, magnetic resonance images of the heart and brain are acquired through medical imaging equipment, and microwave echo signals reflecting the dynamics of blood flow in brain tissue are continuously captured by a dedicated microwave signal acquisition device deployed at the patient's bedside, thereby completing the synchronous acquisition of three key multimodal data of cardiac structure, brain structure and cerebral blood flow dynamics in myocardial infarction patients.

[0064] Step 102: Generate waveform data based on the phase difference and amplitude difference between multiple channels in the microwave echo signal.

[0065] Among them, phase difference refers to the difference between the phase values ​​of microwave echo signals from different antenna channels at the same time, amplitude difference refers to the difference between the signal strengths of microwave echo signals from different antenna channels at the same time, and waveform data is a dynamic sequence that can characterize the change of blood volume inside brain tissue over time.

[0066] In this embodiment, step 102 includes the following process:

[0067] Step 1021: Perform coherent accumulation processing on the microwave echo signal to obtain accumulated signals from multiple channels.

[0068] In step 1021, multiple channels refer to multiple independent antenna receiving units arranged in the microwave detection device, each antenna receiving unit corresponding to an independent signal channel; the accumulated signal refers to the signal output by each channel after coherent accumulation processing, representing the total echo energy received by that channel.

[0069] In this embodiment, the original microwave echo signal acquired by each antenna channel is first segmented and aligned according to the heartbeat cycle. Then, the signal values ​​at corresponding time points within several consecutive cycles in the same channel are weighted and averaged to obtain the cumulative signal with improved signal-to-noise ratio for each channel.

[0070] In practical applications, assume that the acquisition device contains 4 antenna channels, each channel continuously acquires microwave echo signals for 10 heartbeat cycles, and each cycle contains 512 sampling points; firstly, the signals of the 10 cycles are aligned one by one according to the sampling point positions, and then the arithmetic mean of the 10 signal values ​​at each sampling point position is calculated, and finally a cumulative signal containing 512 points is generated for each channel.

[0071] Step 1022: Calculate the amplitude difference and phase difference between the accumulated signals of the multiple channels.

[0072] In step 1022, the amplitude difference is calculated by subtracting the amplitude value of the second channel from the amplitude value of the first channel at the same time point for any two different channels' accumulated signals; the phase difference is calculated by subtracting the phase value of the second channel from the phase value of the first channel at the same time point for the same two channels' accumulated signals.

[0073] In this embodiment of the application, after obtaining the accumulated signals of all channels, the amplitude sequence and phase sequence are first extracted from each accumulated signal. Then, all possible channel pairings are traversed. For each pair of channels, the difference between their amplitude sequences is calculated point by point to obtain the amplitude difference sequence, and the difference between their phase sequences is calculated point by point simultaneously to obtain the phase difference sequence.

[0074] In practical applications, continuing with the 4-channel example above, there are a total of 6 different channel pairing combinations. For the combination of the first channel and the second channel, the amplitude values ​​of 512 points of the accumulated signal of the first channel and the amplitude values ​​of 512 points of the accumulated signal of the second channel are read respectively, and the corresponding points are subtracted to obtain an amplitude difference sequence containing 512 points. At the same time, the phase values ​​of 512 points of the accumulated signal of the first channel and the phase values ​​of 512 points of the accumulated signal of the second channel are read, and the corresponding points are subtracted to obtain a phase difference sequence containing 512 points. The same operation is performed on the other 5 channel pairing combinations.

[0075] Step 1023: Spatiotemporally integrate the amplitude difference and phase difference to generate a comprehensive difference sequence.

[0076] In step 1023, spatiotemporal integration refers to merging the amplitude difference and phase difference calculated at different time points from different channel pairings according to a preset fusion rule to form a single sequence that can comprehensively reflect the overall pattern of signal differences between multiple channels; the comprehensive difference sequence is the output result of the integration process.

[0077] In this embodiment, the amplitude difference sequence and phase difference sequence calculated from all channel pairings are used as input. First, each sequence is normalized to the same numerical range. Then, the amplitude difference values ​​from all channel pairings at the same time point are weighted and averaged, and the phase difference values ​​from all channel pairings at the same time point are weighted and averaged. Finally, the two average values ​​at each time point are multiplied to generate a comprehensive difference sequence that corresponds one-to-one with the time point of the original signal.

[0078] In practical applications, assume there are 6 amplitude difference sequences and 6 phase difference sequences, each with 512 points. First, normalize each sequence, for example, by linearly scaling all values ​​to between 0 and 1. For the first time point, sum the values ​​of the 6 amplitude difference sequences at the first point and divide by 6 to obtain the amplitude average A1. Sum the values ​​of the 6 phase difference sequences at the first point and divide by 6 to obtain the phase average P1. Calculate the product of A1 and P1 as the value of the comprehensive difference sequence at the first point. Repeat this process for the second to the 512th time points to finally generate a comprehensive difference sequence containing 512 points.

[0079] Step 1024: Discretize the integrated difference sequence to form waveform data.

[0080] In step 1024, discretization sampling refers to extracting a specific number of data points from the comprehensive difference sequence at fixed time intervals.

[0081] In this embodiment of the application, the comprehensive difference sequence is downsampled, that is, according to a preset sampling interval, one data point is retained every fixed number of original data points, thereby reducing the number of data points contained in the comprehensive difference sequence to a preset length, and the downsampled sequence is used as the final waveform data.

[0082] In practical applications, assuming the comprehensive difference sequence has 512 points and the preset waveform data length is 128 points, the sampling interval is set to 4. Starting from the first point of the comprehensive difference sequence, one point is taken every 4 points, that is, the 1st, 5th, 9th... up to the 509th point is extracted, for a total of 128 points. These 128 points are arranged in order to form waveform data.

[0083] This application converts multi-channel microwave echo signals into waveform data that characterizes the dynamics of cerebral blood flow, providing standardized temporal features that can reflect real-time changes in blood flow volume within brain tissue for subsequent analysis.

[0084] Step 103: Using the first feature data in the cardiac magnetic resonance image, the second feature data in the brain magnetic resonance image, and the waveform data as nodes, and establishing connection edges between the corresponding nodes according to a preset clinical medical knowledge base, a heterogeneous graph is formed.

[0085] In this graph, a node is the basic unit that represents a specific feature extracted from different data sources; a connecting edge is a line segment that connects two nodes and indicates that there is a relationship between the features represented by the two nodes.

[0086] A clinical medical knowledge base is a pre-organized and stored collection of data describing the interrelationships between the cardiovascular and cerebrovascular systems in terms of anatomical structure and physiological function.

[0087] Heterogeneous graphs are a graph data structure that allows for the existence of multiple different types of nodes and multiple different types of connecting edges.

[0088] In this embodiment, step 103 includes the following process:

[0089] Step 1031: Based on the cardiac magnetic resonance imaging, brain magnetic resonance imaging and waveform data, extract and construct the first node corresponding to the cardiac image features, the second node corresponding to the brain image features and the third node corresponding to the waveform features, respectively.

[0090] In step 1031, the first node is the node corresponding to each independent feature extracted from the cardiac magnetic resonance image, and the feature data contained in the node is used to describe the state of the myocardium; the second node is the node corresponding to each independent feature extracted from the brain magnetic resonance image, and the feature data contained in the node is used to describe the structural state of the brain tissue; the third node is the node corresponding to each data point extracted from the waveform data according to the preset sampling points, and the feature data contained in the node is used to describe the dynamic state of cerebral blood flow at a specific moment.

[0091] In this embodiment, firstly, the cardiac magnetic resonance image is processed to extract several quantization features. Each quantization feature corresponds to a first node, and the value of the quantization feature is used as the initial feature value of the first node. Next, the brain magnetic resonance image is processed to extract several quantization features. Each quantization feature corresponds to a second node, and the value of the quantization feature is used as the initial feature value of the second node. Then, the waveform data is arranged in chronological order, the waveform value corresponding to each time point is taken as a feature, and a third node is generated for each feature. The waveform value is used as the initial feature value of the third node.

[0092] In practical applications, assuming three features are extracted from cardiac magnetic resonance imaging, namely, left ventricular ejection fraction 55%, myocardial injury area 12%, and ventricular wall motion score index 1.8, three first nodes are generated, with initial feature values ​​of

[55] ,

[12] , and [1.8], respectively. Assuming two features are extracted from brain magnetic resonance imaging, namely, white matter lesion volume 8 cubic centimeters and microbleed count 3, two second nodes are generated, with initial feature values ​​of [8] and [3], respectively. Assuming the waveform data contains 128 sampling points, 128 third nodes are generated. Assuming the value of the first sampling point is 0.56, the initial feature value of the first third node is [0.56], and so on until the 128th sampling point.

[0093] Step 1032: Based on the connection rules defined in the clinical medical knowledge base, establish undirected edges, directed edges, or bidirectional edges representing the correlation of features within various types of nodes and between nodes of different categories.

[0094] In step 1032, the connection rules are a series of predefined logical judgment conditions used to determine whether a connection edge needs to be established between any two nodes and in what direction the connection edge should be established; an undirected edge indicates that there is a mutual association between two nodes and the association has no directionality, a directed edge indicates that the association has unidirectionality from the starting node to the target node, and a bidirectional edge indicates that there is a relationship between two nodes in two opposite directions.

[0095] In this embodiment of the application, all generated nodes are traversed, and each pair of nodes is checked according to the predefined relationship entries in the clinical medical knowledge base. For example, if the knowledge base records "a decrease in left ventricular ejection fraction may affect the overall blood flow to the brain", the corresponding rule is: to establish a directed edge between all first nodes representing left ventricular ejection fraction and all third nodes representing cerebral blood flow, with the direction pointing from the first node to the third node.

[0096] If the knowledge base records that "the extent of myocardial injury is correlated with abnormal ventricular wall motion", then the corresponding rule is: establish an undirected edge between the first node representing the extent of myocardial injury and the first node representing the ventricular wall motion score; in this way, gradually establish corresponding connection edges between all node pairs that satisfy the connection rule.

[0097] In practical applications, continuing the previous example, suppose the clinical medical knowledge base defines a rule: there is a correlation between the volume of white matter lesions and the stability of cerebral blood flow pulsation signals, and this correlation is bidirectional; then, when traversing nodes, a bidirectional edge will be established between each second node representing the volume of white matter lesions and each third node representing the waveform value; suppose the knowledge base defines another rule: the extent of myocardial injury may cause changes in the blood flow regulation function of specific brain regions, which is a unidirectional influence process; then, a directed edge will be established between the first node representing the extent of myocardial injury and each third node, pointing from the first node to the third node.

[0098] Step 1033: Integrate all types of nodes and all types of connection edges to obtain heterogeneous edges.

[0099] In this embodiment of the application, all first nodes, all second nodes and all third nodes generated in step 1031, as well as all undirected edges, directed edges and bidirectional edges established in step 1032, are merged together to form a complete graph data structure, which is the required heterogeneous graph.

[0100] In practical applications, integrating all the elements from the above examples, the heterogeneous graph contains a total of 133 nodes, including 3 first nodes, 2 second nodes, and 128 third nodes. The number of connecting edges is determined by the specific connection rules. For example, according to the two rules mentioned above, at least 128 edges will be established, including bidirectional edges between white matter lesion nodes and waveform nodes, plus 128 directed edges from myocardial injury nodes to waveform nodes, for a total of 256 connecting edges. Finally, by storing the information of all nodes and all connecting edges, including node type, node feature value, connecting edge type, and connecting direction, in a unified data structure, the complete heterogeneous graph is obtained.

[0101] This application constructs a heterogeneous graph that integrates the structural and blood flow dynamics of the heart and brain, providing a structured data foundation that clearly reflects the physiological relationship between the heart and brain for subsequent graph neural network analysis.

[0102] Step 104: Process the heterogeneous graph using a graph neural network to obtain associated features.

[0103] The associated feature refers to the comprehensive vector representation obtained after learning by the graph neural network. In this application, the feature specifically describes how cardiac injury features in the context of acute myocardial infarction affect brain structure and function features through physiological coupling pathways.

[0104] In this embodiment, step 104 includes the following process, such as... Figure 2 As shown:

[0105] Step 1041: Obtain the initial feature vector of each node in the heterogeneous graph.

[0106] In step 1041, the initial feature vector refers to the data vector carried by the node before entering the graph neural network. In this application, the initial feature vector of the first node quantifies the state of cardiac damage, the initial feature vector of the second node quantifies the structural lesions of the brain, and the initial feature vector of the third node represents the dynamic cerebral blood flow state caused by the myocardial infarction event.

[0107] In this embodiment, numerical information of each feature node is first extracted from the heterogeneous graph constructed for myocardial infarction risk prediction. Specifically, the percentage value of each first node representing the left ventricular ejection fraction is obtained, the dimensionless value of each first node representing the myocardial strain value is obtained, the cubic centimeter value of each second node representing the white matter lesion volume is obtained, and finally the dimensionless waveform value of each third node representing the relative value of cerebral blood volume at a specific sampling time is obtained. These values ​​are then encapsulated into corresponding one-dimensional arrays to form the initial state of each node.

[0108] In practical applications, following the construction example in step 103, the initial feature vector of a first node in the figure is a one-dimensional array containing the element 12, which represents that the proportion of myocardial injury is 12%. At the same time, the initial feature vector of a second node is a one-dimensional array containing the element 8, which represents that the volume of white matter lesions is 8 cubic centimeters. In addition, the initial feature vector of a third node is a one-dimensional array containing the element 0.56, which represents that the relative value of cerebral blood volume at a specific time is 0.56, and this value has been standardized.

[0109] Step 1042: Input the structural information of the heterogeneous graph and the initial feature vectors of all nodes into the graph neural network, and perform multi-layer message passing through the graph neural network. In each layer, for each current node in the heterogeneous graph, perform aggregation and update operations to obtain the updated feature value vector.

[0110] Step 1042 may specifically include the following steps:

[0111] A1: Based on the structural information of the heterogeneous graph and combined with the preset coupling mapping relationship between cardiovascular and cerebrovascular systems, determine all adjacent nodes that have a direct connection relationship with the current node.

[0112] In step A1, the coupling mapping relationship between cardiovascular and cerebrovascular systems refers to data entries describing the physiological connections between specific cardiac features and specific brain features; in this application, this relationship is encoded as connection edges with clinical meaning labels in a heterogeneous graph, used to identify key pathophysiological associations during message passing.

[0113] In this embodiment of the application, when the graph neural network processes messages at each layer, for each current node in the graph, its direct neighbors are first found based on the connection edges of the heterogeneous graph, and then the neighboring nodes are distinguished according to the clinical meaning labels carried by the connection edges. For example, if the current node is the first node representing anterior wall myocardial injury, the method will select the third node with the label of affecting forward blood flow and representing the blood flow waveform of a specific brain region as the key neighbor.

[0114] In practical applications, assuming the current node is a myocardial injury range node with an initial feature vector of 12, the heterogeneous graph is first queried and it is found that the node is connected to 32 third nodes representing the fluctuation of blood volume in the anterior part of the brain through a directed edge labeled as affecting forward blood flow, and is also connected to a second node representing the ventricular wall motion score through an undirected edge labeled as coordinating with ventricular wall motion. In the current scenario, the influence of forward blood flow is given higher weight, so these 32 third nodes are identified as core neighbor nodes, while the ventricular wall motion score node is a secondary neighbor node.

[0115] A2: The initial feature vectors of all adjacent nodes are weighted and summed using the first aggregation function preset in the graph neural network to generate an intermediate feature vector.

[0116] In step A2, the first aggregation function refers to the calculation rule for weighted fusion of features of neighboring nodes. In this application, the weight of the function is set based on the association strength coefficient predefined in the knowledge base. This coefficient reflects the medical consensus strength of the contribution of a specific heart-brain coupling relationship to the risk of brain injury. It should be noted that the specific expression used for the function in this embodiment is not specifically limited, and can be set according to the actual situation.

[0117] In this embodiment, for the current node, the first aggregation function determines its weight based on the association strength coefficient of the corresponding connecting edges of each adjacent node, and then multiplies the feature vector of each adjacent node by its respective weight coefficient and sums them to obtain an intermediate feature vector that integrates the information of neighbors of different importance.

[0118] In practical applications, continuing the previous example, for the nodes in the current myocardial injury range, the correlation strength coefficient of its 32 core waveform nodes is preset to 0.8, and after average distribution, the coefficient of each node is 0.025. The correlation strength coefficient of the secondary ventricular wall motion nodes is 0.2. Assuming that the average feature vector of the 32 waveform nodes is 0.5, the weighted contribution is 0.4. The feature vector of the ventricular wall motion nodes is 1.8, so the weighted contribution is 0.36. Adding the two together, we get the intermediate feature vector of 0.76.

[0119] A3: Merge the intermediate feature vector with the feature value vector of the current node itself to form a merged vector.

[0120] In step A3, the merged vector refers to the vector that integrates the node's own information with its network influence information; in this application, this operation aims to make the node features simultaneously include its own pathological state and its propagation effect through the heart-brain coupling network.

[0121] In this embodiment of the application, after obtaining the intermediate feature vector, the initial feature vector of the current node is concatenated with the intermediate feature vector to form a merged vector that simultaneously contains the source feature value and the aggregated influence value.

[0122] In practical applications, the self-feature vector of the current myocardial injury range node is 12, and the intermediate feature vector is 0.76. The merged vector formed by concatenating the two is a one-dimensional array containing the elements 12 and 0.76.

[0123] A4: The merged vector is processed by the preset update rules in the graph neural network to output the updated feature value vector.

[0124] In step A4, the update rule refers to the calculation process of performing a nonlinear transformation on the merged vector to obtain a new feature vector. In this application, the rule is implemented through a neural network layer designed for the cardiovascular risk prediction task, which aims to map the merged vector to a higher-dimensional feature space that better represents the overall risk state.

[0125] In this embodiment, the merged vector is input into a fully connected neural network layer. This layer performs linear transformation and nonlinear activation processing on the input, and outputs an updated feature vector with higher dimension and more abstract representation of the comprehensive state of the node in the heart-brain connection network.

[0126] In practical applications, the merged vector is input into a fully connected layer with 8 neurons. After linear transformation and ReLU activation function processing, an 8-dimensional updated feature value vector is obtained. For example, this vector contains elements 2.1, 0, 5.7, 0.3, 0, 1.2, 0, 4.4. This vector represents the multidimensional state encoding of the myocardial injury node in the brain injury risk map after myocardial infarction.

[0127] Step 1043: After at least two layers of message passing process in the graph neural network, the updated feature value vectors of all nodes are extracted from the last layer of the graph neural network, and associated features are generated based on the updated feature value vectors of all nodes.

[0128] In step 1043, the message passing process refers to the complete computational loop in which node features in the graph neural network propagate and update information according to the graph structure, and the last layer refers to the final computational layer after the graph neural network has completed a preset number of message passing operations.

[0129] In this embodiment of the application, the message passing process defined in step 1042 is first executed at least twice, so that the information of the three types of nodes, namely heart features, brain structure features and blood flow waveform features, can be interacted and fused in multiple rounds through the connection edges; then, after the last message passing is completed, the feature value vectors of all nodes after the final update are collected from the output layer of the graph neural network; finally, these vectors are spliced ​​and integrated according to the original category order of the nodes to form the final associated features.

[0130] In practical applications, assuming the graph neural network is configured with a three-layer message passing mechanism, after completing the three-layer processing consisting of steps A1 to A4, the feature value vectors output by all 133 nodes in the third layer are obtained. These vectors may have different dimensions; for example, the feature vector of the heart node is 8-dimensional, the feature vector of the brain structure node is 6-dimensional, and the feature vector of the blood flow waveform node is 4-dimensional. Then, these 133 feature vectors are concatenated end to end in the order of the first node, the second node, and the third node to form a comprehensive vector with a total dimension of 3×8+2×6+128×4=548 dimensions. This comprehensive vector is the associated feature used for subsequent risk prediction.

[0131] This application fully explores the deep connections between nodes through multi-layer message passing and integrates the final state of all nodes into a unified association feature, thereby providing a comprehensive input representation rich in heart-brain association information for subsequent multi-task prediction.

[0132] Step 105: Input the associated features into a multi-task learning model for prediction, and output the probability of future brain injury and the degree of impairment of self-regulation function after myocardial infarction.

[0133] Among them, the risk probability refers to the estimate of the likelihood of brain injury events such as stroke or cognitive decline occurring within a specific future time window, with a value between 0 and 1; the result of the degree of impairment of self-regulation function refers to the quantitative assessment of the impact on the cerebrovascular system's ability to automatically regulate blood flow according to blood pressure fluctuations to maintain stable cerebral perfusion.

[0134] It should be noted that the structural design of the multi-task learning model in steps 1051 to 1052 below is exemplary, and the specific structure of the model in this application embodiment is not limited, and can be set accordingly according to the actual situation.

[0135] In this embodiment, step 105 includes the following process:

[0136] Step 1051: Input the associated features into the multi-task learning model, and process the associated features through the shared parameter layer of the multi-task learning model to obtain a shared feature representation.

[0137] In step 1051, the shared parameter layer refers to the neural network layer in the multi-task learning model that is used by all subsequent prediction task branches, and the shared feature representation refers to the intermediate data representation obtained after the associated features are processed by the shared layer, which can simultaneously support the prediction of multiple downstream tasks.

[0138] In this embodiment of the application, the associated feature vector obtained in step 1043 is first input into the multi-task learning model. The shared parameter layer of the model performs linear transformation and nonlinear activation processing on the input feature vector to extract common feature information that is discriminative for both brain injury risk assessment and brain function assessment. The processed output is the shared feature representation.

[0139] In practical applications, following the embodiment of step 1043, the 548-dimensional associated feature vector is input into the shared parameter layer, which is a fully connected layer with 256 neurons. After a linear transformation of the weight matrix W1 and the bias vector b1 of this layer, i.e., a matrix-vector multiplication and vector addition operation is performed, and then processed by the ReLU activation function, a 256-dimensional vector is finally obtained. This 256-dimensional vector is the shared feature representation.

[0140] Step 1052: Process the shared feature representation through the first task branch of the multi-task learning model to obtain the probability of future brain injury, and process the shared feature representation through the second task branch of the multi-task learning model to obtain the result of the degree of impairment of self-regulation function.

[0141] Step 1052 may specifically include the following steps:

[0142] Step B1: Input the shared feature representation into the first neural network layer of the first task branch. The first neural network layer performs linear transformation and nonlinear activation operations on the shared feature representation to obtain the first hidden layer features.

[0143] In step B1, the first hidden layer feature refers to the intermediate feature obtained after the shared feature representation has been processed by the preliminary neural network layer of the first task branch, which is more focused on the brain injury risk prediction task.

[0144] In this embodiment of the application, in the first task branch, the shared feature representation is first input into its first neural network layer. This layer performs a weighted summation operation, i.e., a linear transformation, on the input. Then, the result is activated by a nonlinear function such as ReLU to generate a first hidden layer feature with adjusted dimensions and greater task relevance.

[0145] In practical applications, the 256-dimensional shared feature representation is input into the first neural network layer of the first task branch, which has 128 neurons. After linear transformation of the weight matrix W2 and bias vector b2 of this layer, and then processing by the ReLU activation function, a 128-dimensional vector is obtained, which is the first hidden layer feature.

[0146] Step B2: Input the first hidden layer features into the second neural network layer of the first task branch. The second neural network layer performs dimensionality reduction processing on the first hidden layer features and outputs scalar values.

[0147] In step B2, the scalar value refers to a single real value, which is an unnormalized predicted value directly associated with the risk of brain injury, obtained by further transformation of the first hidden layer features.

[0148] In this embodiment of the application, after obtaining the first hidden layer features, they are input into the second neural network layer of the first task branch. This layer maps the high-dimensional feature vector into a single numerical value through linear transformation, thereby outputting a scalar value that can initially reflect the magnitude of the risk.

[0149] In practical applications, the 128-dimensional first hidden layer features are input into the second neural network layer of the first task branch. This layer has only one neuron. After the linear transformation of the weight vector W3 and the bias value b3 of this layer, that is, after performing a dot product operation and adding the bias, a real value is output, for example, 2.74.

[0150] Step B3: Based on the scalar value, use a preset probability transformation function to obtain the probability of future brain injury.

[0151] In step B3, the probability transformation function is a mathematical function that maps any real-valued scalar to the interval between 0 and 1. In this application, the Sigmoid function is used.

[0152] In this embodiment of the application, after obtaining the scalar value, the scalar value is input into a preset Sigmoid function for calculation. The output value of the function is limited to between 0 and 1, which is the final risk probability value.

[0153] In practical applications, the scalar value 2.74 is input into the Sigmoid function for calculation, where the expression for the Sigmoid function is: ,in Let represent the risk probability, e be the natural constant, and x be the input scalar value. Then, substitute x = 2.74 into the formula to calculate. Therefore, the probability of risk obtained is 0.94.

[0154] For the second task branch, its processing flow is structurally parallel and similar to that of the first task branches B1 to B3, but the parameters of the neural network layers are independent and the task objectives are different. Specifically, the second task branch also receives the shared feature representation, processes it sequentially through its independent neural network layer, and finally outputs a quantitative value that represents the degree of impairment of self-regulation function. This value can be a continuous value or a discrete score. For example, the output value is 0.65, which can be directly used as the result of the degree of impairment.

[0155] This application uses a multi-task learning model to simultaneously complete two closely related clinical predictions. Shared feature extraction improves data utilization efficiency and ensures the consistency and synergy of the two prediction results in the risk assessment scenario after myocardial infarction.

[0156] Figure 3 A schematic diagram of the structure of a brain injury risk prediction system based on multimodal data fusion provided in this application embodiment is shown below. Figure 3 As shown, the system includes:

[0157] The acquisition module 31 is used to acquire multimodal data of patients with myocardial infarction, including cardiac magnetic resonance imaging, brain magnetic resonance imaging, and microwave echo signals of brain tissue.

[0158] The generation module 32 is used to generate waveform data based on the phase difference and amplitude difference between multiple channels in the microwave echo signal.

[0159] The module 33 is used to establish the first feature data in the cardiac magnetic resonance image, the second feature data in the brain magnetic resonance image, and the waveform data as nodes, and establish connection edges between the corresponding nodes according to the preset clinical medical knowledge base to form a heterogeneous graph.

[0160] Processing module 34 is used to process the heterogeneous graph using a graph neural network to obtain associated features.

[0161] The input module 35 is used to input the associated features into a multi-task learning model for prediction, and output the probability of future brain injury after myocardial infarction and the degree of impairment of self-regulation function.

[0162] The multimodal data fusion-based risk prediction system for post-myocardial infarction brain injury in this application is used to implement the aforementioned multimodal data fusion-based risk prediction method for post-myocardial infarction brain injury. Therefore, the specific implementation of the multimodal data fusion-based risk prediction system for post-myocardial infarction brain injury can be found in the embodiment section of the multimodal data fusion-based risk prediction method for post-myocardial infarction brain injury. The specific implementation can be referred to the description of the corresponding embodiments, which will not be repeated here.

[0163] This application also provides an electronic device, comprising: a memory for storing a computer program; and a processor for executing the computer program to implement the steps of any of the above-described methods for predicting brain injury risk after myocardial infarction based on multimodal data fusion.

[0164] This application also provides a computer-readable storage medium storing a computer program, which, when executed by a processor, implements the steps of any of the above-described methods for predicting brain injury risk after myocardial infarction based on multimodal data fusion.

[0165] In one exemplary embodiment, the aforementioned computer-readable storage medium may include, but is not limited to, various media capable of storing computer programs, such as USB flash drives, read-only memory, random access memory, portable hard drives, magnetic disks, or optical disks.

[0166] The embodiments of this application also provide a computer program product, which includes a computer program that, when executed by a processor, implements the steps in any of the embodiments of the method for predicting brain injury after myocardial infarction based on multimodal data fusion.

[0167] Those skilled in the art will further recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of both. To clearly illustrate the interchangeability of hardware and software, the components and steps of the various examples have been generally described in terms of functionality in the foregoing description. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0168] The foregoing has provided a detailed description of a method and system for predicting brain injury risk after myocardial infarction based on multimodal data fusion. Specific examples have been used to illustrate the principles and implementation methods of this application. The descriptions of the embodiments above are merely for the purpose of helping to understand the method and its core ideas. It should be noted that those skilled in the art can make various improvements and modifications to this application without departing from its principles, and these improvements and modifications also fall within the protection scope of this application.

Claims

1. A method for predicting brain injury risk after myocardial infarction based on multimodal data fusion, characterized in that, include: Acquire multimodal data of patients with myocardial infarction, including cardiac magnetic resonance imaging, brain magnetic resonance imaging, and microwave echo signals of brain tissue; Based on the phase difference and amplitude difference between multiple channels in the microwave echo signal, waveform data is generated; The first feature data in the cardiac magnetic resonance image, the second feature data in the brain magnetic resonance image, and the waveform data are used as nodes, and connection edges are established between the corresponding nodes according to a preset clinical medical knowledge base to form a heterogeneous graph. The heterogeneous graph is processed using a graph neural network to obtain associated features; The associated features are input into a multi-task learning model for prediction, and the results of the probability of future brain injury after myocardial infarction and the degree of impairment of self-regulation function are output.

2. The method according to claim 1, characterized in that, The process of using a graph neural network to process the heterogeneous graph to obtain associated features includes: Obtain the initial feature vector of each node in the heterogeneous graph; The structural information of the heterogeneous graph and the initial feature vectors of all nodes are input into the graph neural network. Multi-layer message passing is performed through the graph neural network. In each layer, for each current node in the heterogeneous graph, aggregation and update operations are performed to obtain the updated feature value vector. After at least two layers of message passing in the graph neural network, the updated feature vectors of all nodes are extracted from the last layer of the graph neural network, and associated features are generated based on the updated feature vectors of all nodes.

3. The method according to claim 2, characterized in that, The method of multi-layer message passing through the graph neural network involves performing aggregation and update operations on each current node in the heterogeneous graph at each layer to obtain an updated feature value vector, including: Based on the structural information of the heterogeneous graph and combined with the preset coupling mapping relationship between cardiovascular and cerebrovascular systems, all adjacent nodes that have a direct connection relationship with the current node are determined. The initial feature vectors of all adjacent nodes are weighted and summed using the first aggregation function preset in the graph neural network to generate an intermediate feature vector. The intermediate feature vector is merged with the feature value vector of the current node to form a merged vector; The merged vector is processed using the preset update rules in the graph neural network to output the updated feature value vector.

4. The method according to claim 1, characterized in that, The step of inputting the associated features into a multi-task learning model for prediction, and outputting the probability of future brain injury after myocardial infarction and the degree of impairment of self-regulation function, includes: The associated features are input into a multi-task learning model, and the associated features are processed through the shared parameter layer of the multi-task learning model to obtain a shared feature representation. The shared feature representation is processed through the first task branch of the multi-task learning model to obtain the probability of future brain injury, and the shared feature representation is processed through the second task branch of the multi-task learning model to obtain the degree of impairment of self-regulation function.

5. The method according to claim 4, characterized in that, The process of processing the shared feature representation through the first task branch of the multi-task learning model to obtain the probability of future brain injury includes: The shared feature representation is input into the first neural network layer of the first task branch. The first neural network layer performs linear transformation and nonlinear activation operations on the shared feature representation to obtain the first hidden layer features. The first hidden layer features are input into the second neural network layer of the first task branch. The second neural network layer performs dimensionality reduction processing on the first hidden layer features and outputs a scalar value. Based on the scalar value, the probability of future brain injury is obtained using a preset probability transformation function.

6. The method according to claim 1, characterized in that, The generation of waveform data based on the phase difference and amplitude difference between multiple channels in the microwave echo signal includes: The microwave echo signal is coherently accumulated to obtain accumulated signals from multiple channels; Calculate the amplitude difference and phase difference between the accumulated signals from multiple channels; The amplitude difference and phase difference are spatiotemporally integrated to generate a comprehensive difference sequence; The comprehensive difference sequence is discretized and sampled to form waveform data.

7. The method according to claim 1, characterized in that, The process of using the first feature data from the cardiac magnetic resonance imaging, the second feature data from the brain magnetic resonance imaging, and the waveform data as nodes, and establishing connection edges between corresponding nodes according to a preset clinical medical knowledge base to form a heterogeneous graph, includes: Based on the cardiac magnetic resonance imaging, brain magnetic resonance imaging, and waveform data, a first node corresponding to cardiac image features, a second node corresponding to brain image features, and a third node corresponding to waveform features are extracted and constructed respectively. Based on the connection rules defined in the clinical medical knowledge base, undirected edges, directed edges, or bidirectional edges representing the correlation of features are established within various types of nodes and between nodes of different categories. Integrate all types of nodes and all types of connection edges to obtain heterogeneous edges.

8. A system for predicting brain injury risk after myocardial infarction based on multimodal data fusion, characterized in that, include: The acquisition module is used to acquire multimodal data of patients with myocardial infarction, including cardiac magnetic resonance imaging, brain magnetic resonance imaging, and microwave echo signals of brain tissue. The generation module is used to generate waveform data based on the phase difference and amplitude difference between multiple channels in the microwave echo signal; A module is established to use the first feature data in the cardiac magnetic resonance image, the second feature data in the brain magnetic resonance image, and the waveform data as nodes, and to establish connection edges between the corresponding nodes according to a preset clinical medical knowledge base to form a heterogeneous graph. The processing module is used to process the heterogeneous graph using a graph neural network to obtain associated features; The input module is used to input the associated features into a multi-task learning model for prediction, and output the probability of future brain injury after myocardial infarction and the degree of impairment of self-regulation function.

9. An electronic device, characterized in that, include: Memory, used to store computer programs; A processor, configured to execute the computer program to implement the steps of the method for predicting brain injury risk after myocardial infarction based on multimodal data fusion as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, enables the method for predicting brain injury risk after myocardial infarction based on multimodal data fusion as described in any one of claims 1 to 7.

Images