Postoperative cognitive dysfunction dynamic prediction method based on multi-modal data fusion

Abstract

This invention discloses a dynamic prediction method for postoperative cognitive impairment based on multimodal data fusion, specifically relating to the fields of medical data analysis and artificial intelligence. The method involves acquiring and processing preoperative patient data to construct a standardized time-series dataset; extracting static baseline features, time-series dynamic features, and high-dimensional nonlinear features, and dynamically fusing them based on time attention and cross-modal interaction mechanisms to generate a multimodal joint representation vector; constructing a dynamic prediction model based on a long short-term memory network and a Transformer architecture, using the multimodal joint representation vector as input and the postoperative cognitive impairment risk probability and cognitive function score change trajectory within a preset time window as outputs to train the model; inputting real-time collected patient multimodal data into the trained dynamic prediction model, outputting the dynamic predicted probability and risk level of postoperative cognitive impairment, and updating the prediction results in real time based on the inflow of new data.

Description

Technical Field

[0001] This invention relates to the fields of medical data analysis and artificial intelligence technology, and more specifically, to a method for dynamic prediction of postoperative cognitive dysfunction based on multimodal data fusion. Background Technology

[0002] Postoperative cognitive impairment is a central nervous system complication that occurs after surgery, mainly manifested as a decline in cognitive abilities such as memory, attention, and executive function. Its incidence rate is as high as 10% to 60% in elderly surgical patients, seriously affecting their postoperative recovery quality and ability to live independently, and increasing the long-term risk of dementia and mortality. Therefore, accurate and dynamic early prediction of postoperative cognitive impairment is of significant clinical importance.

[0003] Currently, the prediction of postoperative cognitive impairment in clinical practice mainly relies on the empirical judgment of anesthesiologists or neurologists, or the use of a single assessment tool (such as the Mini-Mental State Examination). Existing technical solutions have the following shortcomings:

[0004] Limited data sources: Traditional methods rely heavily on preoperative neuropsychological assessments or a small number of intraoperative physiological indicators, failing to fully utilize multidimensional data including imaging, electrophysiology, and biomarkers, resulting in insufficient information utilization and limited predictive accuracy.

[0005] Lack of dynamism: Existing predictions are mostly static and single-point predictions, that is, risk assessment is carried out at a fixed time point before or after surgery. However, the occurrence and development of postoperative cognitive dysfunction is a dynamic process, which is affected by intraoperative events (such as blood pressure fluctuations and hypoxia) and postoperative rehabilitation in real time. Static predictions cannot capture this time-varying nature and are difficult to achieve real-time early warning.

[0006] The fusion method is simple: Even if a few studies attempt to combine multiple data, they usually use a simple feature splicing method, which fails to effectively model the complex interaction relationships and time dependencies between different modal data, and cannot fully explore the synergistic value of multimodal data. Summary of the Invention

[0007] To overcome the aforementioned deficiencies of the prior art, embodiments of the present invention provide a dynamic prediction method for postoperative cognitive dysfunction based on multimodal data fusion, in order to solve the problems mentioned in the background art.

[0008] To achieve the above objectives, the present invention provides the following technical solution:

[0009] S1: Acquire the patient's preoperative multimodal data, intraoperative dynamic physiological data, and postoperative short-term assessment data; perform time alignment, cleaning, and standardization on the multimodal data to construct a standardized time-series dataset;

[0010] S2: Extract the static baseline features, time-series dynamic features, and high-dimensional nonlinear features from the standardized time-series dataset respectively; based on the time attention mechanism and cross-modal interaction mechanism, dynamically fuse the extracted features to generate a multimodal joint representation vector with spatiotemporal correlation;

[0011] S3: Construct a dynamic prediction model based on long short-term memory network and Transformer architecture; use the multimodal joint representation vector as input and the postoperative cognitive impairment risk probability and cognitive function score change trajectory within a preset time window as output to train the dynamic prediction model;

[0012] S4: Input the real-time collected multimodal patient data into the trained dynamic prediction model, output the dynamic prediction probability and risk level of postoperative cognitive dysfunction, and update the prediction results in real time according to the inflow of new data.

[0013] Preferably, in step S1, a standardized multimodal data acquisition protocol is first established. Based on this protocol, preoperative multimodal data, intraoperative dynamic physiological data, and postoperative short-term assessment data of the patient are acquired. Specifically, the preoperative multimodal data includes: neuropsychological scale assessment data, such as the Montreal Basic Cognitive Assessment Scale and the Hospital Anxiety and Depression Scale, used to quantify the patient's baseline cognitive function and emotional state; resting-state functional magnetic resonance imaging data, which is obtained by acquiring brain function images of the patient in a resting state before surgery, used for subsequent construction of brain functional connectivity networks; electroencephalogram (EEG) data, which is obtained by acquiring preoperative resting-state EEG signals with eyes closed through scalp electrodes, used to extract neuroelectrophysiological characteristics such as EEG power spectrum and functional connectivity; blood biomarker data, including but not limited to Aβ42, Tau protein, neurofilament light chain protein, inflammatory factors, etc., obtained through preoperative venous blood sampling; and genomic data, focusing on acquiring single nucleotide polymorphism (SNP) site information related to cognitive function, such as APOE ε4 allele typing.

[0014] Intraoperative dynamic physiological data are continuously collected using an anesthesia monitor, EEG monitor, and intraoperative recording system in a high-frequency sampling manner. The sampling frequency varies from once per second to once per millisecond, depending on the parameter type. Specific data include: brain oxygen saturation, which is continuously monitored for brain tissue oxygenation using near-infrared spectroscopy; blood pressure, including non-invasive and invasive arterial blood pressure, recording systolic, diastolic, and mean arterial pressure; heart rate; anesthesia depth index, such as bispectral index or anesthesia entropy index; body temperature, including core temperature and surface temperature; and parameters such as end-tidal carbon dioxide partial pressure and blood oxygen saturation may also be included.

[0015] Postoperative short-term assessment data were collected at multiple pre-set time points during the postoperative recovery period, specifically 24 hours, 48 ​​hours, 72 hours postoperatively and before discharge. The data collected included: delirium assessment scale scores, such as the confusion assessment method or delirium rating scale; visual analog scale scores, used to assess the degree of postoperative pain; and Mini-Mental State Examination scores, used to assess the recovery of postoperative cognitive function.

[0016] Preferably, in step S2, firstly, based on the standardized time-series dataset constructed in S1, three types of complementary feature representations are extracted: static baseline features, time-series dynamic features, and high-dimensional nonlinear features. For the extraction of static baseline features, the focus is on non-time-series or quasi-static data collected preoperatively, specifically including scores of various dimensions of neuropsychological scales, blood biomarker concentration values, genomic risk locus typing information, and structural and morphological parameters such as whole-brain gray matter volume and cortical thickness extracted from resting-state functional magnetic resonance imaging. A multilayer perceptron is used as the encoder to map the aforementioned multi-source static data into a low-dimensional embedding space, and a two-layer fully connected network is used in conjunction with the ReLU activation function. The algorithm employs Dropout regularization to generate a static baseline feature vector with fixed dimensions. For extracting temporal dynamic features, it processes continuously acquired intraoperative physiological time-series data, including time series of parameters such as cerebral oxygen saturation, blood pressure, heart rate, anesthesia depth index, and body temperature. A bidirectional long short-term memory network (Bi-LSTM) is used as the temporal encoder. The multi-parameter physiological vector at each time point is taken as input. A forward LSTM layer captures the temporal dependencies from the start of surgery to the current time point, and a backward LSTM layer captures the reverse temporal dependencies from the current time point to the end of surgery. The hidden states in both directions are concatenated to form the temporal dynamic feature vector for each time step. This Bi-LSTM structure effectively models the contextual information of intraoperative events, enabling the full representation of complex temporal patterns such as the duration of hypotension events and the cumulative effect of blood pressure fluctuations.

[0017] Preferably, in step S3, a multi-task dynamic prediction model is first constructed based on a long short-term memory network and a Transformer architecture. This model employs an encoder-decoder structure, using the multimodal joint representation vector sequence generated in step S2 as input and the postoperative cognitive impairment risk probability and cognitive function score change trajectory within a preset time window as output, thereby achieving dynamic prediction of postoperative cognitive impairment. The overall model architecture includes three core components: a temporal encoding module, a multi-task decoding module, and a joint loss optimization module. The temporal encoding module uses a Transformer encoder as its basic architecture to handle the temporal dependencies of the multimodal joint representation vector. Let the input sequence be... ,in For the number of time steps, To define the dimension of the joint representation vector; firstly, temporal information is introduced through positional encoding, which uses a sine-cosine function:

[0018]

[0019]

[0020] in, This is represented as a position encoding function, where pos is the time position index. The dimension index is used; the positional encoding is added to the input features and then input into a multi-layer Transformer encoder layer; each encoder layer contains a multi-head self-attention mechanism and a feedforward neural network, and the calculation method of multi-head self-attention is as follows:

[0021]

[0022] The calculation for each attention head is as follows:

[0023]

[0024] By stacking multiple Transformer encoders, the model captures long-range dependencies between any time steps in the sequence and outputs encoded temporal feature representations. ;

[0025] The multi-task decoding module adopts a dual-branch parallel structure, including a risk probability prediction branch and a trajectory prediction branch. The two branches share the output features of the temporal coding module, but each uses an independent decoding network. The risk probability prediction branch outputs the probability of the patient developing postoperative cognitive impairment at multiple future time points; this branch supports the output features of the last layer of the temporal coding module. The current state is summarized and input into a sequence decoder composed of a long short-term memory network; the prediction time window is assumed to contain... At each future time point, the LSTM decoder generates the hidden state step by step:

[0026]

[0027] in, This represents the hidden state of the LSTM decoder at the k-th future time point. This is the prediction result from the previous time step (the true labels are used during training, and the predicted values ​​are used during inference).

[0028] Preferably, in step S4, the trained dynamic prediction model is first deployed to the clinical prediction system, which connects in real time with the hospital information system, anesthesia monitoring equipment, and postoperative follow-up record platform. When a new patient enters the perioperative management process, the system automatically collects real-time multimodal data, including preoperative assessment data, intraoperative continuous physiological monitoring data, and postoperative short-term follow-up data. Following the data preprocessing and feature extraction processes described in S1 and S2, the system converts the real-time collected raw data into a multimodal joint representation vector acceptable to the model. This representation vector is then input into the trained dynamic prediction model, which calculates through forward propagation and outputs the dynamic prediction probability and risk level of postoperative cognitive dysfunction for the patient within a preset future time window.

[0029] The model output includes predictions in three dimensions; the first dimension is the temporal risk probability prediction, which is the probability value that the patient will experience postoperative cognitive impairment at multiple time points such as postoperative day 1, day 3, day 7, and day 14. The data is visualized as a probability curve, with the horizontal axis representing postoperative time points and the vertical axis representing predicted probability values, intuitively showing the dynamic trend of risk over time. The second dimension is the prediction of cognitive function trajectory, with the model simultaneously outputting predicted cognitive function scores for the patient at the aforementioned future time points. This generates a cognitive function change curve, reflecting the expected recovery trajectory or deterioration trend of the patient's postoperative cognitive function, providing clinicians with continuous quantitative references in addition to binary risk assessment; the third dimension is risk level classification, where the system automatically performs graded early warning based on predicted probability values, setting three risk thresholds: when When the risk level is determined to be low, the system displays a green icon; when... When the risk level is determined to be medium, the system displays a yellow indicator and prompts clinicians to monitor the patient's condition; when When a high-risk level is identified, the system displays it in red and triggers an active warning. The warning threshold is dynamically adjusted according to the actual clinical application scenario, and the threshold settings for different medical institutions can be flexibly modified through the system configuration interface.

[0030] The technical effects and advantages of this invention are as follows:

[0031] This invention significantly improves the prediction accuracy of postoperative cognitive impairment by fusing preoperative, intraoperative, and postoperative multimodal data to construct a multimodal joint representation vector. It introduces a time attention mechanism and a cross-modal interaction mechanism to achieve dynamic fusion of multi-source data, enhancing the model's ability to model key time points and nonlinear relationships between modalities. A multi-task dynamic prediction model based on Transformer and LSTM supports the synchronous output of risk probability and cognitive function change trajectories, providing richer clinical decision support. A sliding window and incremental update strategy are employed to achieve real-time dynamic updates of prediction results, adapting to the continuous influx of perioperative clinical data. Attached Figure Description

[0032] Figure 1 This is a schematic diagram of the method flow of the present invention. Detailed Implementation

[0033] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0034] Please see Figure 1 As shown, this invention provides a method for dynamic prediction of postoperative cognitive dysfunction based on multimodal data fusion, including:

[0035] S1: Acquire the patient's preoperative multimodal data, intraoperative dynamic physiological data, and postoperative short-term assessment data; perform time alignment, cleaning, and standardization on the multimodal data to construct a standardized time-series dataset;

[0036] In step S1, a standardized multimodal data acquisition protocol is first established. Based on this protocol, preoperative multimodal data, intraoperative dynamic physiological data, and postoperative short-term assessment data of the patient are acquired. Specifically, the preoperative multimodal data includes: neuropsychological scale assessment data, such as the Montreal Basic Cognitive Assessment Scale and the Hospital Anxiety and Depression Scale, used to quantify the patient's baseline cognitive function and emotional state; resting-state functional magnetic resonance imaging data, which is obtained by acquiring brain function images of the patient in a resting state before surgery, used for subsequent construction of brain functional connectivity networks; electroencephalogram (EEG) data, which is obtained by acquiring preoperative resting-state EEG signals with eyes closed through scalp electrodes, used to extract neuroelectrophysiological characteristics such as EEG power spectrum and functional connectivity; blood biomarker data, including but not limited to Aβ42, Tau protein, neurofilament light chain protein, inflammatory factors, etc., obtained through preoperative venous blood sampling; and genomic data, focusing on acquiring single nucleotide polymorphism (SNP) site information related to cognitive function, such as APOE ε4 allele typing.

[0037] Intraoperative dynamic physiological data are continuously collected using an anesthesia monitor, EEG monitor, and intraoperative recording system in a high-frequency sampling manner. The sampling frequency varies from once per second to once per millisecond, depending on the parameter type. Specific data include: brain oxygen saturation, which is continuously monitored for brain tissue oxygenation using near-infrared spectroscopy; blood pressure, including non-invasive and invasive arterial blood pressure, recording systolic, diastolic, and mean arterial pressure; heart rate; anesthesia depth index, such as bispectral index or anesthesia entropy index; body temperature, including core temperature and surface temperature; and parameters such as end-tidal carbon dioxide partial pressure and blood oxygen saturation may also be included.

[0038] Postoperative short-term assessment data were collected at multiple pre-set time points during the postoperative recovery period, specifically 24 hours, 48 ​​hours, 72 hours and before discharge. The data collected included: delirium assessment scale scores, such as the confusion assessment method or delirium rating scale; visual analog scale scores, used to assess the degree of postoperative pain; and Mini-Mental State Examination scores, used to assess the recovery of postoperative cognitive function.

[0039] After data acquisition, the raw multimodal data underwent preprocessing: for functional magnetic resonance imaging (fMRI) data, head motion correction, temporal layer correction, spatial registration to a standard template, spatial smoothing, and bandpass filtering were performed sequentially; for electroencephalogram (EEG) data, baseline drift removal, artifact removal, power frequency filtering, and rereference processing were performed; for physiological time series data, missing values ​​were handled using linear interpolation or spline interpolation, and low-pass filtering was applied to remove high-frequency noise; for scale scores and biomarker data, outlier detection and correction were performed, and the data were mapped to a unified dimension using Z-score standardization or max-min normalization; after the above cleaning and standardization processes were completed, all modal data were aligned in chronological order using the patient's unique identifier and timestamp as indexes to construct a standardized time-series dataset.

[0040] S2: Extract the static baseline features, time-series dynamic features, and high-dimensional nonlinear features from the standardized time-series dataset respectively; based on the time attention mechanism and cross-modal interaction mechanism, dynamically fuse the extracted features to generate a multimodal joint representation vector with spatiotemporal correlation;

[0041] In S2, firstly, based on the standardized time-series dataset constructed in S1, three types of complementary feature representations are extracted: static baseline features, time-series dynamic features, and high-dimensional nonlinear features. For the extraction of static baseline features, the focus is on preoperatively collected non-time-series or quasi-static data, specifically including scores of various dimensions of neuropsychological scales, blood biomarker concentration values, genomic risk locus typing information, and structural morphological parameters such as whole-brain gray matter volume and cortical thickness extracted from resting-state functional magnetic resonance imaging. A multilayer perceptron is used as the encoder to map the above multi-source static data into a low-dimensional embedding space, and a two-layer fully connected network is used in conjunction with the ReLU activation function and... Dropout regularization generates a static baseline feature vector with fixed dimensions. For extracting temporal dynamic features, it processes continuously acquired intraoperative physiological time-series data, including time series of parameters such as cerebral oxygen saturation, blood pressure, heart rate, anesthesia depth index, and body temperature. A bidirectional long short-term memory network (Bi-LSTM) is used as the temporal encoder, taking the multi-parameter physiological vector at each time point as input. A forward LSTM layer captures the temporal dependencies from the start of surgery to the current time point, and a backward LSTM layer captures the reverse temporal dependencies from the current time point to the end of surgery. The hidden states in both directions are concatenated to form the temporal dynamic feature vector for each time step. This Bi-LSTM structure can effectively model the contextual information of intraoperative events, enabling the full representation of complex temporal patterns such as the duration of hypotension events and the cumulative effect of blood pressure fluctuations.

[0042] For the extraction of high-dimensional nonlinear features, an individualized brain functional connectivity network was constructed using preoperative resting-state functional magnetic resonance imaging data. Brain regions defined by automated anatomical atlases were used as network nodes, and the time-series correlation of blood oxygenation level-dependent signals between brain regions was calculated as the functional connectivity strength to construct a brain functional connectivity matrix. Based on this, a graph convolutional neural network was used to perform deep feature learning on the graph structure data. The functional connectivity information of each brain region and its neighboring brain regions was aggregated through multi-layer graph convolution operations to extract high-dimensional nonlinear features that reflect the topological characteristics of the brain functional network. These features characterize the patient's preoperative brain functional integration and separation capabilities, the integrity of key functional networks such as the default mode network, etc.

[0043] After extracting the three types of features, the process enters the dynamic fusion stage. This stage involves deep feature fusion based on temporal attention and cross-modal interaction mechanisms. First, the temporal dynamic feature sequence output by the Bi-LSTM is input into the temporal attention module. This module automatically learns the importance of different time stages in the prediction task by introducing attention weights along the time dimension. Specifically, the temporal dynamic features at each time step are linearly transformed, and then a normalized attention weight distribution is calculated using the Softmax function. The attention weights are then weighted and summed with the original temporal dynamic features to generate a temporally attention-enhanced temporal dynamic feature. The static baseline feature vector, the temporal dynamic feature vector enhanced by time attention, and the high-dimensional nonlinear feature vector extracted by the graph convolutional neural network are then input into the cross-modal interaction module. This module realizes deep interaction between multimodal features based on the multi-head attention mechanism. The features of the three modalities are mapped to the query, key, and value spaces, respectively. The interaction weight between any two modalities is calculated through multi-head attention. For example, the interaction between preoperative static features and intraoperative temporal features, the interaction between preoperative static features and brain network map features, and the interaction between intraoperative temporal features and brain network map features are calculated to capture the nonlinear correlation between different modalities.

[0044] The extracted temporal dynamic features are defined as follows: The high-dimensional nonlinear characteristics are The static baseline characteristics are The features of the three different modalities are mapped to the query through linear transformation. ,key ,value The independent subspaces are mapped as follows:

[0045]

[0046]

[0047]

[0048] in, , , This is a learnable weight matrix. To address the issue of inconsistent feature dimensions across different modalities, zero-padding is performed on shorter feature sequences and truncation is performed on longer feature sequences before mapping, resulting in... , , The mapped feature dimensions are uniformly aligned to the hidden layer dimensions of the model. After alignment, the attention weights between modalities are calculated using a scaled dot product attention mechanism:

[0049]

[0050] This enables information exchange and complementarity between different modalities under a unified dimension;

[0051] The interaction weights are calculated using a scaled dot product attention approach, which involves parallel computation of multiple attention heads, each focusing on a different modal interaction mode. The outputs of the multiple attention heads are concatenated and linearly transformed to generate enhanced features that incorporate cross-modal interaction information.

[0052] The three types of features enhanced by cross-modal interaction are concatenated and input into a tensor fusion layer. This fusion layer uses multi-layer nonlinear transformation, and through a fully connected network combined with batch normalization and nonlinear activation functions, it maps the concatenated high-dimensional features to a compact low-dimensional representation space, generating the final multimodal joint representation vector.

[0053] S3: Construct a dynamic prediction model based on long short-term memory network and Transformer architecture; use the multimodal joint representation vector as input and the postoperative cognitive impairment risk probability and cognitive function score change trajectory within a preset time window as output to train the dynamic prediction model;

[0054] In step S3, a multi-task dynamic prediction model is first constructed based on a Long Short-Term Memory (LSTM) network and a Transformer architecture. This model employs an encoder-decoder structure, using the multimodal joint representation vector sequence generated in step S2 as input and the postoperative cognitive impairment risk probability and cognitive function score change trajectory within a preset time window as output, thus achieving dynamic prediction of postoperative cognitive impairment. The overall model architecture includes three core components: a temporal encoding module, a multi-task decoding module, and a joint loss optimization module. The temporal encoding module uses a Transformer encoder as its basic architecture to handle the temporal dependencies of the multimodal joint representation vector. Let the input sequence be... ,in For the number of time steps, To define the dimension of the joint representation vector; firstly, temporal information is introduced through positional encoding, which uses a sine-cosine function:

[0055]

[0056]

[0057] in, This is represented as a position encoding function, where pos is the time position index. The dimension index is used; the positional encoding is added to the input features and then input into a multi-layer Transformer encoder layer; each encoder layer contains a multi-head self-attention mechanism and a feedforward neural network, and the calculation method of multi-head self-attention is as follows:

[0058]

[0059] The calculation for each attention head is as follows:

[0060]

[0061] By stacking multiple Transformer encoders, the model captures long-range dependencies between any time steps in the sequence and outputs encoded temporal feature representations. ;

[0062] The multi-task decoding module adopts a dual-branch parallel structure, including a risk probability prediction branch and a trajectory prediction branch. The two branches share the output features of the temporal coding module, but each uses an independent decoding network. The risk probability prediction branch outputs the probability of the patient developing postoperative cognitive impairment at multiple future time points; this branch supports the output features of the last layer of the temporal coding module. The summarized representation of the current state is input to a sequence decoder composed of a long short-term memory network; assuming the prediction time window contains... At each future time point, the LSTM decoder generates the hidden state step by step:

[0063]

[0064] in, This represents the hidden state of the LSTM decoder at the k-th future time point. The hidden state at each time step is the prediction result from the previous time step (using the true label during training and the predicted value during inference); The input is fed into a fully connected layer with a Sigmoid activation function, and the output is the risk probability at that time point. The specific calculation method is as follows:

[0065]

[0066] in, Indicates the first The predicted probability of postoperative cognitive impairment at a future point in time. and These are learnable parameters; the trajectory prediction branch outputs a predicted curve of the patient's cognitive function score at multiple future time points; this branch is also based on the output features of the temporal coding module, employing an independent LSTM decoder structure, but the output layer uses a linear activation function to predict continuous cognitive function score values; let the cognitive function score sequence be... The predicted value for each time point is:

[0067]

[0068] in, For the trajectory prediction branch of the LSTM decoder in the 1st... The implicit state of the step, and This is a learnable parameter. This branch can output a continuous curve of cognitive function score changes, reflecting the dynamic trend of postoperative cognitive function recovery or deterioration in patients;

[0069] During the model training phase, a joint loss function is defined to simultaneously optimize the two prediction tasks; the risk probability prediction branch adopts the binary classification cross-entropy loss function:

[0070]

[0071] in, Represented as the cross-entropy loss function, For the sample size, For the first The sample at the th A true label at a specific point in time;

[0072] The trajectory prediction branch uses the mean squared error loss function, and the specific calculation method is as follows:

[0073]

[0074] in, Represented as the mean squared error loss function, For the first The sample at the th Real cognitive function scores at each time point Represented as the first The sample at the th Predicted cognitive function scores at each time point;

[0075] To prevent imbalanced optimization caused by the difference in magnitude between the two loss functions in multi-task learning, a dynamic weighting mechanism is introduced. The joint loss function is calculated as follows:

[0076]

[0077] in, Represented as a loss function, and These are task weight coefficients, dynamically adjusted based on validation set performance or automatically learned using an uncertainty-weighted method. The regularization coefficient is . Represented as all learnable parameters of the model. This is the L2 regularization term, used to prevent overfitting;

[0078] During training, the Adam optimizer is used for parameter updates, and the learning rate is dynamically adjusted using a cosine annealing strategy. The initial learning rate is set to... The weight decay coefficient is The model training uses mini-batch gradient descent, with the batch size set to 32 or 64 depending on the dataset size. The training epochs are set to 200, and an early stopping mechanism is used. Training stops when the validation set loss no longer decreases for 20 consecutive epochs, and the model parameters with the best performance on the validation set are saved.

[0079] S4: Input the real-time collected multimodal patient data into the trained dynamic prediction model, output the dynamic prediction probability and risk level of postoperative cognitive dysfunction, and update the prediction results in real time according to the inflow of new data.

[0080] In step S4, the trained dynamic prediction model is first deployed to the clinical prediction system. This system connects with the hospital information system, anesthesia monitoring equipment, and postoperative follow-up record platform in real time. When a new patient enters the perioperative management process, the system automatically collects real-time multimodal data, including preoperative assessment data, intraoperative continuous physiological monitoring data, and postoperative short-term follow-up data. Following the data preprocessing and feature extraction processes described in S1 and S2, the system transforms the real-time collected raw data into a multimodal joint representation vector acceptable to the model. This representation vector is then input into the trained dynamic prediction model. The model calculates through forward propagation and outputs the dynamic prediction probability and risk level of postoperative cognitive dysfunction for the patient within a preset future time window.

[0081] The model output includes predictions in three dimensions: the first dimension is the temporal risk probability prediction, which is the probability value that the patient will experience postoperative cognitive impairment at multiple time points such as postoperative day 1, day 3, day 7, and day 14. The data is visualized as a probability curve, with the horizontal axis representing postoperative time points and the vertical axis representing predicted probability values, intuitively showing the dynamic trend of risk over time. The second dimension is the prediction of cognitive function trajectory, with the model simultaneously outputting predicted cognitive function scores for the patient at the aforementioned future time points. This generates a cognitive function change curve, reflecting the expected recovery trajectory or deterioration trend of the patient's postoperative cognitive function, providing clinicians with continuous quantitative references in addition to binary risk assessment; the third dimension is risk level classification, where the system automatically performs graded early warning based on predicted probability values, setting three risk thresholds: when When the risk level is determined to be low, the system displays a green icon; when... When the risk level is determined to be medium, the system displays a yellow indicator and prompts clinicians to monitor the patient's condition; when When a high-risk level is identified, the system displays it in red and triggers an active warning. The warning threshold is dynamically adjusted according to the actual clinical application scenario, and the threshold settings of different medical institutions can be flexibly modified through the system configuration interface.

[0082] Regarding the dynamic update mechanism, a sliding window strategy is adopted to achieve real-time updates of prediction results. Whenever new multimodal data flows into the system, whether it is physiological monitoring data updated every second during surgery or scale assessment data recorded every hour after surgery, the system automatically triggers an incremental prediction update process. Specifically, the system maintains a fixed-length time window. When new data arrives, it is appended to the end of the current patient's data sequence, while the oldest data point at the beginning of the window is removed to keep the amount of data within the window constant. Based on the updated data sequence, the feature extraction and dynamic fusion process of S2 is re-executed to generate an updated multimodal joint representation vector, which is then input into the model for forward computation to obtain the updated prediction results. For continuous high-frequency physiological data during surgery, the system adopts an adaptive sampling strategy, reducing the update frequency to reduce computational resource consumption when the patient's condition is stable, and automatically increasing the update frequency when abnormal fluctuations in key physiological parameters are detected, thereby achieving close tracking of high-risk conditions.

[0083] Finally, the above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A method for dynamic prediction of postoperative cognitive impairment based on multimodal data fusion, characterized in that, include: S1: Acquire patients' preoperative multimodal data, intraoperative dynamic physiological data, and postoperative short-term assessment data; The multimodal data is time-aligned, cleaned, and standardized to construct a standardized time-series dataset; S2: Extract static baseline features, temporal dynamic features, and high-dimensional nonlinear features from the standardized time-series dataset, respectively; wherein, the temporal dynamic features are extracted using a bidirectional long short-term memory network to capture the bidirectional temporal dependencies from the start of the surgery to the current time point and from the current time to the end of the surgery; based on the time attention mechanism and the cross-modal interaction mechanism, the extracted features are dynamically fused to generate a multimodal joint representation vector with spatiotemporal correlation; S3: Construct a dynamic prediction model based on long short-term memory network and Transformer architecture; use the multimodal joint representation vector as input and the postoperative cognitive impairment risk probability and cognitive function score change trajectory within a preset time window as output to train the dynamic prediction model; S4: Input the real-time collected patient multimodal data into the trained dynamic prediction model, and output the dynamic prediction probability and risk level of postoperative cognitive dysfunction; the dynamic prediction model adopts an adaptive sampling strategy to process the real-time collected patient multimodal data: reduce the data update frequency when the patient's physiological state is stable, and automatically increase the update frequency of real-time multimodal data when abnormal fluctuations in key physiological parameters are detected, and re-perform feature extraction based on the updated data sequence to update the prediction results in real time.

2. The method for dynamic prediction of postoperative cognitive dysfunction based on multimodal data fusion according to claim 1, characterized in that: In S1, the preoperative multimodal data includes neuropsychological scale assessment data, resting-state functional magnetic resonance imaging data, electroencephalogram data, blood biomarker data, and genomics data; the intraoperative dynamic physiological data includes cerebral oxygen saturation, blood pressure, heart rate, anesthesia depth index, body temperature, end-tidal carbon dioxide partial pressure, and blood oxygen saturation; and the postoperative short-term assessment data includes delirium assessment scale score, visual analog scale score, and mini mental status test score.

3. The method for dynamic prediction of postoperative cognitive dysfunction based on multimodal data fusion according to claim 1, characterized in that: In step S1, data cleaning and standardization include: performing head motion correction, temporal layer correction, spatial registration, spatial smoothing, and bandpass filtering on functional magnetic resonance imaging (fMRI) data; performing baseline drift removal, artifact removal, power frequency filtering, and rereference processing on electroencephalogram (EEG) data; processing missing values ​​in physiological time series data using interpolation methods and applying low-pass filtering to remove high-frequency noise; detecting and correcting outliers in scale scores and biomarker data, and mapping the data to a unified dimension using Z-score standardization or max-min normalization methods; and aligning all modal data in chronological order using the patient's unique identifier and timestamp as indexes to construct a standardized time series dataset.

4. The method for dynamic prediction of postoperative cognitive dysfunction based on multimodal data fusion according to claim 1, characterized in that: In S2, the method for extracting static baseline features is as follows: a multilayer perceptron is used as an encoder to map multi-source static data to a low-dimensional embedding space, and a fixed-dimensional static baseline feature vector is generated by using a two-layer fully connected network in conjunction with the ReLU activation function and Dropout regularization.

5. The method for dynamic prediction of postoperative cognitive dysfunction based on multimodal data fusion according to claim 1, characterized in that: In S2, the method for extracting temporal dynamic features is as follows: a bidirectional long short-term memory network is used as a temporal encoder, the multi-parameter physiological vector at each time point is used as input, and the forward and backward temporal dependencies are captured by forward LSTM layers and backward LSTM layers respectively. The hidden states in the two directions are concatenated to form the temporal dynamic feature vector of each time step.

6. The method for dynamic prediction of postoperative cognitive dysfunction based on multimodal data fusion according to claim 1, characterized in that: In S2, the method for extracting high-dimensional nonlinear features is as follows: an individualized brain functional connectivity network is constructed using preoperative resting-state functional magnetic resonance imaging data, and a graph convolutional neural network is used to perform deep feature learning on the graph structure data. The functional connectivity information of each brain region and its neighboring brain regions is aggregated through multi-layer graph convolution operations to extract high-dimensional nonlinear features that reflect the topological characteristics of the brain functional network.

7. The method for dynamic prediction of postoperative cognitive dysfunction based on multimodal data fusion according to claim 1, characterized in that: In S2, the dynamic fusion stage includes: inputting the temporal dynamic feature sequence output by Bi-LSTM into the temporal attention module, calculating the normalized attention weight distribution through the Softmax function, and generating a temporal dynamic feature vector enhanced by temporal attention; inputting the static baseline feature vector, the temporal dynamic feature vector enhanced by temporal attention, and the high-dimensional nonlinear feature vector extracted by the graph convolutional neural network into the cross-modal interaction module, which realizes deep interaction between multimodal features based on the multi-head attention mechanism; concatenating the three types of features after interaction enhancement and inputting them into the tensor fusion layer to generate a multimodal joint representation vector.

8. The method for dynamic prediction of postoperative cognitive dysfunction based on multimodal data fusion according to claim 1, characterized in that: In S3, the dynamic prediction model includes a temporal encoding module, a multi-task decoding module, and a joint loss optimization module. The temporal encoding module uses a Transformer encoder to handle the temporal dependencies of the multimodal joint representation vectors. The multi-task decoding module adopts a dual-branch parallel structure, including a risk probability prediction branch and a trajectory prediction branch, which respectively output the risk probability of postoperative cognitive impairment and the trajectory of changes in cognitive function scores. The joint loss optimization module uses a dynamically weighted joint loss function to optimize the model.

9. The method for dynamic prediction of postoperative cognitive dysfunction based on multimodal data fusion according to claim 1, characterized in that: In step S4, a sliding window strategy is used to update the prediction results in real time. The system maintains a fixed-length time window. When new data arrives, the data is appended to the end of the current patient's data sequence, while the oldest data point at the beginning of the window is removed to keep the amount of data in the window constant. Based on the updated data sequence, the feature extraction and dynamic fusion process is re-executed to obtain the updated prediction results.

Images