Real-time warning method and device for cardiovascular and cerebrovascular events during the perioperative period of non-cardiac surgery

Abstract

The present application relates to the technical field of medical data processing, and proposes a method and device for real-time early warning of cardiovascular and cerebrovascular events during the perioperative period of non-cardiac surgery. The method comprises: obtaining intraoperative data of the target surgical patient during the surgical procedure of the target surgical patient; inputting the intraoperative data into a trained time series large model for time series feature extraction and processing to obtain a time series feature representation of the target surgical patient; obtaining a joint feature representation of the target surgical patient through a pre-constructed multimodal data atlas; wherein the multimodal data atlas is constructed based on the multimodal data of multiple historical surgical patients and the preoperative data of the target surgical patient, and the joint feature representation of the target surgical patient is used to characterize the health status of the target surgical patient; based on the joint feature representation and time series feature representation of the target surgical patient, the risk of cardiovascular and cerebrovascular events in the target surgical patient is predicted. The use of this method can improve the accuracy of cardiovascular and cerebrovascular event risk prediction for surgical patients.

Description

Technical Field

[0001] The present application relates to the field of medical data processing technology, and in particular to a method, device, electronic device and computer program product for real-time early warning of cardiovascular and cerebrovascular events during the perioperative period of non-cardiac surgery. Background Art

[0002] Patients undergoing non-cardiac surgery may experience various cardiovascular and cerebrovascular events during the perioperative period, such as myocardial infarction, stroke, and arrhythmia. These events pose a serious threat to the patient's life and health. Therefore, how to predict the risk of perioperative cardiovascular and cerebrovascular events in patients so that doctors can take timely and effective intervention measures has become a key issue facing those skilled in the art.

[0003] To address these issues, existing technologies have proposed cardiovascular event early warning methods based on deep learning models. These methods input multimodal data from surgical patients, both preoperatively and intraoperatively, into a cardiovascular event early warning model for risk prediction, resulting in corresponding risk warning results. However, given the heterogeneous nature of preoperative and intraoperative data, existing early warning models typically employ simple feature concatenation or independent data processing branches, lacking a unified representation framework. This results in the deep correlations between different modal data not being fully explored, reducing the accuracy of cardiovascular event risk prediction. Summary of the Invention

[0004] In view of this, the embodiments of the present application provide a real-time warning method, device, electronic device and computer program product for cardiovascular and cerebrovascular events during the perioperative period of non-cardiac surgery, which can improve the accuracy of risk prediction of cardiovascular and cerebrovascular events in surgical patients.

[0005] A first aspect of an embodiment of the present application provides a real-time early warning method for cardiovascular and cerebrovascular events during perioperative non-cardiac surgery, comprising:

[0006] During the operation of the target surgical patient, obtaining intraoperative data of the target surgical patient;

[0007] Input the intraoperative data into the trained time series large model for time series feature extraction and processing to obtain the time series feature representation of the target surgical patient;

[0008] Obtaining a joint feature representation of a target surgical patient through a pre-constructed multimodal data atlas; wherein the multimodal data atlas is constructed based on multimodal data of multiple historical surgical patients and preoperative data of the target surgical patient, and the joint feature representation of the target surgical patient is used to characterize the health status of the target surgical patient;

[0009] Based on the joint feature representation and time series feature representation of the target surgical patients, the risk of cardiovascular and cerebrovascular events in the target surgical patients is predicted.

[0010] The technical solution of the embodiment of the present application constructs a multimodal data atlas based on the multimodal data of multiple historical surgical patients and the preoperative data of the target surgical patient in advance; when the target surgical patient undergoes surgery, the intraoperative data of the target surgical patient will be obtained in real time, and then the intraoperative data will be input into the trained time series large model for time series feature extraction processing to obtain the time series feature representation of the target surgical patient, and a joint feature representation that can be used to characterize the health status of the target surgical patient is obtained through the multimodal data atlas; finally, based on the joint feature representation and the time series feature representation, the risk of cardiovascular and cerebrovascular events in the target surgical patient is predicted. The above process uses the multimodal data atlas to fuse heterogeneous preoperative data and intraoperative data, realizes deep semantic fusion and dynamic association modeling of multimodal data, can fully explore the correlation between cross-modal data, solve the problem of data feature fragmentation, and thus improve the accuracy of risk prediction of cardiovascular and cerebrovascular events in surgical patients.

[0011] In one implementation of the embodiment of the present application, the process of constructing a multimodal data graph includes:

[0012] Perform feature extraction and encoding on the preoperative data of the target surgical patients to obtain multimodal features;

[0013] Performing spatiotemporal correlation and synchronous alignment processing on multimodal features;

[0014] Based on the multimodal features processed by spatiotemporal correlation synchronization and alignment, the node representation of the target surgical patient in each different modality is constructed;

[0015] Determining association weights of the node representations of the target surgical patient in each of the different modalities based on similarities between the node representations of the multiple historical surgical patients in each of the different modalities and the node representations of the target surgical patient in each of the different modalities; wherein the node representations of the multiple historical surgical patients in each of the different modalities are constructed based on multimodal data of the multiple historical surgical patients;

[0016] The node representations of the target surgical patient in different modalities are fused according to the corresponding association weights to obtain the joint feature representation of the target surgical patient.

[0017] In one implementation of the embodiment of the present application, predicting the risk of cardiovascular and cerebrovascular events in a target surgical patient based on the joint feature representation and the time series feature representation of the target surgical patient includes:

[0018] Generate a similar patient virtual relationship graph based on the joint feature representation of the target surgical patient and the joint feature representation of multiple historical surgical patients. The similar patient virtual relationship graph is used to represent the similarity between the target surgical patient and multiple historical surgical patients.

[0019] Based on the virtual relationship graph of similar patients, a similar patient set of the target surgical patient is constructed;

[0020] Determine an enhanced feature representation of the target surgical patient based on the similarity weight between each patient in the similar patient set and the target surgical patient, and the node representation of each patient in the similar patient set under different modalities;

[0021] Based on enhanced feature representation and time series feature representation, the risk of cardiovascular and cerebrovascular events in target surgical patients is predicted.

[0022] In one implementation of the embodiment of the present application, a virtual relationship graph of similar patients is generated based on the joint feature representation of the target surgical patient and the joint feature representations of multiple historical surgical patients, including:

[0023] Calculate the similarity weights between the target surgical patient and the multiple historical surgical patients based on the joint feature representation of the target surgical patient and the joint feature representation of multiple historical surgical patients;

[0024] The joint feature representation of the target surgical patient and the joint feature representation of multiple historical surgical patients are used as nodes of the similar patient virtual relationship graph. The edges between the nodes of the similar patient virtual relationship graph are constructed according to the similarity weights between the target surgical patient and multiple historical surgical patients, and the edges whose corresponding similarity weights are less than the set threshold are deleted to obtain the similar patient virtual relationship graph.

[0025] In one implementation of the embodiment of the present application, predicting the risk of cardiovascular and cerebrovascular events in a target surgical patient based on the enhanced feature representation and the time series feature representation includes:

[0026] The enhanced feature representation and the temporal feature representation are input into a deep learning model that has been trained to assess spatiotemporal risk, and the spatiotemporal risk representation of the target surgical patient at the current moment is obtained.

[0027] Based on the spatiotemporal risk representation at the current moment, the spatiotemporal risk representation of the target surgical patient at multiple moments in the future is predicted;

[0028] Based on the spatiotemporal risk characterization of the target surgical patient at multiple moments in the future, the risk level of the target surgical patient for cardiovascular and cerebrovascular events at multiple moments in the future is determined.

[0029] In one implementation of the embodiment of the present application, after generating a similar patient virtual relationship graph based on the joint feature representation of the target surgical patient and the joint feature representations of multiple historical surgical patients, the method further includes:

[0030] When intraoperative data changes, the virtual relationship graph of similar patients is updated according to the updated temporal feature representation.

[0031] In one implementation of the embodiment of the present application, intraoperative data is input into a trained time series large model for time series feature extraction processing to obtain a time series feature representation of the target surgical patient, including:

[0032] Based on distributed stream computing technology and using a sliding window strategy, the continuously flowing intraoperative data is processed to obtain the physiological signal data stream of the target surgical patient;

[0033] The physiological signal data stream is input into the time series large model for time series feature extraction and processing to obtain the time series feature representation of the target surgical patient.

[0034] A second aspect of the embodiments of the present application provides a real-time early warning device for cardiovascular and cerebrovascular events during perioperative period of non-cardiac surgery, comprising:

[0035] An intraoperative data acquisition module is used to acquire intraoperative data of a target surgical patient during the surgical procedure of the target surgical patient;

[0036] The time series feature extraction module is used to input intraoperative data into the trained time series large model for time series feature extraction processing to obtain the time series feature representation of the target surgical patient;

[0037] A joint feature acquisition module is used to obtain a joint feature representation of a target surgical patient using a pre-constructed multimodal data atlas. The multimodal data atlas is constructed based on multimodal data from multiple historical surgical patients and pre-operative data of the target surgical patient. The joint feature representation of the target surgical patient is used to characterize the health status of the target surgical patient.

[0038] The risk prediction module is used to predict the risk of cardiovascular and cerebrovascular events in target surgical patients based on the joint feature representation and time series feature representation of the target surgical patients.

[0039] A third aspect of an embodiment of the present application provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, the method for real-time early warning of perioperative cardiovascular and cerebrovascular events during non-cardiac surgery provided in the first aspect of an embodiment of the present application is implemented.

[0040] The fourth aspect of the embodiments of the present application provides a computer program product. When the computer program product runs on an electronic device, the electronic device executes the real-time warning method for perioperative cardiovascular and cerebrovascular events in non-cardiac surgery provided by the first aspect of the embodiments of the present application.

[0041] The fifth aspect of the embodiment of the present application provides a computer-readable storage medium, which stores a computer program. When the computer program is executed by a processor, it implements the real-time warning method for perioperative cardiovascular and cerebrovascular events in non-cardiac surgery as provided in the first aspect of the embodiment of the present application.

[0042] It can be understood that the beneficial effects of the second to fifth aspects mentioned above can be found in the relevant description of the first aspect mentioned above, and will not be repeated here. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] Figure 1 This is a flow chart of a real-time early warning method for perioperative cardiovascular and cerebrovascular events in non-cardiac surgery provided by an embodiment of the present application;

[0044] Figure 2 This is a schematic diagram of an overall technical framework of a real-time early warning method for perioperative cardiovascular and cerebrovascular events in non-cardiac surgery provided by an embodiment of the present application;

[0045] Figure 3 yes Figure 2 Schematic diagram of the working principle of the multimodal graph fusion representation module in the shown framework;

[0046] Figure 4 This is a structural diagram of a real-time early warning device for perioperative cardiovascular and cerebrovascular events in non-cardiac surgery provided by an embodiment of the present application;

[0047] Figure 5 This is a schematic diagram of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0048] In the following description, for the purpose of illustration rather than limitation, specific details such as specific system structures and technologies are provided to provide a thorough understanding of the embodiments of the present application. However, it should be clear to those skilled in the art that the present application can also be implemented in other embodiments without these specific details. In other cases, detailed descriptions of well-known systems, devices, circuits, and methods are omitted to avoid obstructing the description of the present application with unnecessary details. In addition, in the description of the present application specification and the appended claims, the terms "first," "second," "third," etc. are only used to distinguish descriptions and are not to be understood as indicating or implying relative importance.

[0049] Perioperative cardiovascular and cerebrovascular complications are a major cause of death or disability in surgical patients. Early warning of these events relies on a comprehensive assessment of multimodal data, including the patient's electronic health record, preoperative electrocardiogram (ECG), medical imaging, and intraoperative physiological monitoring. Existing early warning models typically use simple feature concatenation or independent data processing branches when processing multimodal data. These models fail to fully exploit the deep connections between data from different modalities, impacting the comprehensiveness and accuracy of cardiovascular and cerebrovascular event risk prediction.

[0050] To address the above technical issues, the embodiments of the present application propose a real-time warning method, device, electronic device, and computer program product for perioperative cardiovascular and cerebrovascular events during non-cardiac surgery. By introducing multimodal data graphs, deep semantic fusion and dynamic association modeling can be achieved, fully exploring the correlation between cross-modal data, thereby effectively improving the comprehensiveness and accuracy of cardiovascular and cerebrovascular event risk prediction. For more specific technical implementation details of the embodiments of the present application, please refer to the various method embodiments described below.

[0051] It should be understood that the execution subjects of the various method embodiments proposed in the present application can be various types of electronic devices, such as mobile phones, tablet computers, desktop computers, wearable devices, medical devices, augmented reality (AR) / virtual reality (VR) devices, laptops, ultra-mobile personal computers (UMPCs), netbooks, personal digital assistants (PDAs), large-screen TVs, etc. The embodiments of the present application do not impose any restrictions on the specific types of the electronic devices.

[0052] See also Figure 1 , shows a real-time early warning method for cardiovascular and cerebrovascular events during the perioperative period of non-cardiac surgery provided by an embodiment of the present application, comprising:

[0053] 101. During the operation of the target surgical patient, obtain the intraoperative data of the target surgical patient;

[0054] Overall, the technical solution of the embodiment of this application proposes a real-time intraoperative warning model based on machine learning. By integrating the multimodal data features of surgical patients, a self-evolving virtual relationship graph of similar patients is constructed, and pre-training technology is combined to achieve personalized and accurate warning of cardiovascular and cerebrovascular events. The core of this technical solution lies in building a full-process technical framework of "multimodal data fusion - low-latency computing - pre-training drive - personalized dynamic modeling - spatiotemporal risk deduction" to achieve accurate risk prediction of cardiovascular and cerebrovascular events.

[0055] As an example, Figure 2 This is a schematic diagram of the overall technical framework of the real-time early warning method for perioperative cardiovascular and cerebrovascular events in non-cardiac surgery provided in an embodiment of the present application. Figure 2 The technical framework shown includes the following five functional modules: a multimodal graph fusion representation module, a low-latency distributed stream processing module, a lightweight reasoning module for large time series models, a self-evolving similar patient matching module, and a spatiotemporal risk deduction and warning module. The intraoperative real-time warning model is composed of these five functional modules. Each functional module is briefly described below.

[0056] Multimodal graph fusion representation module: In order to address the problem that multimodal data has different formats and types and is difficult to efficiently fuse and comprehensively represent, the multimodal graph fusion representation module can effectively fuse multimodal data by constructing a unified fusion framework to extract cross-modal correlation features and solve the problem of data feature fragmentation. In general, the multimodal graph fusion representation module first extracts and encodes the features of each modality through the encoder, then performs spatiotemporal synchronous correlation alignment, calculates the correlation weights between different data modalities, and finally establishes highly structured patient feature nodes. These patient feature nodes together constitute a multimodal data graph. By using the multimodal graph fusion representation module, deep semantic fusion and dynamic correlation modeling of multimodal data can be achieved, and the important features of each modality data can be fully mined to provide a highly robust joint representation for intraoperative risk warning.

[0057] Low-Latency Distributed Stream Processing Module: Intraoperative data is characterized by large volumes and the need for real-time processing. Traditional early warning models typically use fixed time windows or offline batch processing to analyze intraoperative physiological monitoring signals. This results in low computational efficiency, insufficient dynamic adaptability, and significant lag in processing intraoperative physiological monitoring signals. To address this issue and meet the requirements for both real-time and efficient early warning, the low-latency distributed stream processing module utilizes a collaborative design combining distributed stream computing technology with an edge-first computing framework. By leveraging this distributed computing architecture and stream processing paradigm, it reduces computational latency while maintaining prediction accuracy, meeting the clinical need for real-time intraoperative risk response and enabling real-time processing and transmission of intraoperative physiological monitoring data. Specifically, the low-latency distributed stream processing module uses a sliding window strategy to dynamically partition the continuously flowing multi-channel physiological data. It can also combine multiple collaborative strategies to accurately capture changes in the patient's physiological status during surgery. Furthermore, an edge-first computing architecture is introduced to prioritize computing tasks on edge devices, dynamically allocating them based on the device's real-time computing power. Combined with intelligent task scheduling, this significantly reduces computational latency and improves computational efficiency.

[0058] Lightweight Inference Module for Large Time Series Models: To address the complex nature of intraoperative physiological monitoring signals and the need for real-time processing, the Lightweight Inference Module for Large Time Series Models leverages an open-source pre-trained large time series model. Through targeted fine-tuning and lightweighting techniques, it achieves model parameter compression and computational optimization, enabling efficient deployment of large time series models on edge devices and supporting real-time inference of high-frequency intraoperative physiological data. Specifically, the Lightweight Inference Module for Large Time Series Models leverages the pre-trained large time series model to extract high-quality features from intraoperative physiological monitoring signals and ensures real-time processing capabilities through lightweight optimization techniques. This module, through medical domain migration and fine-tuning, adapts the large time series model to the characteristics of intraoperative monitoring data, enhancing its ability to identify key risk patterns. Furthermore, by combining knowledge distillation, model structure optimization, and computational acceleration, it significantly reduces model computational complexity, ensuring feature extraction accuracy while meeting the low latency and high efficiency requirements of the intraoperative environment, providing accurate and reliable feature input for intraoperative risk warning.

[0059] Self-Evolving Similar Patient Matching Module: Surgical patients often exhibit significant individual differences. Each patient's physiology, genetic background, age, gender, medical history, lifestyle, and other factors vary widely, and these differences can significantly impact intraoperative responses and the risk of cardiovascular and cerebrovascular events. For example, elderly and younger patients exhibit significant differences in cardiovascular tolerance, metabolic capacity, and immune response. Patients with a history of chronic illness or multiple chronic conditions may also have different surgical risks and intraoperative responses. Some patients, despite having no significant medical history, may still experience sudden and severe cardiovascular and cerebrovascular events due to their physiological state, anesthetic response, or the specific type of surgery. Factors such as a patient's psychological state, drug metabolism, weight, and even the type of surgery performed can all contribute to significant individual differences in the mechanisms of cardiovascular and cerebrovascular events. To address this issue, the Self-Evolving Similar Patient Matching Module builds a self-evolving virtual relationship graph of similar patients based on multimodal data features of surgical patients. This module dynamically matches similar patient groups and, through transfer learning, personalizes model parameters to ensure that the predicted surgical patient is as similar as possible to the existing training population, thereby improving the accuracy of cardiovascular and cerebrovascular event warnings. Specifically, the self-evolving similar patient matching module first uses pre-training to develop a similarity measurement model using existing large-scale patient data. This model then combines the patient feature nodes output by the multimodal graph fusion representation module to construct a dynamically updated self-evolving virtual relationship graph of similar patients. Furthermore, during real-time early warning, the structure of the virtual relationship graph of similar patients is adaptively adjusted based on the calculated similarity of newly input data, enabling personalized patient group matching and knowledge transfer for intraoperative risk prediction.

[0060] Spatiotemporal risk deduction and early warning module: Traditional early warning models rely on static thresholds or single-time point risk assessments, and are unable to model the spatiotemporal evolution of intraoperative risks in real time. They can only output current or short-term risk probabilities, lack multi-step prediction capabilities, and have difficulty locating key intraoperative risk triggering nodes. To address this issue, the spatiotemporal risk deduction and early warning module analyzes the spatiotemporal interaction mechanism of intraoperative risks, generates future risk evolution trajectories, and triggers hierarchical early warnings based on dynamic intervals. Specifically, the spatiotemporal risk deduction and early warning module is based on the spatiotemporal Transformer architecture, integrating the spatial correlation in the self-evolving virtual relationship graph of similar patients with the temporal evolution characteristics of real-time intraoperative physiological monitoring signals, exploring the spatiotemporal interaction between individual characteristics of surgical patients and group risk patterns, dynamically analyzing the spatiotemporal distribution characteristics of key cardiovascular and cerebrovascular events, generating future risk evolution trajectories, and triggering graded early warning signals based on confidence intervals.

[0061] For a more detailed description of the working principles of the above five functional modules, please refer to the description below.

[0062] Before implementing the real-time early warning method for perioperative cardiovascular and cerebrovascular events in non-cardiac surgery provided by the embodiment of the present application, it is necessary to pre-construct a multimodal data map. The multimodal data map can be constructed based on the multimodal data of a large number of historical surgical patients and the preoperative data of the target surgical patients. Among them, the multimodal data includes two categories: preoperative data and intraoperative data. The preoperative data includes but is not limited to electronic health records, preoperative electrocardiograms and various medical images. The intraoperative data is real-time time-series physiological monitoring data, such as blood pressure, heart rate and blood oxygen. Historical surgical patients refer to patients who have completed the operation and their preoperative data and intraoperative data have been collected; target surgical patients refer to patients who currently need to undergo surgery and realize early warning of cardiovascular and cerebrovascular events. Their preoperative data have been collected, but the intraoperative data needs to be collected in real time during the operation. The specific construction method of the multimodal data map is introduced below.

[0063] In one implementation of the embodiment of the present application, the process of constructing a multimodal data graph includes:

[0064] (1) Feature extraction and encoding of the preoperative data of the target surgical patients to obtain multimodal features;

[0065] (2) Performing spatiotemporal correlation and synchronous alignment processing on multimodal features;

[0066] (3) Based on the multimodal features processed by spatiotemporal correlation synchronization, the node representation of the target surgical patient in each different modality is constructed;

[0067] (4) Determining the association weights of the node representations of the target surgical patient in each of the different modalities based on the similarities between the node representations of the multiple historical surgical patients in each of the different modalities and the node representations of the target surgical patient in each of the different modalities; wherein the node representations of the multiple historical surgical patients in each of the different modalities are constructed based on the multimodal data of the multiple historical surgical patients;

[0068] (5) The node representations of the target surgical patient in different modalities are fused according to the corresponding association weights to obtain the joint feature representation of the target surgical patient.

[0069] The multimodal data graph is composed of multiple high-dimensional structured feature nodes, each of which corresponds to a surgical patient. It can be used to capture the complex health status of the corresponding surgical patient during the perioperative period and can be called the joint feature representation of the corresponding surgical patient. The construction of the multimodal data graph can be completed by the multimodal graph fusion representation module mentioned above. As an example, Figure 3 yes Figure 2 Schematic diagram of the working principle of the multimodal graph fusion representation module in the framework shown. Figure 3In the process, the surgical patient's multimodal data (including preoperative electronic health records, electrocardiograms, and medical imaging, as well as intraoperative physiological monitoring signals) is first input into a multimodal encoder for feature extraction and encoding. During the early warning model training phase, the model is fed with the patient's complete preoperative and intraoperative data. During the intraoperative real-time early warning phase, which is the early warning model's use phase, the model is fed with the patient's preoperative data in advance, while the intraoperative data needs to be collected in real time during the operation. Using a deep learning-based multimodal encoder, the surgical patient's multimodal data can be converted into a unified feature representation, thereby obtaining the corresponding multimodal features. For example, after inputting data from four modalities into the multimodal encoder for feature extraction and encoding, the corresponding feature representations for each of the four modal data can be obtained. These feature representations constitute the multimodal features. Considering that various intraoperative time-series physiological monitoring signals (such as heart rate, blood oxygen saturation, and respiratory rate) often come from different monitoring devices, these signals may have different sampling frequencies and sampling times, resulting in data from different modalities being misaligned in the temporal dimension. In the spatial dimension, the anatomical structures in medical images also need to be associated with the intraoperative monitoring sites. Therefore, it is necessary to perform spatiotemporal correlation synchronization and alignment on multimodal features. Specifically, interpolation techniques can be used to fill or smooth the time-series data and downsample the high-frequency data to avoid information loss or noise interference caused by time misalignment. Coordinate transformation can be used to establish the association between the image anatomical structures and the intraoperative monitoring sites. Subsequently, based on the multimodal features processed after spatiotemporal correlation synchronization, node representations of the surgical patient in each different modality are constructed. By calculating the association weights between the different modal data, a weighted atlas of the multimodal data is established, thereby obtaining the aforementioned multimodal data atlas. In order to ensure the effective fusion of different modal data, the embodiment of the present application adopts weighted graph fusion technology to convert the data into a graph structure and use the similarity between graph nodes for weighted fusion to achieve more accurate risk prediction. In multimodal features, the feature data of each modality can be used Indicates that the encoder Obtained The surgical patients Nodes under the mode indicate that they are available It can be understood that, assuming that there are M modes in the multimodal features, each surgical patient has M node representations, that is, M feature nodes. In this way, N surgical patients have a total of N*M feature nodes, which constitute the node set of the graph structure. The graph structure can be expressed by the formula Indicates that, Represents the graph structure, Represents a node set, which contains the characteristic nodes of all modes, The edge of the node is used to measure the similarity between modalities. Assume that the target surgical patient is patient , and its node representations in different modes are: 、 … … , the edge weights between feature nodes can be used to represent the similarity of feature nodes, which can be determined by calculating the Euclidean distance or cosine similarity between feature nodes. For example, using the formula Patients can be calculated In the Feature nodes and patients under modality In the Edge weights between feature nodes under modality , where Sim(⋅) is the similarity metric function and M represents the number of data modalities. The associated weight of each feature node It can be dynamically adjusted through its relationship with adjacent nodes. , where N represents the number of patients. The associated weight of each feature node Afterwards, the features of different modal nodes are fused according to the corresponding association weights by weighted atlas fusion to obtain the patient Joint feature representation of , the available formula Representation. The fused joint feature representation Patients High-dimensional structured feature nodes can be used to characterize patients The high-dimensional structured feature nodes of all patients together form a multimodal data graph. It can be seen that by querying this multimodal data graph, the joint feature representation of each surgical patient can be obtained, and the complex health status of each surgical patient can be evaluated.

[0070] Before the target surgical patient undergoes surgery, a multimodal data atlas is constructed in the above manner. Subsequently, during the target surgical patient's surgery, the target surgical patient's intraoperative data is obtained in real time. The target surgical patient's time series feature representation is extracted based on the intraoperative data, and the target surgical patient's joint feature representation is obtained through the multimodal data atlas. Finally, the time series feature representation and the joint feature representation are used to complete the accurate risk prediction of cardiovascular and cerebrovascular events.

[0071] 102. Input the intraoperative data into the trained time series large model for time series feature extraction and processing to obtain the time series feature representation of the target surgical patient;

[0072] The intraoperative data of the target surgical patients obtained can first be input into the above-mentioned low-latency distributed stream processing module for processing. Through the distributed computing architecture and stream processing paradigm, this module can ensure that when a large amount of continuous data flows in, it can maintain low-latency and high-throughput data processing energy, thereby meeting the requirements of real-time and efficient early warning.

[0073] During surgery, intraoperative data primarily includes various physiological monitoring signals, such as heart rate, blood oxygen levels, and blood pressure. This data is continuously streamed in, resulting in large volumes and frequent updates. By adopting a streaming computing paradigm, intraoperative data can be processed immediately upon receipt, eliminating the need for batch storage and subsequent processing, thus ensuring real-time data processing capabilities. Furthermore, a sliding window strategy can be used to dynamically partition intraoperative data, combining multiple collaborative window partitioning strategies to adapt to diverse application scenarios.

[0074] As an example, surgical patients In time The collected multi-channel intraoperative physiological monitoring signals can be expressed as: ,in, Indicates surgical patients In time The collected A kind of physiological monitoring signal. Introducing the sliding window strategy, the data stream is divided into windows, assuming that the size of each window is , we can get: ,in, Indicates from time Time The data flow within this window.

[0075] The following are three different window division strategies:

[0076] (1) Adaptive sliding window: The window size can be dynamically adjusted according to the risk characteristics of different stages of the operation For example, the window size can be reduced during critical moments of surgery or when the patient's condition fluctuates greatly. To increase the sampling frequency, the window size can be appropriately increased during non-critical surgical moments or when the patient's condition is stable. To reduce the amount of calculation.

[0077] (2) Multiple overlapping windows: Multiple sliding windows of different sizes can be maintained simultaneously to capture physiological state changes at different time scales; for example, a microscopic window of 5-10 seconds is used to capture instantaneous physiological state changes, a medium window of 30-60 seconds is used to monitor short-term physiological state change trends, and a macroscopic window of 5-10 minutes is used to analyze long-term physiological state change trends.

[0078] (3) Priority window: Different processing priorities can be assigned to windows of different physiological indicators based on clinical knowledge and the current surgical stage to ensure that real-time analysis of key indicators is prioritized.

[0079] The low-latency distributed stream processing module can also adopt an edge-first computing strategy, deploying intraoperative data processing tasks on edge devices within the operating room as much as possible, thereby minimizing data transmission delays and improving the real-time response capabilities of risk warnings. For example, data processing tasks can be prioritized on edge devices closest to the data source. Today's edge devices such as monitors are equipped with dedicated signal processing chips that can perform basic tasks such as signal filtering and waveform feature extraction, and transmit task processing results to downstream modules, reducing data bandwidth usage. Edge servers can be deployed in edge computing units within the operating room and equipped with optimized hardware accelerators. They can perform more complex feature extraction and process comprehensive data from multiple monitoring devices. In addition, the system has the ability to adaptively distribute computing loads and can intelligently schedule computing tasks based on the computing power of edge devices, completing all computing tasks locally or uploading some computing tasks to cloud servers for execution when necessary to ensure the processing efficiency and stability of computing tasks. For example, an intelligent computing task allocation mechanism can be adopted to continuously monitor the performance indicators of edge devices such as CPU usage, memory usage, and computing queue length, evaluate the available computing power of edge devices in real time, decompose risk prediction tasks into multiple subtasks with different computational complexities, and prioritize them according to clinical importance. The execution location of each computing task can be dynamically determined based on the current edge device load and task priority.

[0080] The aforementioned lightweight inference module for large temporal models leverages advanced large temporal model technology to extract high-quality temporal feature representations from real-time intraoperative physiological monitoring data, providing critical input for subsequent risk warnings. By combining a pre-trained large temporal model with lightweight technology, this module achieves efficient and accurate extraction of temporal features from complex physiological signals while meeting the real-time requirements of the intraoperative environment.

[0081] In practice, open-source pre-trained time series big models can be used. These models, pre-trained on large-scale time series data from various fields such as finance, meteorology, industrial sensors, and healthcare, possess universal representation and pattern recognition capabilities for time series data. Through self-supervised learning tasks, these big models can understand key semantic information in time series data, such as trends, periodicity, anomalies, and multivariate correlations. This effectively captures long- and short-term temporal dependencies, providing a foundation for analyzing real-time intraoperative physiological monitoring data.

[0082] Based on the open-source time series big model, targeted supervised learning fine-tuning can be introduced to make the time series big model better adapt to the characteristics of intraoperative real-time physiological monitoring data. Specifically, first, by using a large-scale surgical anesthesia database for medical field migration fine-tuning, including intraoperative monitoring data sets of various surgical types, diverse patient data covering different age groups and disease spectrums, and professionally labeled data sets containing marked risk events, the time series big model can be adapted to the special properties of medical physiological signals; secondly, dedicated optimization of intraoperative monitoring scenarios is carried out, and further fine-tuning is carried out for specific scenarios of intraoperative monitoring, so that the time series big model can adapt to the data acquisition frequency and format characteristics of intraoperative monitoring equipment, enhance the ability to recognize specific intraoperative risk patterns, and optimize the sensitivity to anesthesia-related physiological changes; finally, through fine-tuning of multimodal physiological signal integration, the time series big model can effectively process the combination of multiple physiological signals during surgery.

[0083] In addition to introducing supervised learning fine-tuning, integrated reinforcement learning methods can also be used to enhance the performance of large time series models. For example, the two tasks of physiological signal mask prediction and clinical risk assessment can be jointly optimized to achieve a deep understanding of intraoperative physiological monitoring data by the large time series model. By introducing an integrated reinforcement learning framework, it is possible to specifically process multimodal signal inputs such as heart rate, blood pressure, and blood oxygen from intraoperative monitoring equipment, and simultaneously perform dual-task learning of signal reconstruction and disease prediction. In addition, a composite reward function can be used to guide the update of model parameters, and the training stability can be ensured by limiting the update step size. At the same time, a gradient sharing mechanism can be implemented so that the two learning objectives of signal reconstruction and risk assessment can collaboratively adjust the shared feature layer.

[0084] The fine-tuning process within the integrated reinforcement learning framework can generate multiple specialized model variants, forming a pool of expert teachers for subsequent multi-teacher distillation. For example, the physiological signal prediction expert model focuses on time series prediction and anomaly detection in multimodal physiological data; the risk event identification expert model focuses on early warning of intraoperative complications; the anesthetic response modeling expert model excels at capturing dynamic changes in physiological indicators after drug intervention; and the multimodal signal integration expert model specializes in cross-channel correlation analysis. These specialized model variants each strengthen their specific capabilities during fine-tuning, enabling the model to not only adapt to the basic capabilities of the first stage of training but also more accurately perform downstream tasks such as intraoperative risk warning, anesthetic drug response prediction, and real-time patient monitoring, effectively bridging the gap between general-purpose time series representation and surgical risk warning applications. This differentiated, specialized fine-tuning strategy lays the foundation for subsequent multi-teacher distillation, enabling lightweight models to inherit specialized knowledge from different areas, forming comprehensive and efficient intraoperative monitoring capabilities.

[0085] The pre-trained time series large model can also be specially lightweight optimized to improve the real-time performance of risk warning operations. Specifically, the multi-teacher-multi-level distillation framework can be used to integrate multiple specialized fine-tuned time series large models as a teacher set, and their comprehensive knowledge can be transferred to a single lightweight student model. First, a teacher expert pool is constructed, which includes model variants with different skill orientations, such as physiological signal prediction experts, risk event identification experts, anesthesia reaction modeling experts, and multimodal signal integration experts. During the distillation process, three levels of knowledge transfer are implemented: the output layer distribution distillation transfers the predicted probability distribution of each expert for intraoperative physiological monitoring data; the representation layer feature distillation ensures that the student model obtains the rich feature representation capabilities of the intermediate layer; and the multi-source attention distillation guides the student to form an effective attention mechanism for key physiological signal patterns.

[0086] In addition, in order to solve the problem of knowledge conflicts that may arise between multiple teachers, the framework can also introduce a teacher consistency judgment mechanism. During the knowledge distillation process, different teacher models may have inconsistencies in output results due to differences in training data or network structure. This disagreement will interfere with the student model. The framework analyzes the distribution differences of the output of the teacher model in specific areas and dynamically adjusts the distillation weights in the divergent areas to weaken the influence of the teacher model in the low-consistency area, thereby improving the stability and effectiveness of the distillation process. In addition, in order to improve the adaptability of the model to specific tasks, the framework can screen the teacher model based on sample characteristics and select the optimal subset of teacher models to participate in the training of a specific batch. This process combines the feature distribution, category information and task requirements of the samples to ensure that the student model can efficiently learn the knowledge most relevant to the current sample.

[0087] Another aspect of lightweight optimization is model structural optimization. For feature extraction of intraoperative physiological monitoring data, the structure of large time-series models can be streamlined and optimized. For example, influence analysis can identify network layers critical for feature extraction, retaining their full computational capacity while simplifying other layers. The contribution of different feature channels to intraoperative risk identification can be analyzed, retaining high-contribution channels and pruning low-contribution ones, thereby reducing computational complexity while maintaining model performance. Furthermore, various computational acceleration techniques can be employed to further improve feature extraction efficiency. For example, model quantization can be used to reduce model computations from 32-bit floating point to 8-bit or 4-bit integers, and mixed-precision strategies can be flexibly adopted based on the accuracy sensitivity of different network layers. Operator fusion and optimization can be implemented to deeply optimize the computational graph, merging continuous operations and significantly reducing the storage and data transfer overhead of intermediate results. Parallel computing optimization strategies can be carefully designed to take advantage of the multi-core or heterogeneous computing characteristics of intraoperative edge devices, fully utilizing available hardware resources to accelerate feature extraction and achieve efficient and real-time physiological monitoring signal processing within the limited intraoperative computing environment.

[0088] In one implementation of the embodiment of the present application, intraoperative data is input into a trained time series large model for time series feature extraction processing to obtain a time series feature representation of the target surgical patient, including:

[0089] (1) Based on distributed stream computing technology and using a sliding window strategy, the continuously flowing intraoperative data is processed to obtain the physiological signal data stream of the target surgical patient;

[0090] (2) Input the physiological signal data stream into the time series large model for time series feature extraction and processing to obtain the time series feature representation of the target surgical patient.

[0091] In general, the low-latency distributed stream processing module is based on distributed stream computing technology and uses a sliding window strategy to process the continuously flowing intraoperative data to obtain the physiological signal data stream of the target surgical patient. The physiological signal data stream is input into the time series large model lightweight inference module for processing, and the time series large model is used to extract the time series features of the physiological signal data stream, thereby obtaining the time series feature representation of the target surgical patient. As an example, the surgical patient The physiological signal data flow during surgery can be expressed as:

[0092]

[0093] in, Indicates patient In time Physiological monitoring characteristics of the patient, such as heart rate, blood oxygen and blood pressure. Input to the lightweight time series model after distillation By calculation, we can get the time series feature representation , that is .

[0094] 103. Obtain joint feature representation of target surgical patients through pre-constructed multimodal data atlas;

[0095] Through the multimodal data atlas described above, the joint feature representation of the target surgical patient can be obtained , which can be used to characterize the health status of the target surgical patient.

[0096] 104. Based on the joint feature representation and time series feature representation of the target surgical patients, the risk of cardiovascular and cerebrovascular events in the target surgical patients is predicted.

[0097] Combined feature representation of target surgical patients and time series feature representation , it can predict the risk of cardiovascular and cerebrovascular events in target surgical patients. To address the problem of strong individual patient heterogeneity, the embodiment of the present application establishes a self-evolving virtual relationship graph of similar patients based on the multimodal data characteristics of surgical patients, dynamically matches similar patient groups, and achieves personalized adaptation of model parameters through transfer learning. The patient matching relationship is dynamically adjusted during the real-time warning process, thereby improving the accuracy of cardiovascular and cerebrovascular event warnings.

[0098] In one implementation of the embodiment of the present application, predicting the risk of cardiovascular and cerebrovascular events in a target surgical patient based on the joint feature representation and the time series feature representation of the target surgical patient includes:

[0099] (1) Generate a similar patient virtual relationship graph based on the joint feature representation of the target surgical patient and the joint feature representation of multiple historical surgical patients. The similar patient virtual relationship graph is used to represent the similarity between the target surgical patient and multiple historical surgical patients.

[0100] (2) Based on the virtual relationship graph of similar patients, a similar patient set of the target surgical patient is constructed;

[0101] (3) Determine the enhanced feature representation of the target surgical patient based on the similarity weight between each patient in the similar patient set and the target surgical patient, as well as the node representation of each patient in the similar patient set under different modalities;

[0102] (4) Based on the enhanced feature representation and time series feature representation, the risk of cardiovascular and cerebrovascular events in target surgical patients is predicted.

[0103] The self-evolving similar patient matching module utilizes pre-training and leverages multimodal data from a large number of existing surgical patients to establish a self-evolving virtual relationship graph of similar patients. Multimodal data comes from a wide range of sources, including electronic health records, preoperative electrocardiograms, medical imaging, and time-series physiological monitoring data. These data are high-dimensional, heterogeneous, and time-dependent. Each modality is preprocessed and feature extracted. The multimodal graph fusion representation module maps the different types of data into a unified feature space, generating structured patient feature nodes and thus obtaining the multimodal data graph. This multimodal data graph can be used to obtain a joint feature representation of the target surgical patient and a joint feature representation of multiple historical surgical patients. Based on these joint feature representations, the similarity between the health status of each surgical patient can be assessed, thereby generating a virtual relationship graph of similar patients. Based on this virtual relationship graph of similar patients, a certain number of historical surgical patients with a high degree of similarity to the target surgical patient can be identified, thereby obtaining a set of similar patients to the target surgical patient. Then, based on the similarity weight between each patient in the similar patient set and the target surgical patient, as well as the node representation of each patient in the similar patient set under different modalities, the enhanced feature representation of the target surgical patient is calculated. Finally, based on the enhanced feature representation and temporal feature representation of the target surgical patient, the risk of cardiovascular and cerebrovascular events in the target surgical patient is predicted. The enhanced feature representation and temporal feature representation of the target surgical patient serve as the input to the spatiotemporal risk deduction and warning module. The spatiotemporal risk deduction and warning module analyzes the spatiotemporal interaction mechanism of intraoperative risk, generates the future risk evolution trajectory, and triggers hierarchical warnings based on dynamic intervals, thereby obtaining warning results for cardiovascular and cerebrovascular events for the target surgical patient.

[0104] As an example, suppose the target surgical patient is patient At the beginning of the operation, the patient The real-time physiological monitoring data of patients is far less than that of the existing historical patient groups. In this case, the historical risk patterns of similar patients are used for knowledge transfer. The enhanced feature representation of can be calculated by the following formula:

[0105]

[0106] in, Is a patient The enhanced feature representation of Is a patient A set of similar patients, Similar patients Node representation in different modalities, that is, patient Multimodal fusion features, Indicates patient With patients The similarity weight of .

[0107] In one implementation of the embodiment of the present application, a virtual relationship graph of similar patients is generated based on the joint feature representation of the target surgical patient and the joint feature representations of multiple historical surgical patients, including:

[0108] (1) Based on the joint feature representation of the target surgical patient and the joint feature representation of multiple historical surgical patients, calculate the similarity weights between the target surgical patient and multiple historical surgical patients;

[0109] (2) The joint feature representation of the target surgical patient and the joint feature representation of multiple historical surgical patients are used as nodes of the similar patient virtual relationship graph. The edges between the nodes of the similar patient virtual relationship graph are constructed according to the similarity weights between the target surgical patient and multiple historical surgical patients, and the edges whose corresponding similarity weights are less than the set threshold are deleted, thereby obtaining the similar patient virtual relationship graph.

[0110] Based on the joint feature representation of the target surgical patient and the joint feature representation of multiple historical surgical patients, the similarity weights between the target surgical patient and multiple historical surgical patients can be calculated. Specifically, the weighted cosine similarity can be used for measurement, as shown in the following formula:

[0111]

[0112] in, Indicates patient and patients The similarity weight between and Respectively represent patients and The multimodal feature node is also the joint feature representation. It is a trainable similarity transformation matrix used to adjust the weights of different features. During the pre-training phase, the parameters of the similarity transformation matrix W can be optimized through supervised learning. The training data comes from large-scale patient data and its known similarity labels (such as common symptoms, treatment responses, or health status groupings). Through supervised learning, the model can capture deep similarity relationships between patients and accurately reflect the degree of similarity between different patients in pathological characteristics, health status, and treatment response. To further improve the generalization ability of the model, contrastive learning and multi-task learning strategies can also be introduced in the pre-training phase. On the one hand, through contrastive learning, the model learns how to more effectively distinguish the feature distributions of similar and dissimilar patients, thereby enhancing the ability to discern boundary relationships. On the other hand, multi-task learning allows the model to simultaneously focus on the potential similarities of different symptoms or surgical types, making it adaptable to multi-scenario application needs.

[0113] After completing the calculation of the similarity weight, the joint feature representation of the target surgical patient and the joint feature representation of multiple historical surgical patients are used as nodes in the virtual relationship graph of similar patients. According to the similarity weights between the target surgical patient and multiple historical surgical patients, the edges between the nodes of the virtual relationship graph of similar patients are constructed, and the nodes with the corresponding similarity weights less than the set threshold are deleted. That is, only the edges with similarity weight greater than the threshold are retained Patient pairs are sparsely graphed to obtain a virtual relationship graph of similar patients. . Similar patients virtual relationship diagram It can be expressed by the following formula:

[0114]

[0115] in Represents a set of feature nodes for different surgical patients, Represents the set of similar relationships between surgical patients. The virtual relationship graph of similar patients can intuitively present the similarity structure of the patient group and provide a basis for subsequent dynamic matching and risk prediction. In order to maintain computational efficiency, edges with low similarity weights can be periodically removed to ensure that each patient node is only connected to the most relevant similar patients, thus constructing a similar patient set .

[0116] In one implementation of the embodiment of the present application, after generating a similar patient virtual relationship graph based on the joint feature representation of the target surgical patient and the joint feature representations of multiple historical surgical patients, the method further includes:

[0117] When intraoperative data changes, the virtual relationship graph of similar patients is updated according to the updated temporal feature representation.

[0118] Because a patient's intraoperative physiological monitoring characteristics are dynamically changing, the self-evolving similar patient matching module uses a self-evolving update mechanism to adjust the structure of the similar patient virtual relationship graph in real time. When a patient's intraoperative data changes, their temporal feature representation is also updated. Based on this self-evolving update mechanism, the module updates the similar patient virtual relationship graph accordingly.

[0119] As an example, suppose the target surgical patient is patient , in patients During surgery, when the patient's status changes, the node's features are also updated synchronously, as shown in the following formula:

[0120]

[0121] in, Indicates patient Updated physiological monitoring features, Represents the large time series model obtained after distillation in the low-latency distributed stream processing module. is the fusion weight of historical features and new data, The node features before The node features after.

[0122] After that, the similarity weights need to be recalculated to update the weights of the edges in the virtual relationship graph of similar patients, as shown in the following formula:

[0123]

[0124] in, Indicates updated patient and patients The similarity weight between Indicates the patient before the update and patients The similarity weight between is the weight of the fusion of the old similarity weight and the updated similarity weight in the virtual relationship graph of similar patients. , then remove the corresponding edge in the virtual relationship graph of similar patients, otherwise add the corresponding edge in the virtual relationship graph of similar patients, thereby completing the update of the virtual relationship graph of similar patients.

[0125] The spatiotemporal risk deduction and warning module described above is based on the spatiotemporal Transformer architecture. It comprehensively analyzes the spatial characteristics of the virtual relationship graph of similar patients and the temporal changes of intraoperative physiological monitoring signals, captures the interaction between the evolution trend of individual patient risks and group patterns, and explores the risk evolution trajectory of individual patients and their interactions in similar patient groups, thereby predicting the evolution trend of cardiovascular and cerebrovascular events and generating graded warning signals.

[0126] In one implementation of the embodiment of the present application, predicting the risk of cardiovascular and cerebrovascular events in a target surgical patient based on the enhanced feature representation and the time series feature representation includes:

[0127] (1) The enhanced feature representation and the temporal feature representation are input into a deep learning model that has been trained to assess spatiotemporal risk, and the spatiotemporal risk representation of the target surgical patient at the current moment is obtained;

[0128] (2) Based on the spatiotemporal risk representation at the current moment, predict the spatiotemporal risk representation of the target surgical patient at multiple moments in the future;

[0129] (3) Based on the spatiotemporal risk characterization of the target surgical patient at multiple moments in the future, determine the risk level of the target surgical patient experiencing cardiovascular and cerebrovascular events at multiple moments in the future.

[0130] Each surgical patient is not only affected by his or her own temporal physiological monitoring, but also by the risk patterns of similar patient groups. , the enhanced feature representation of the target surgical patient can be calculated , the enhanced features of the target surgical patient are represented as and time series feature representation This input is processed into a deep learning model trained for assessing spatiotemporal risk, resulting in a spatiotemporal risk representation of the target surgical patient at the current moment. In practice, this deep learning model can be trained based on a spatiotemporal Transformer architecture using time series feature data with risk numerical labels and enhanced feature data, enabling it to learn the relationship between temporal and spatial features. The processing of this deep learning model can be expressed using the following formula:

[0131]

[0132] in, represents the spatiotemporal Transformer model, Is a patient The spatiotemporal risk characterization at the current moment can be used for the next step of risk deduction.

[0133] Next, the deduction of future risks can be based on the spatiotemporal risk representation at the current moment. , gradually calculate the patient The spatiotemporal risk representation at multiple moments in the future. Here we can define the risk state transfer function This function draws on the concept of "state transfer function" in reinforcement learning and applies it to risk prediction, inferring future risk assessment values from current risk assessment values. This function is used for multi-step prediction, as shown in the following formulas:

[0134]

[0135]

[0136] ︙

[0137]

[0138] in, Indicates the time of use Spatiotemporal risk characterization , predict the time Spatiotemporal risk characterization , and so on, is the prediction time window.

[0139] Finally, according to the patient Spatiotemporal risk characterization at multiple moments in the future to identify patients The risk level of cardiovascular and cerebrovascular events at multiple times in the future. Spatiotemporal risk characterization at multiple moments in the future can constitute a predicted risk trajectory , based on the risk trajectory and the various risk thresholds set , you can determine the patient The risk level of cardiovascular and cerebrovascular events at multiple moments in the future is shown in the following formula:

[0140]

[0141] in, Indicates patient In the future The risk level of cardiovascular and cerebrovascular events at any time can include low risk, medium risk and high risk.

[0142] It can be seen that through the spatiotemporal risk deduction and early warning module, the future multi-step risk trajectory of surgical patients can be predicted in real time, and real-time intraoperative early warnings for cardiovascular and cerebrovascular events can be dynamically generated, providing medical staff with forward-looking decision-making support, thereby improving the safety of the surgical process.

[0143] The technical solution of the embodiment of the present application constructs a multimodal data atlas based on the multimodal data of multiple historical surgical patients and the preoperative data of the target surgical patient in advance; when the target surgical patient undergoes surgery, the intraoperative data of the target surgical patient will be obtained in real time, and then the intraoperative data will be input into the trained time series large model for time series feature extraction processing to obtain the time series feature representation of the target surgical patient, and a joint feature representation that can be used to characterize the health status of the target surgical patient is obtained through the multimodal data atlas; finally, based on the joint feature representation and the time series feature representation, the risk of cardiovascular and cerebrovascular events in the target surgical patient is predicted. The above process uses the multimodal data atlas to fuse heterogeneous preoperative data and intraoperative data, realizes deep semantic fusion and dynamic association modeling of multimodal data, can fully explore the correlation between cross-modal data, solve the problem of data feature fragmentation, and thus improve the accuracy of risk prediction of cardiovascular and cerebrovascular events in surgical patients.

[0144] In summary, the embodiment of the present application proposes a real-time intraoperative warning model with a framework of "multimodal data fusion-low-latency computing-pre-training drive-personalized dynamic modeling-spatiotemporal risk deduction". By using a multimodal graph fusion representation module, it can solve the problem of multi-source heterogeneous data integration and provide stable and robust multimodal joint information support for risk assessment; by using a low-latency distributed stream processing module, it can realize rapid distributed processing of patient time-series physiological monitoring data to meet the requirements of real-time and high efficiency of warning; by using a lightweight reasoning module of a large time-series model, it can realize efficient extraction and accurate modeling of complex time-series physiological features; by using a self-evolving similar patient matching module, by constructing a virtual relationship graph of similar patients, it can dynamically capture the relationship between the patient's intraoperative status and group characteristics to solve the problem of strong individual differences in surgical patients; by using a spatiotemporal risk deduction and warning module, it can jointly analyze the spatial correlation between the patient's individual time-series physiological data and similar patient groups, capture the evolution trend of patient risks based on spatiotemporal joint representation, and realize future multi-step risk prediction through the risk state transfer function.

[0145] It should be understood that the size of the serial numbers of the steps in the above embodiments does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of this application.

[0146] The above mainly describes a real-time early warning method for cardiovascular and cerebrovascular events during the perioperative period of non-cardiac surgery. The following will describe a real-time early warning device for cardiovascular and cerebrovascular events during the perioperative period of non-cardiac surgery.

[0147] See also Figure 4 , shows a real-time early warning device for perioperative cardiovascular and cerebrovascular events in non-cardiac surgery provided by an embodiment of the present application, comprising:

[0148] The intraoperative data acquisition module 401 is used to acquire intraoperative data of the target surgical patient during the surgical procedure of the target surgical patient;

[0149] The time series feature extraction module 402 is used to input the intraoperative data into the trained time series large model to perform time series feature extraction processing to obtain the time series feature representation of the target surgical patient;

[0150] A joint feature acquisition module 403 is configured to acquire a joint feature representation of a target surgical patient using a pre-constructed multimodal data atlas. The multimodal data atlas is constructed based on multimodal data of multiple historical surgical patients and pre-operative data of the target surgical patient. The joint feature representation of the target surgical patient is used to characterize the health status of the target surgical patient.

[0151] The risk prediction module 404 is used to predict the risk of cardiovascular and cerebrovascular events in the target surgical patient based on the joint feature representation and the time series feature representation of the target surgical patient.

[0152] In one implementation of the embodiment of the present application, the real-time warning device for perioperative cardiovascular and cerebrovascular events in non-cardiac surgery further includes:

[0153] The multimodal encoding module is used to extract and encode the preoperative data of the target surgical patient to obtain multimodal features;

[0154] The spatiotemporal correlation module is used to perform spatiotemporal correlation synchronization and alignment processing on multimodal features;

[0155] The node representation construction module is used to construct the node representation of the target surgical patient in different modalities based on the multimodal features processed by spatiotemporal correlation synchronization alignment;

[0156] an association weight determination module, configured to determine association weights of the node representations of the target surgical patient in each of the different modalities based on similarities between the node representations of the multiple historical surgical patients in each of the different modalities and the node representations of the target surgical patient in each of the different modalities; wherein the node representations of the multiple historical surgical patients in each of the different modalities are constructed based on multimodal data of the multiple historical surgical patients;

[0157] The node representation fusion module is used to fuse the node representations of the target surgical patient in different modalities according to the corresponding association weights to obtain the joint feature representation of the target surgical patient.

[0158] In one implementation of the embodiment of the present application, the risk prediction module includes:

[0159] a virtual relationship graph generating unit, configured to generate a similar patient virtual relationship graph based on a joint feature representation of a target surgical patient and joint feature representations of multiple historical surgical patients, wherein the similar patient virtual relationship graph is used to represent the degree of similarity between the target surgical patient and multiple historical surgical patients;

[0160] A similar patient set construction unit is used to construct a similar patient set for a target surgical patient based on a similar patient virtual relationship graph;

[0161] an enhanced feature representation determining unit, configured to determine an enhanced feature representation of the target surgical patient based on a similarity weight between each patient in the similar patient set and the target surgical patient, and a node representation of each patient in the similar patient set under different modalities;

[0162] The risk prediction unit is used to predict the risk of cardiovascular and cerebrovascular events in target surgical patients based on enhanced feature representation and time series feature representation.

[0163] In one implementation of the embodiment of the present application, the virtual relationship graph generating unit includes:

[0164] A similarity weight calculation subunit, configured to calculate the similarity weights between the target surgical patient and the plurality of historical surgical patients based on the joint feature representation of the target surgical patient and the joint feature representation of the plurality of historical surgical patients;

[0165] The virtual relationship graph generation subunit is used to use the joint feature representation of the target surgical patient and the joint feature representation of multiple historical surgical patients as nodes of the similar patient virtual relationship graph, construct edges between the nodes of the similar patient virtual relationship graph according to the similarity weights between the target surgical patient and multiple historical surgical patients, and delete the edges whose corresponding similarity weights are less than the set threshold, so as to obtain the similar patient virtual relationship graph.

[0166] In one implementation of the embodiment of the present application, the risk prediction unit includes:

[0167] The moment-to-moment risk assessment subunit is used to input the enhanced feature representation and the temporal feature representation into a trained deep learning model for assessing spatiotemporal risk for processing, thereby obtaining the spatiotemporal risk representation of the target surgical patient at the current moment;

[0168] A future risk prediction subunit is used to predict the spatiotemporal risk representation of the target surgical patient at multiple future moments based on the spatiotemporal risk representation at the current moment;

[0169] The risk level determination subunit is used to determine the risk level of the target surgical patient for cardiovascular and cerebrovascular events at multiple moments in the future based on the spatiotemporal risk characterization of the target surgical patient at multiple moments in the future.

[0170] In one implementation of the embodiment of the present application, the risk prediction module further includes:

[0171] The virtual relationship graph updating unit is used to update the virtual relationship graph of similar patients according to the updated time series feature representation when the intraoperative data changes.

[0172] In one implementation of the embodiment of the present application, the time series feature extraction module includes:

[0173] A distributed stream processing unit is used to process the continuously flowing intraoperative data based on distributed stream computing technology and adopt a sliding window strategy to obtain a physiological signal data stream of the target surgical patient;

[0174] The time series feature extraction unit is used to input the physiological signal data stream into the time series large model for time series feature extraction processing to obtain the time series feature representation of the target surgical patient.

[0175] An embodiment of the present application also provides a computer-readable storage medium, which stores a computer program. When the computer program is executed by a processor, it implements the real-time warning method for perioperative cardiovascular and cerebrovascular events in non-cardiac surgery described in any of the above embodiments.

[0176] An embodiment of the present application also provides a computer program product. When the computer program product is run on an electronic device, the electronic device executes the real-time early warning method for perioperative cardiovascular and cerebrovascular events in non-cardiac surgery as described in any of the above embodiments.

[0177] Figure 5 Schematic diagram of an electronic device provided by an embodiment of the present application. Figure 5 As shown, the electronic device 5 of this embodiment includes: a processor 50, a memory 51, and a computer program 52 stored in the memory 51 and executable on the processor 50. When the processor 50 executes the computer program 52, the steps of the above-mentioned embodiments of the method for real-time early warning of cardiovascular and cerebrovascular events during perioperative non-cardiac surgery are implemented, such as Figure 1 Alternatively, when the processor 50 executes the computer program 52, the functions of the modules / units in the above-mentioned device embodiments are realized, for example, Figure 4 Functions of modules 401 - 404 of the illustrated apparatus.

[0178] The computer program 52 may be divided into one or more modules / units, which are stored in the memory 51 and executed by the processor 50 to implement the present application. The one or more modules / units may be a series of computer program instruction segments capable of implementing specific functions, and the instruction segments are used to describe the execution process of the computer program 52 in the electronic device 5.

[0179] The processor 50 may be a central processing unit (CPU), other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA), other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor or any conventional processor.

[0180] The memory 51 can be an internal storage unit of the electronic device 5, such as a hard drive or memory of the electronic device 5. The memory 51 can also be an external storage device of the electronic device 5, such as a plug-in hard drive, a Smart Media Card (SMC), a Secure Digital (SD) card, a flash memory card, etc. equipped with the electronic device 5. Furthermore, the memory 51 can include both an internal storage unit of the electronic device 5 and an external storage device. The memory 51 is used to store the computer program and other programs and data required by the electronic device. The memory 51 can also be used to temporarily store data that has been output or is about to be output.

[0181] If the integrated unit is implemented as a software functional unit and sold or used as a standalone product, it can be stored in a computer-readable storage medium. Based on this understanding, the present application can implement all or part of the process steps in the above-mentioned method embodiments by instructing the relevant hardware through a computer program. The computer program can be stored in a computer-readable storage medium. When executed by a processor, the computer program can implement the steps of each of the above-mentioned method embodiments. The computer program includes computer program code, which can be in source code form, object code form, executable file, or some intermediate form. The computer-readable medium can include: any entity or device capable of carrying the computer program code, recording medium, USB flash drive, mobile hard drive, magnetic disk, optical disk, computer memory, read-only memory (ROM), random access memory (RAM), electric carrier signal, telecommunication signal, and software distribution medium. It should be noted that the content of the computer-readable medium can be appropriately increased or decreased based on the requirements of legislation and patent practice in a jurisdiction. For example, in some jurisdictions, based on legislation and patent practice, computer-readable media does not include electric carrier signals and telecommunication signals.

[0182] The above-described embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. These modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the various embodiments of the present application, and should all be included in the scope of protection of the present application.

Claims

1. A real-time early warning method for cardiovascular and cerebrovascular events during the perioperative period of non-cardiac surgery, characterized in that: include: During the operation of the target surgical patient, obtaining intraoperative data of the target surgical patient; Inputting the intraoperative data into a trained time series large model for time series feature extraction processing to obtain a time series feature representation of the target surgical patient; Obtaining a joint feature representation of the target surgical patient through a pre-constructed multimodal data atlas; wherein the multimodal data atlas is constructed based on multimodal data of multiple historical surgical patients and pre-operative data of the target surgical patient, and the joint feature representation of the target surgical patient is used to characterize the health status of the target surgical patient; Predicting the risk of cardiovascular and cerebrovascular events in the target surgical patient based on the joint feature representation and the time series feature representation of the target surgical patient; The process of constructing the multimodal data map includes: Performing feature extraction and encoding processing on the preoperative data of the target surgical patient to obtain multimodal features; Performing spatiotemporal correlation synchronization alignment processing on the multimodal features; Based on the multimodal features after the spatiotemporal correlation synchronization alignment processing, constructing the node representation of the target surgical patient in each different modality; Determining association weights of the node representations of the target surgical patient in each of the different modalities based on similarities between the node representations of the multiple historical surgical patients in each of the different modalities and the node representations of the target surgical patient in each of the different modalities; wherein the node representations of the multiple historical surgical patients in each of the different modalities are constructed based on multimodal data of the multiple historical surgical patients; fusing the node representations of the target surgical patient in different modalities according to the corresponding association weights to obtain a joint feature representation of the target surgical patient; The predicting the risk of cardiovascular and cerebrovascular events in the target surgical patient based on the joint feature representation and the time series feature representation of the target surgical patient includes: generating a similar patient virtual relationship graph based on the joint feature representation of the target surgical patient and the joint feature representations of the plurality of historical surgical patients, wherein the similar patient virtual relationship graph is used to represent the degree of similarity between the target surgical patient and the plurality of historical surgical patients; Constructing a similar patient set for the target surgical patient according to the similar patient virtual relationship graph; Determining an enhanced feature representation of the target surgical patient according to a similarity weight between each patient in the similar patient set and the target surgical patient, and a node representation of each patient in the similar patient set under different modalities; The risk of cardiovascular and cerebrovascular events in the target surgical patient is predicted based on the enhanced feature representation and the time series feature representation.

2. The method according to claim 1, wherein Generating a similar patient virtual relationship graph based on the joint feature representation of the target surgical patient and the joint feature representations of the plurality of historical surgical patients includes: Calculating similarity weights between each of the target surgical patient and the plurality of historical surgical patients based on the joint feature representation of the target surgical patient and the joint feature representations of the plurality of historical surgical patients; The joint feature representation of the target surgical patient and the joint feature representation of the multiple historical surgical patients are used as nodes of the similar patient virtual relationship graph. The edges between the nodes of the similar patient virtual relationship graph are constructed according to the similarity weights between the target surgical patient and the multiple historical surgical patients, and the edges whose corresponding similarity weights are less than the set threshold are deleted, thereby obtaining the similar patient virtual relationship graph.

3. The method according to claim 1, wherein The predicting the risk of cardiovascular and cerebrovascular events in the target surgical patient based on the enhanced feature representation and the time series feature representation includes: Inputting the enhanced feature representation and the temporal feature representation into a trained deep learning model for assessing spatiotemporal risk for processing, thereby obtaining a spatiotemporal risk representation of the target surgical patient at the current moment; Based on the spatiotemporal risk representation at the current moment, predicting the spatiotemporal risk representation of the target surgical patient at multiple moments in the future; The risk level of the target surgical patient for cardiovascular and cerebrovascular events at multiple moments in the future is determined based on the spatiotemporal risk characterization of the target surgical patient at multiple moments in the future.

4. The method according to claim 1, wherein After generating a similar patient virtual relationship graph based on the joint feature representation of the target surgical patient and the joint feature representations of the plurality of historical surgical patients, the method further includes: When the intraoperative data changes, the similar patient virtual relationship graph is updated according to the updated temporal feature representation.

5. The method according to any one of claims 1 to 4, characterized in that Inputting the intraoperative data into the trained time series large model for time series feature extraction processing to obtain the time series feature representation of the target surgical patient includes: Processing the continuously flowing intraoperative data based on distributed stream computing technology and using a sliding window strategy to obtain a physiological signal data stream of the target surgical patient; The physiological signal data stream is input into the time series large model for time series feature extraction processing to obtain the time series feature representation of the target surgical patient.

6. A real-time early warning device for cardiovascular and cerebrovascular events during the perioperative period of non-cardiac surgery, characterized in that: include: An intraoperative data acquisition module is used to acquire intraoperative data of a target surgical patient during the surgical procedure of the target surgical patient; A time series feature extraction module is used to input the intraoperative data into a trained time series large model for time series feature extraction processing to obtain a time series feature representation of the target surgical patient; a joint feature acquisition module, configured to acquire a joint feature representation of the target surgical patient using a pre-constructed multimodal data atlas, wherein the multimodal data atlas is constructed based on multimodal data of multiple historical surgical patients and pre-operative data of the target surgical patient, and the joint feature representation of the target surgical patient is used to characterize the health status of the target surgical patient; a risk prediction module, configured to predict the risk of the target surgical patient experiencing cardiovascular and cerebrovascular events based on the joint feature representation of the target surgical patient and the time series feature representation; A multimodal encoding module, configured to extract and encode the preoperative data of the target surgical patient to obtain multimodal features; A spatiotemporal correlation module, configured to perform spatiotemporal correlation synchronization alignment processing on the multimodal features; A node representation construction module is used to construct a node representation of the target surgical patient in each different modality based on the multimodal features after the spatiotemporal correlation synchronization alignment processing; an association weight determination module, configured to determine an association weight of the node representation of the target surgical patient in each of the different modalities based on similarities between the node representations of the multiple historical surgical patients in each of the different modalities and the node representations of the target surgical patient in each of the different modalities; wherein the node representations of the multiple historical surgical patients in each of the different modalities are constructed based on the multimodal data of the multiple historical surgical patients; a node representation fusion module, configured to fuse the node representations of the target surgical patient in different modalities according to the corresponding association weights to obtain a joint feature representation of the target surgical patient; The risk prediction module includes: a virtual relationship graph generating unit, configured to generate a similar patient virtual relationship graph based on the joint feature representation of the target surgical patient and the joint feature representations of the plurality of historical surgical patients, wherein the similar patient virtual relationship graph is used to represent the degree of similarity between the target surgical patient and the plurality of historical surgical patients; A similar patient set construction unit is used to construct a similar patient set for the target surgical patient based on the similar patient virtual relationship graph; an enhanced feature representation determining unit, configured to determine an enhanced feature representation of the target surgical patient based on a similarity weight between each patient in the similar patient set and the target surgical patient, and a node representation of each patient in the similar patient set under different modalities; A risk prediction unit is used to predict the risk of cardiovascular and cerebrovascular events in the target surgical patient based on the enhanced feature representation and the time series feature representation.

7. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the computer program, the real-time early warning method for perioperative cardiovascular and cerebrovascular events in non-cardiac surgery according to any one of claims 1 to 5 is implemented.

8. A computer program product, characterized in that When the computer program product is run on an electronic device, the electronic device is enabled to execute the real-time early warning method for perioperative cardiovascular and cerebrovascular events in non-cardiac surgery as claimed in any one of claims 1 to 5.

Images