Real-time early warning method and device for cardiovascular and cerebrovascular events in perioperative period of non-cardiac surgery

Abstract

The invention relates to the technical field of medical data processing, and provides a real-time early warning method and device for cardiovascular and cerebrovascular events in a non-cardiac perioperative period. The method comprises the following steps: in an operation process of a target operation patient, acquiring intraoperative data of the target operation patient; inputting the intra-operative data into a trained time sequence large model to perform time sequence feature extraction processing to obtain time sequence feature representation of a target operative patient; through a pre-constructed multi-modal data graph, obtaining a joint feature representation of the target surgical patient; wherein the multi-modal data map is constructed based on multi-modal data of a plurality of historical surgical patients and preoperative data of a target surgical patient, and the combined feature representation of the target surgical patient is used for representing the health state of the target surgical patient; and according to the joint feature representation and the time sequence feature representation of the target surgical patient, predicting the risk of the cardiovascular and cerebrovascular event of the target surgical patient. By adopting the method, the accuracy of cardiovascular and cerebrovascular event risk prediction of the surgical patient can be improved.

Description

Technical Field

[0001] The present application relates to the field of medical data processing technology, and in particular to a real-time early warning method, device, electronic equipment and computer program product for cardiovascular and cerebrovascular events during the perioperative period of non-cardiac surgery. Background Art

[0002] Patients undergoing non-cardiac surgery may experience various cardiovascular and cerebrovascular events during the perioperative period, such as myocardial infarction, stroke, and arrhythmia, which can pose a serious threat to the life and health of the patients. Therefore, how to predict the risk of cardiovascular and cerebrovascular events that may occur in patients during the perioperative period so that doctors can take timely measures to effectively intervene has become a key issue that technicians in this field need to solve.

[0003] In response to the above problems, the prior art has proposed a cardiovascular and cerebrovascular event warning method based on a deep learning model. This method inputs the multimodal data of surgical patients before and during surgery into the cardiovascular and cerebrovascular event warning model for risk prediction and processing, and can obtain corresponding risk warning results. However, in view of the heterogeneous and heterogeneous characteristics of preoperative and intraoperative data, the existing warning models usually use simple feature splicing or independent data processing branches, lacking a unified representation framework, which will result in the deep correlation between different modal data not being fully explored, reducing the accuracy of cardiovascular and cerebrovascular event risk prediction. Summary of the invention

[0004] In view of this, the embodiments of the present application provide a real-time warning method, device, electronic device and computer program product for cardiovascular and cerebrovascular events during the perioperative period of non-cardiac surgery, which can improve the accuracy of risk prediction of cardiovascular and cerebrovascular events in surgical patients.

[0005] A first aspect of an embodiment of the present application provides a real-time warning method for cardiovascular and cerebrovascular events during perioperative period of non-cardiac surgery, comprising: During the operation of the target surgical patient, obtaining intraoperative data of the target surgical patient; Input the intraoperative data into the trained time series big model for time series feature extraction and processing to obtain the time series feature representation of the target surgical patient; A joint feature representation of a target surgical patient is obtained through a pre-constructed multimodal data atlas; wherein the multimodal data atlas is constructed based on multimodal data of multiple historical surgical patients and preoperative data of the target surgical patient, and the joint feature representation of the target surgical patient is used to characterize the health status of the target surgical patient; Based on the joint feature representation and time series feature representation of the target surgical patients, the risk of cardiovascular and cerebrovascular events in the target surgical patients is predicted.

[0006] The technical solution of the embodiment of the present application pre-constructs a multimodal data map based on the multimodal data of multiple historical surgical patients and the preoperative data of the target surgical patient; when the target surgical patient undergoes surgery, the intraoperative data of the target surgical patient will be obtained in real time, and then the intraoperative data will be input into the trained time series large model for time series feature extraction processing to obtain the time series feature representation of the target surgical patient, and the joint feature representation that can be used to characterize the health status of the target surgical patient is obtained through the multimodal data map; finally, according to the joint feature representation and the time series feature representation, the risk of cardiovascular and cerebrovascular events in the target surgical patient is predicted. The above process uses the multimodal data map to fuse heterogeneous preoperative data and intraoperative data, realizes the deep semantic fusion and dynamic association modeling of multimodal data, can fully explore the correlation between cross-modal data, solve the problem of data feature fragmentation, and thus improve the accuracy of risk prediction of cardiovascular and cerebrovascular events in surgical patients.

[0007] In one implementation of the embodiment of the present application, the process of constructing a multimodal data graph includes: Perform feature extraction and encoding on the preoperative data of the target surgical patients to obtain multimodal features; Performing time-space correlation and synchronous alignment processing on multimodal features; Based on the multi-modal features processed by time-space correlation synchronization alignment, the node representation of the target surgical patient in each different modality is constructed; Determine the association weight of the node representation of the target surgical patient in each different modality according to the similarity between the node representation of the multiple historical surgical patients in each different modality and the node representation of the target surgical patient in each different modality; wherein the node representation of the multiple historical surgical patients in each different modality is constructed based on the multimodal data of the multiple historical surgical patients; The node representations of the target surgical patient in different modalities are fused according to the corresponding association weights to obtain the joint feature representation of the target surgical patient.

[0008] In one implementation of the embodiment of the present application, the risk of cardiovascular and cerebrovascular events in the target surgical patient is predicted based on the joint feature representation and the time series feature representation of the target surgical patient, including: Generate a similar patient virtual relationship graph based on the joint feature representation of the target surgical patient and the joint feature representation of multiple historical surgical patients. The similar patient virtual relationship graph is used to characterize the similarity between the target surgical patient and multiple historical surgical patients. According to the virtual relationship graph of similar patients, a similar patient set of the target surgical patient is constructed; Determine an enhanced feature representation of the target surgical patient according to a similarity weight between each patient in the similar patient set and the target surgical patient, and a node representation of each patient in the similar patient set under different modalities; Based on enhanced feature representation and time series feature representation, the risk of cardiovascular and cerebrovascular events in target surgical patients is predicted.

[0009] In one implementation of the embodiment of the present application, a virtual relationship graph of similar patients is generated according to the joint feature representation of the target surgical patient and the joint feature representations of multiple historical surgical patients, including: According to the joint feature representation of the target surgical patient and the joint feature representation of multiple historical surgical patients, the similarity weights between the target surgical patient and multiple historical surgical patients are calculated; The joint feature representation of the target surgical patient and the joint feature representation of multiple historical surgical patients are used as nodes of the virtual relationship graph of similar patients. The edges between the nodes of the virtual relationship graph of similar patients are constructed according to the similarity weights between the target surgical patient and multiple historical surgical patients, and the edges whose corresponding similarity weights are less than the set threshold are deleted to obtain the virtual relationship graph of similar patients.

[0010] In one implementation of the embodiment of the present application, the risk of cardiovascular and cerebrovascular events in a target surgical patient is predicted based on the enhanced feature representation and the time series feature representation, including: The enhanced feature representation and the temporal feature representation are input into a deep learning model that has been trained to assess spatiotemporal risk for processing, thereby obtaining the spatiotemporal risk representation of the target surgical patient at the current moment; Based on the spatiotemporal risk representation at the current moment, predict the spatiotemporal risk representation of the target surgical patient at multiple moments in the future; Based on the spatiotemporal risk characterization of the target surgical patient at multiple moments in the future, the risk level of the target surgical patient experiencing cardiovascular and cerebrovascular events at multiple moments in the future is determined.

[0011] In an implementation of the embodiment of the present application, after generating a similar patient virtual relationship graph according to the joint feature representation of the target surgical patient and the joint feature representations of multiple historical surgical patients, the method further includes: When intraoperative data changes, the virtual relationship graph of similar patients is updated according to the updated temporal feature representation.

[0012] In one implementation of the embodiment of the present application, the intraoperative data is input into the trained time series large model for time series feature extraction processing to obtain the time series feature representation of the target surgical patient, including: Based on distributed stream computing technology and using a sliding window strategy, the continuously flowing intraoperative data is processed to obtain the physiological signal data stream of the target surgical patient; The physiological signal data stream is input into the large time series model for time series feature extraction and processing to obtain the time series feature representation of the target surgical patient.

[0013] A second aspect of the embodiment of the present application provides a real-time warning device for cardiovascular and cerebrovascular events during the perioperative period of non-cardiac surgery, comprising: An intraoperative data acquisition module is used to acquire intraoperative data of a target surgical patient during the surgical procedure of the target surgical patient; The time series feature extraction module is used to input the intraoperative data into the trained time series large model for time series feature extraction processing to obtain the time series feature representation of the target surgical patient; A joint feature acquisition module is used to acquire a joint feature representation of a target surgical patient through a pre-constructed multimodal data atlas; wherein the multimodal data atlas is constructed based on multimodal data of multiple historical surgical patients and preoperative data of the target surgical patient, and the joint feature representation of the target surgical patient is used to characterize the health status of the target surgical patient; The risk prediction module is used to predict the risk of cardiovascular and cerebrovascular events in target surgical patients based on the joint feature representation and time series feature representation of the target surgical patients.

[0014] A third aspect of an embodiment of the present application provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, a real-time early warning method for perioperative cardiovascular and cerebrovascular events during non-cardiac surgery is implemented as provided in the first aspect of an embodiment of the present application.

[0015] The fourth aspect of the embodiments of the present application provides a computer program product. When the computer program product runs on an electronic device, the electronic device executes the real-time early warning method for perioperative cardiovascular and cerebrovascular events in non-cardiac surgery as provided in the first aspect of the embodiments of the present application.

[0016] The fifth aspect of the embodiments of the present application provides a computer-readable storage medium, which stores a computer program. When the computer program is executed by a processor, it implements the real-time warning method for perioperative cardiovascular and cerebrovascular events in non-cardiac surgery as provided in the first aspect of the embodiments of the present application.

[0017] It can be understood that the beneficial effects of the second to fifth aspects mentioned above can be found in the relevant description of the first aspect mentioned above, and will not be repeated here. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] Figure 1 This is a flow chart of a real-time early warning method for cardiovascular and cerebrovascular events during the perioperative period of non-cardiac surgery provided by an embodiment of the present application; Figure 2This is a schematic diagram of an overall technical framework of a real-time early warning method for perioperative cardiovascular and cerebrovascular events in non-cardiac surgery provided by an embodiment of the present application; Figure 3 yes Figure 2 Schematic diagram of the working principle of the multimodal graph fusion representation module in the shown framework; Figure 4 It is a structural schematic diagram of a real-time warning device for perioperative cardiovascular and cerebrovascular events in non-cardiac surgery provided by an embodiment of the present application; Figure 5 It is a schematic diagram of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0019] In the following description, for the purpose of illustration rather than limitation, specific details such as specific system structures, technologies, etc. are proposed, so as to thoroughly understand the embodiments of the present application. However, it should be clear to those skilled in the art that the present application can also be implemented in other embodiments without these specific details. In other cases, the detailed description of well-known systems, devices, circuits and methods is omitted to prevent unnecessary details from hindering the description of the present application. In addition, in the description of the present application specification and the attached claims, the terms "first", "second", "third" etc. are only used to distinguish the description, and cannot be interpreted as indicating or suggesting relative importance.

[0020] Perioperative cardiovascular and cerebrovascular complications are important causes of death or disability in surgical patients. Early warning of such cardiovascular and cerebrovascular events depends on a comprehensive evaluation of multimodal data such as the electronic health records, preoperative electrocardiograms, medical images, and intraoperative physiological monitoring of surgical patients. Existing early warning models generally use simple feature splicing or independent data processing branches when processing multimodal data, which cannot fully explore the deep correlation between different modal data, affecting the comprehensiveness and accuracy of cardiovascular and cerebrovascular event risk prediction.

[0021] In response to the above technical problems, the embodiments of the present application propose a real-time warning method, device, electronic device and computer program product for cardiovascular and cerebrovascular events during the perioperative period of non-cardiac surgery. By introducing a multimodal data map, deep semantic fusion and dynamic association modeling can be achieved, and the correlation of cross-modal data can be fully explored, thereby effectively improving the comprehensiveness and accuracy of cardiovascular and cerebrovascular event risk prediction. For more specific technical implementation details of the embodiments of the present application, please refer to the various method embodiments described below.

[0022] It should be understood that the execution subjects of the various method embodiments proposed in the present application can be various types of electronic devices, such as mobile phones, tablet computers, desktop computers, wearable devices, medical devices, augmented reality (AR) / virtual reality (VR) devices, laptop computers, ultra-mobile personal computers (UMPC), netbooks, personal digital assistants (PDA), large-screen TVs, etc. The embodiments of the present application do not impose any restrictions on the specific type of the electronic device.

[0023] See also Figure 1 , shows a real-time warning method for cardiovascular and cerebrovascular events during the perioperative period of non-cardiac surgery provided by an embodiment of the present application, comprising: 101. During the operation of the target surgical patient, obtain the intraoperative data of the target surgical patient; In general, the technical solution of the embodiment of the present application proposes a real-time intraoperative warning model based on machine learning, which integrates the multimodal data features of surgical patients, constructs a self-evolving virtual relationship diagram of similar patients, and combines pre-training technology to achieve personalized accurate warning of cardiovascular and cerebrovascular events. The core of this technical solution is to build a full-process technical framework of "multimodal data fusion-low-latency computing-pre-training drive-personalized dynamic modeling-space-time risk deduction" to achieve accurate risk prediction of cardiovascular and cerebrovascular events.

[0024] As an example, Figure 2 It is a schematic diagram of an overall technical framework of a real-time early warning method for perioperative cardiovascular and cerebrovascular events in non-cardiac surgery provided in an embodiment of the present application. Figure 2 The technical framework shown includes the following five functional modules: multimodal graph fusion representation module, low-latency distributed stream processing module, time series large model lightweight reasoning module, self-evolving similar patient matching module and spatiotemporal risk deduction and warning module. The intraoperative real-time warning model is composed of these five functional modules. Each functional module is briefly described below.

[0025] Multimodal graph fusion representation module: In order to solve the problem that multimodal data has different formats and types and is difficult to efficiently fuse and comprehensively represent, the multimodal graph fusion representation module can effectively fuse multimodal data by building a unified fusion framework to extract cross-modal correlation features and solve the problem of data feature fragmentation. In general, the multimodal graph fusion representation module first extracts and encodes the features of each modality through the encoder, then performs spatiotemporal synchronous correlation alignment, calculates the correlation weights between different data modalities, and finally establishes highly structured patient feature nodes. These patient feature nodes together form a multimodal data graph. Using the multimodal graph fusion representation module, deep semantic fusion and dynamic correlation modeling of multimodal data can be achieved, and the important features of each modality data can be fully mined to provide a highly robust joint representation for intraoperative risk warning.

[0026] Low-latency distributed stream processing module: Intraoperative data has the characteristics of large data volume and real-time processing. Traditional early warning models usually use fixed time windows or offline batch processing modes to analyze intraoperative physiological monitoring signals. Their computational efficiency is low, dynamic adaptability is insufficient, and there is a significant lag in the processing of intraoperative physiological monitoring signals. In order to solve this problem and meet the requirements of real-time and high-efficiency early warning, the low-latency distributed stream processing module adopts the collaborative design of distributed stream computing technology and edge-first computing framework. Through the distributed computing architecture and streaming processing paradigm, it can reduce computing delay while ensuring prediction accuracy, meet the clinical needs of real-time risk response during surgery, and realize real-time processing and transmission of intraoperative physiological monitoring data. Specifically, the low-latency distributed stream processing module uses a sliding window strategy to dynamically divide the multi-channel physiological data that continuously flows in during surgery, and can combine multiple collaborative strategies to accurately capture the changes in the physiological state of surgical patients during surgery. At the same time, the edge-first computing architecture is introduced to prioritize the deployment of computing tasks on edge devices for execution, and dynamically allocate computing tasks based on the real-time computing power status of the device. By combining intelligent task scheduling, computing delay can be greatly reduced and computing efficiency can be improved.

[0027] Lightweight reasoning module for time series large models: In view of the characteristics of intraoperative physiological monitoring signals being relatively complex and requiring real-time processing, the lightweight reasoning module for time series large models, based on the open source pre-trained time series large models, achieves model parameter compression and computational optimization through targeted fine-tuning and lightweight technology, so that the time series large model can be efficiently deployed on edge devices and supports real-time reasoning of high-frequency intraoperative physiological data. Specifically, the lightweight reasoning module for time series large models uses the pre-trained time series large model to extract high-quality features from intraoperative physiological monitoring signals, and ensures real-time processing capabilities through lightweight optimization technology. This module fine-tunes the migration of the time series large model in the medical field to adapt it to the characteristics of intraoperative monitoring data, and can enhance the ability to identify key risk patterns. In addition, by combining technologies such as knowledge distillation, model structure optimization, and computational acceleration, the complexity of model calculation can be significantly reduced, while ensuring the accuracy of feature extraction, meeting the requirements of the intraoperative environment for low latency and high efficiency, and providing accurate and reliable feature input for intraoperative risk warning.

[0028] Self-evolving similar patient matching module: There are often significant individual differences between different surgical patients. Each surgical patient has different physiological, genetic background, age, gender, medical history, lifestyle and other factors. These differences may significantly affect the intraoperative response and the risk of cardiovascular and cerebrovascular events. For example, there are large differences between elderly patients and young patients in cardiovascular tolerance, metabolic capacity, immune response and other aspects; patients with a history of chronic diseases or multiple chronic diseases may also have different surgical risks and intraoperative reactions; although some patients have no obvious history of disease, they may still suffer from sudden serious cardiovascular and cerebrovascular events due to the particularity of physiological state, anesthesia reaction or surgical type; factors such as the patient's psychological state, drug metabolism characteristics, weight, and even surgical type will lead to significant individual differences in the occurrence mechanism of cardiovascular and cerebrovascular events. In order to solve the problem of individual differences among patients, the self-evolving similar patient matching module establishes a self-evolving similar patient virtual relationship diagram based on the multimodal data characteristics of surgical patients, dynamically matches similar patient groups, and realizes personalized adaptation of model parameters through transfer learning, so that the currently predicted surgical patients are as similar as possible to the existing training population, thereby improving the accuracy of cardiovascular and cerebrovascular event warning. Specifically, the self-evolving similar patient matching module first uses the existing large-scale patient data to train a similarity measurement model through pre-training, and combines the patient feature nodes output by the multimodal graph fusion representation module to construct a dynamically updated self-evolving similar patient virtual relationship graph. Moreover, in the real-time warning process, the structure of the virtual relationship graph of similar patients is adaptively adjusted according to the calculated similarity of the new input, realizing personalized patient group matching and knowledge transfer for intraoperative risk prediction.

[0029] Spatiotemporal risk deduction and early warning module: Traditional early warning models rely on static thresholds or single-time point risk assessments, and are unable to model the spatiotemporal evolution of intraoperative risks in real time. They can only output current or short-term risk probabilities, lack multi-step prediction capabilities, and are difficult to locate key intraoperative risk trigger nodes. To address this problem, the spatiotemporal risk deduction and early warning module analyzes the spatiotemporal interaction mechanism of intraoperative risks, generates future risk evolution trajectories, and triggers hierarchical early warnings based on dynamic intervals. Specifically, the spatiotemporal risk deduction and early warning module is based on the spatiotemporal Transformer architecture, integrating the spatial correlation in the self-evolving virtual relationship graph of similar patients with the temporal evolution characteristics of real-time intraoperative physiological monitoring signals, mining the spatiotemporal interaction between individual characteristics of surgical patients and group risk patterns, dynamically analyzing the spatiotemporal distribution characteristics of key cardiovascular and cerebrovascular events, generating future risk evolution trajectories, and triggering graded early warning signals based on confidence intervals.

[0030] For a more detailed explanation of the working principles of the above five functional modules, please refer to the description below.

[0031] Before implementing the real-time early warning method for perioperative cardiovascular and cerebrovascular events in non-cardiac surgery provided by the embodiment of the present application, it is necessary to pre-construct a multimodal data map, which can be constructed based on the multimodal data of a large number of historical surgical patients and the preoperative data of target surgical patients. Among them, multimodal data includes two categories: preoperative data and intraoperative data. Preoperative data includes but is not limited to electronic health records, preoperative electrocardiograms, and various medical images. Intraoperative data is real-time time-series physiological monitoring data, such as blood pressure, heart rate, and blood oxygen. Historical surgical patients refer to patients who have completed surgery, and their preoperative data and intraoperative data have been collected; target surgical patients refer to patients who currently need surgery and realize early warning of cardiovascular and cerebrovascular events. Their preoperative data have been collected, but the intraoperative data needs to be collected in real time during the operation. The specific construction method of the multimodal data map is introduced below.

[0032] In one implementation of the embodiment of the present application, the process of constructing a multimodal data graph includes: (1) Extract and encode the preoperative data of the target surgical patients to obtain multimodal features; (2) Performing temporal and spatial correlation synchronization alignment on multimodal features; (3) Based on the multimodal features processed by spatiotemporal correlation synchronization and alignment, the node representation of the target surgical patient in each different modality is constructed; (4) determining the association weights of the node representations of the target surgical patient in each different modality based on the similarities between the node representations of the multiple historical surgical patients in each different modality and the node representations of the target surgical patient in each different modality; wherein the node representations of the multiple historical surgical patients in each different modality are constructed based on the multimodal data of the multiple historical surgical patients; (5) The node representations of the target surgical patient in different modalities are fused according to the corresponding association weights to obtain the joint feature representation of the target surgical patient.

[0033] The multimodal data graph is composed of multiple high-dimensional structured feature nodes, each of which corresponds to a surgical patient. It can be used to capture the complex health status of the corresponding surgical patient during the perioperative period and can be called the joint feature representation of the corresponding surgical patient. The construction of the multimodal data graph can be completed through the above-mentioned multimodal graph fusion representation module. As an example, Figure 3 yes Figure 2 Schematic diagram of the working principle of the multimodal graph fusion representation module in the framework shown. Figure 3 In the process, the multimodal data of surgical patients (including preoperative electronic health records, electrocardiograms and medical images, as well as intraoperative physiological monitoring signals, etc.) are first input into the multimodal encoder for feature extraction and encoding. In the training stage of the early warning model, the complete preoperative and intraoperative data of surgical patients are input into the model, while in the intraoperative real-time early warning, that is, the use stage of the early warning model, the preoperative data of surgical patients are input into the model in advance, and the intraoperative data needs to be collected in real time during the operation. Using a multimodal encoder based on deep learning, the multimodal data of surgical patients can be converted into a unified feature representation, thereby obtaining the corresponding multimodal features. For example, after inputting the data of four modes into the multimodal encoder for feature extraction and encoding, the feature representations corresponding to each of the four modal data can be obtained, and these feature representations constitute multimodal features. Considering that various intraoperative time-series physiological monitoring signals (such as heart rate, blood oxygen saturation and respiratory rate, etc.) often come from different monitoring devices, these signals may have differences in sampling frequency and sampling time, resulting in data of different modalities being misaligned in the time dimension. In the spatial dimension, the anatomical structure in the medical image also needs to establish a corresponding association with the intraoperative monitoring site. Therefore, it is necessary to perform spatiotemporal correlation synchronization alignment processing on the multimodal features. Specifically, interpolation technology can be used to fill or smooth the time series data, and high-frequency data can be downsampled. This can avoid information loss or noise interference caused by time misalignment. The association between the image anatomical structure and the intraoperative monitoring site can be established by coordinate transformation. Afterwards, based on the multimodal features after spatiotemporal correlation synchronization alignment processing, the node representation of the surgical patient in each different modality is constructed, and the weighted atlas of the multimodal data is established by calculating the association weights between the different modal data, thereby obtaining the above-mentioned multimodal data atlas. In order to ensure the effective fusion of data from different modalities, the embodiment of the present application adopts weighted graph fusion technology to convert data into a graph structure and use the similarity between graph nodes for weighted fusion to achieve more accurate risk prediction. Indicates that, through the encoder Obtained The patients who underwent surgery Nodes under the mode indicate that they are available It can be understood that, assuming that there are M modes in the multimodal features, each surgical patient has M node representations, that is, M feature nodes, so N surgical patients have a total of N*M feature nodes, and these feature nodes constitute the node set of the graph structure. The graph structure can be expressed by the formula It indicates that, Represents the graph structure, Represents a node set, which contains the characteristic nodes of all modes. The edge represents the node, which is used to measure the similarity relationship between modalities. Assume that the target surgical patient is patient , and its node representations in different modes are: , … … , the edge weights between feature nodes can be used to represent the similarity of feature nodes, which can be determined by calculating the Euclidean distance or cosine similarity between feature nodes. For example, using the formula Patients can be calculated In the Feature nodes and patients under modality In the Edge weights between feature nodes under modality , where Sim(⋅) is the similarity metric function and M represents the number of data modalities. The associated weight of each feature node It can be dynamically adjusted through its relationship with adjacent nodes. , where N is the number of patients. The associated weight of each feature node Afterwards, the features of different modal nodes are fused according to the corresponding association weights through weighted atlas fusion to obtain the patient The joint feature representation of , the formula available is Representation. The fused joint feature representation Patients High-dimensional structured feature nodes can be used to characterize patients The high-dimensional structured feature nodes of all patients together constitute a multimodal data map. It can be seen that by querying the multimodal data map, the joint feature representation of each surgical patient can be obtained, and then the complex health status of each surgical patient can be evaluated.

[0034] Before the target surgical patient undergoes surgery, a multimodal data map is first constructed in the above manner. Subsequently, during the target surgical patient's surgery, the target surgical patient's intraoperative data is obtained in real time, and the target surgical patient's time series feature representation is extracted based on the intraoperative data. The target surgical patient's joint feature representation is obtained through the multimodal data map, and finally the time series feature representation and the joint feature representation are used to complete the accurate risk prediction of cardiovascular and cerebrovascular events.

[0035] 102. Input the intraoperative data into the trained time series large model to extract time series features and obtain the time series feature representation of the target surgical patient; The intraoperative data of the target surgical patients obtained can first be input into the above-mentioned low-latency distributed stream processing module for processing. Through the distributed computing architecture and streaming processing paradigm, this module can ensure that when a large amount of continuous data flows in, it can maintain low-latency and high-throughput data processing capabilities, thereby meeting the requirements of real-time and efficient early warning.

[0036] During the operation of the target surgical patient, the intraoperative data mainly includes various physiological monitoring signals, such as heart rate, blood oxygen and blood pressure, etc. These data are continuously streamed, with large data volume and high data update frequency. By adopting the streaming computing paradigm, the intraoperative data is processed immediately once received, without the need for batch storage and then processing, thus ensuring the real-time processing capability of the data. In addition, the sliding window strategy can be used to dynamically divide the intraoperative data, and combined with a variety of collaborative window division strategies to meet the needs of different application scenarios.

[0037] As an example, surgical patients In time The collected multi-channel intraoperative physiological monitoring signals can be expressed as: ,in, Indicates surgical patients In time The collected A kind of physiological monitoring signal. Introduce the sliding window strategy to divide the data stream into windows. Assume that the size of each window is , we can get: ,in, Indicates from time Time The data flow within this window.

[0038] The following are three different window division strategies: (1) Adaptive sliding window: The window size can be dynamically adjusted according to the risk characteristics of different stages of the operation. ; For example, the window size can be reduced during critical moments of surgery or when the patient's condition fluctuates greatly To increase the sampling frequency, and in non-critical moments of surgery or when the patient's condition is stable, the window size can be appropriately increased To reduce the amount of calculation.

[0039] (2) Multiple overlapping windows: Multiple sliding windows of different sizes can be maintained simultaneously to capture changes in physiological states at different time scales; for example, a micro window of 5-10 seconds is used to capture instantaneous changes in physiological states, a medium window of 30-60 seconds is used to monitor short-term trends in physiological states, and a macro window of 5-10 minutes is used to analyze long-term trends in physiological states.

[0040] (3) Priority window: Different processing priorities can be assigned to windows of different physiological indicators based on clinical knowledge and the current surgical stage to ensure that real-time analysis of key indicators is prioritized.

[0041] The low-latency distributed stream processing module can also adopt an edge-first computing strategy to deploy intraoperative data processing tasks on edge devices in the operating room as much as possible, thereby minimizing data transmission delays and improving the real-time response capabilities of risk warnings. For example, data processing tasks are preferentially deployed on edge devices closest to the data source. Today's edge devices such as monitors are equipped with dedicated signal processing chips that can complete basic signal filtering, waveform feature extraction and other tasks, and transmit task processing results to downstream modules to reduce data bandwidth usage. The edge server can be deployed in the edge computing unit in the operating room, equipped with optimized hardware accelerators, capable of performing more complex feature extraction and processing comprehensive data from multiple monitoring devices. In addition, the system has the ability to adaptively distribute computing loads, and can intelligently schedule computing tasks according to the computing power of edge devices, complete all computing tasks locally, or upload some computing tasks to cloud servers for execution when necessary to ensure the processing efficiency and stability of computing tasks. For example, an intelligent computing task allocation mechanism can be used to continuously monitor performance indicators such as CPU usage, memory usage, and computing queue length of edge devices, evaluate the available computing power of edge devices in real time, decompose risk prediction tasks into multiple subtasks with different computational complexities, and prioritize them according to clinical importance. The execution location of each computing task can be dynamically determined based on the current edge device load and task priority.

[0042] The above-mentioned time series large model lightweight inference module uses advanced time series large model technology to extract high-quality time series feature representation from real-time physiological monitoring data during surgery, providing key input for subsequent risk warning. This module combines lightweight technology with pre-trained time series large models to achieve efficient and accurate extraction of complex physiological signal time series features, while meeting the real-time requirements of the intraoperative environment.

[0043] In actual operation, an open-source pre-trained time series big model can be used. It is pre-trained on large-scale time series data in multiple fields such as finance, meteorology, industrial sensors and medical care, and has the general representation and pattern recognition capabilities of time series data. Through self-supervised learning tasks, the time series big model can understand key semantic information such as trends, periodicity, anomalies and multi-variable correlations in time series data, thereby effectively capturing long-term and short-term time series dependencies, providing a basis for the analysis of real-time physiological monitoring data during surgery.

[0044] On the basis of the open source time series big model, targeted supervised learning fine-tuning can be introduced to make the time series big model better adapt to the characteristics of intraoperative real-time physiological monitoring data. Specifically, firstly, fine-tuning is performed by using a large-scale surgical anesthesia database for medical field migration, including intraoperative monitoring data sets of various types of surgeries, diverse patient data covering different age groups and disease spectrums, and professionally labeled data sets containing marked risk events, so that the time series big model can adapt to the special nature of medical physiological signals; secondly, dedicated optimization of intraoperative monitoring scenarios is performed, and further fine-tuning is performed for specific scenarios of intraoperative monitoring, so that the time series big model can adapt to the data collection frequency and format characteristics of intraoperative monitoring equipment, enhance the ability to recognize specific intraoperative risk patterns, and optimize the sensitivity to anesthesia-related physiological changes; finally, through fine-tuning of multimodal physiological signal integration, the time series big model can effectively process the combination of multiple physiological signals during surgery.

[0045] In addition to introducing supervised learning fine-tuning, integrated reinforcement learning methods can also be used to enhance the performance of large time series models. For example, the two tasks of physiological signal mask prediction and clinical risk assessment can be jointly optimized to achieve a deep understanding of intraoperative physiological monitoring data by the large time series model. By introducing an integrated reinforcement learning framework, it is possible to specifically process multimodal signal inputs such as heart rate, blood pressure, and blood oxygen from intraoperative monitoring equipment, and simultaneously perform dual-task learning of signal reconstruction and disease prediction. In addition, a composite reward function can be used to guide the update of model parameters, and the training stability can be ensured by limiting the update step size. At the same time, a gradient sharing mechanism can be implemented so that the two learning objectives of signal reconstruction and risk assessment can collaboratively adjust the shared feature layer.

[0046] The fine-tuning process of the integrated reinforcement learning framework can derive model variants in multiple specialized directions, forming an expert teacher pool for subsequent multi-teacher distillation. For example, the expert model for physiological signal prediction focuses on time series prediction and anomaly detection of multimodal physiological data, the expert model for risk event identification focuses on early warning of intraoperative complications, the expert model for anesthesia response modeling is good at capturing the dynamic changes of physiological indicators after drug intervention, and the expert for multimodal signal integration is good at correlation analysis across signal channels, etc. These specialized model variants each strengthen their specific capabilities during the fine-tuning process, so that the model can not only adapt to the basic capabilities of the first stage of training, but also more accurately perform downstream tasks such as intraoperative risk warning, anesthetic drug response prediction, and real-time patient monitoring, effectively bridging the gap between general time series representation and surgical risk warning applications. This differentiated specialized fine-tuning strategy lays the foundation for subsequent multi-teacher distillation, enabling lightweight models to inherit expertise in different directions and form comprehensive and efficient intraoperative monitoring capabilities.

[0047] Pre-trained time series large models can also be specially optimized for lightweight performance to improve the real-time performance of risk warning operations. Specifically, the multi-teacher-multi-level distillation framework can be used to integrate multiple specialized fine-tuned time series large models as a teacher set, and transfer their comprehensive knowledge to a single lightweight student model. First, a teacher expert pool is constructed, including model variants with different skill orientations such as physiological signal prediction experts, risk event identification experts, anesthesia reaction modeling experts, and multimodal signal integration experts. During the distillation process, three levels of knowledge transfer are implemented: the output layer distribution distillation transfers the predicted probability distribution of each expert for intraoperative physiological monitoring data; the representation layer feature distillation ensures that the student model obtains the rich feature representation capabilities of the intermediate layer; and the multi-source attention distillation guides students to form an effective attention mechanism for key physiological signal patterns.

[0048] In addition, in order to solve the problem of knowledge conflicts that may arise between multiple teachers, the framework can also introduce a teacher consistency judgment mechanism. During the knowledge distillation process, different teacher models may have inconsistencies in output results due to differences in training data or network structure, and this disagreement will interfere with the student model. The framework analyzes the distribution differences of the output of the teacher model in specific areas and dynamically adjusts the distillation weights in the divergent areas to weaken the influence of the teacher model in the low consistency area, thereby improving the stability and effectiveness of the distillation process. In addition, in order to improve the adaptability of the model to specific tasks, the framework can screen the teacher model based on the sample characteristics and select the optimal subset of teacher models to participate in the training of a specific batch. This process combines the feature distribution, category information, and task requirements of the samples to ensure that the student model can efficiently learn the knowledge most relevant to the current sample.

[0049] Another aspect of lightweight optimization is the structural optimization of the model. For feature extraction of intraoperative physiological monitoring data, the structure of the large time series model can be streamlined and optimized. For example, through influence analysis, the network layers that are critical to feature extraction can be identified, and the full computing power of these network layers can be retained, while other network layers can be simplified; the contribution of different feature channels to intraoperative risk identification can be analyzed, and feature channels with high contribution can be retained, and feature channels with low contribution can be pruned, thereby reducing the computational complexity while maintaining model performance. In addition, a variety of computing acceleration technologies can be used to further improve the efficiency of feature extraction. For example, model quantization can be used to reduce model calculations from 32-bit floating points to 8-bit or 4-bit integer calculations, and mixed precision strategies can be flexibly adopted according to the precision sensitivity of different network layers; operator fusion and optimization can be implemented, the computational graph can be deeply optimized, continuous operations can be merged, and the storage and data transmission overhead of intermediate results can be significantly reduced; parallel computing optimization strategies can be carefully designed for the multi-core or heterogeneous computing characteristics of intraoperative edge devices, and the available hardware resources can be fully utilized to accelerate the feature extraction process, so as to achieve efficient and real-time physiological monitoring signal processing in a limited intraoperative computing environment.

[0050] In one implementation of the embodiment of the present application, the intraoperative data is input into the trained time series large model for time series feature extraction processing to obtain the time series feature representation of the target surgical patient, including: (1) Based on distributed stream computing technology and using a sliding window strategy, the continuously flowing intraoperative data is processed to obtain the physiological signal data stream of the target surgical patient; (2) The physiological signal data stream is input into the large time series model for time series feature extraction and processing to obtain the time series feature representation of the target surgical patient.

[0051] In general, the low-latency distributed stream processing module is based on distributed stream computing technology and uses a sliding window strategy to process the continuously flowing intraoperative data to obtain the physiological signal data stream of the target surgical patient. The physiological signal data stream is input into the time series large model lightweight inference module for processing, and the time series large model is used to extract the time series features of the physiological signal data stream, thereby obtaining the time series feature representation of the target surgical patient. As an example, the surgical patient The physiological signal data flow during surgery can be expressed as:

[0052] in, Indicates patient In time Physiological monitoring characteristics of the patient, such as heart rate, blood oxygen, and blood pressure. Input to the distilled lightweight time series large model By calculation, we can get the time series feature representation , that is .

[0053] 103. Obtain joint feature representation of target surgical patients through pre-constructed multimodal data maps; Through the multimodal data atlas described above, the joint feature representation of the target surgical patient can be obtained , which can be used to characterize the health status of the target surgical patient.

[0054] 104. Predict the risk of cardiovascular and cerebrovascular events in target surgical patients based on their joint feature representation and time series feature representation.

[0055] Combined feature representation of target surgical patients And time series feature representation , the risk of cardiovascular and cerebrovascular events in target surgical patients can be predicted. In order to solve the problem of strong individual heterogeneity of patients, the embodiment of the present application establishes a self-evolving virtual relationship graph of similar patients based on the multimodal data characteristics of surgical patients, dynamically matches similar patient groups, realizes personalized adaptation of model parameters through transfer learning, and dynamically adjusts the patient matching relationship in the real-time warning process, thereby improving the accuracy of cardiovascular and cerebrovascular event warning.

[0056] In one implementation of the embodiment of the present application, the risk of cardiovascular and cerebrovascular events in the target surgical patient is predicted based on the joint feature representation and the time series feature representation of the target surgical patient, including: (1) Generate a similar patient virtual relationship graph based on the joint feature representation of the target surgical patient and the joint feature representation of multiple historical surgical patients. The similar patient virtual relationship graph is used to represent the similarity between the target surgical patient and multiple historical surgical patients. (2) Based on the virtual relationship graph of similar patients, a set of similar patients for the target surgical patient is constructed; (3) Determine the enhanced feature representation of the target surgical patient based on the similarity weight between each patient in the similar patient set and the target surgical patient, as well as the node representation of each patient in the similar patient set under different modalities; (4) Predict the risk of cardiovascular and cerebrovascular events in target surgical patients based on enhanced feature representation and time series feature representation.

[0057] The self-evolving similar patient matching module uses the multimodal data of existing large-scale surgical patients through pre-training to establish a self-evolving similar patient virtual relationship graph. Multimodal data comes from a wide range of sources, including electronic health records, preoperative electrocardiograms, medical images, and time-series physiological monitoring data, which are high-dimensional, heterogeneous, and time-series dependent. Preprocess and extract features of each modality data, and map different types of data to a unified feature space through a multimodal graph fusion representation module to generate structured patient feature nodes, thereby obtaining the above-mentioned multimodal data graph. The multimodal data graph can be used to obtain the joint feature representation of the target surgical patient and the joint feature representation of multiple historical surgical patients. Based on these joint feature representations, the similarity between the health status of each surgical patient can be evaluated, thereby generating a similar patient virtual relationship graph. Based on the similar patient virtual relationship graph, a certain number of historical surgical patients with a high degree of similarity to the target surgical patient can be found, thereby obtaining a set of similar patients of the target surgical patient. Then, based on the similarity weight between each patient in the similar patient set and the target surgical patient, as well as the node representation of each patient in the similar patient set under different modalities, the enhanced feature representation of the target surgical patient is calculated. Finally, based on the enhanced feature representation and temporal feature representation of the target surgical patient, the risk of cardiovascular and cerebrovascular events in the target surgical patient is predicted. The enhanced feature representation and temporal feature representation of the target surgical patient are used as the input of the spatiotemporal risk deduction and early warning module, which analyzes the spatiotemporal interaction mechanism of intraoperative risk, generates future risk evolution trajectories, and triggers hierarchical early warnings based on dynamic intervals, thereby obtaining early warning results for cardiovascular and cerebrovascular events in the target surgical patient.

[0058] As an example, suppose the target surgical patient is patient At the beginning of the operation, the patient The real-time physiological monitoring data of patients is far less than that of the existing historical patient groups. In this case, knowledge transfer is carried out by using the historical risk patterns of similar patients. The enhanced feature representation of can be calculated by the following formula:

[0059] in, Is a patient The enhanced feature representation of Is a patient A collection of similar patients, Similar patients Node representation in different modalities, that is, patients Multimodal fusion features, Indicates patient With patients The similarity weight of .

[0060] In one implementation of the embodiment of the present application, a virtual relationship graph of similar patients is generated according to the joint feature representation of the target surgical patient and the joint feature representations of multiple historical surgical patients, including: (1) Based on the joint feature representation of the target surgical patient and the joint feature representation of multiple historical surgical patients, the similarity weights between the target surgical patient and multiple historical surgical patients are calculated; (2) The joint feature representation of the target surgical patient and the joint feature representation of multiple historical surgical patients are used as nodes of the virtual relationship graph of similar patients. The edges between the nodes of the virtual relationship graph of similar patients are constructed according to the similarity weights between the target surgical patient and multiple historical surgical patients, and the edges whose corresponding similarity weights are less than the set threshold are deleted, thereby obtaining the virtual relationship graph of similar patients.

[0061] According to the joint feature representation of the target surgical patient and the joint feature representation of multiple historical surgical patients, the similarity weights between the target surgical patient and multiple historical surgical patients can be calculated. Specifically, the weighted cosine similarity can be used for measurement, as shown in the following formula:

[0062] in, Indicates patient and patients The similarity weight between and Respectively, patients and The multimodal feature node is also the joint feature representation. It is a trainable similarity transformation matrix used to adjust the weights of different features. In the pre-training stage, the parameters of the similarity transformation matrix W can be optimized through supervised learning. The training data comes from large-scale patient data and its known similarity labels (such as common symptoms, treatment responses, or health status groups, etc.). Through supervised learning, the model can capture the deep similarity relationship between patients and accurately reflect the similarity between different patients in pathological characteristics, health status, and treatment response. In order to further improve the generalization ability of the model, contrastive learning and multi-task learning strategies can also be introduced in the pre-training stage. On the one hand, through contrastive learning, the model learns how to more effectively distinguish the feature distribution of similar patients from dissimilar patients, thereby enhancing the ability to distinguish boundary relationships. On the other hand, multi-task learning allows the model to focus on the potential similarities of different symptoms or surgical types at the same time, making it adaptable to multi-scenario application needs.

[0063] After completing the calculation of the similarity weight, the joint feature representation of the target surgical patient and the joint feature representation of multiple historical surgical patients are used as nodes in the virtual relationship graph of similar patients. The edges between the nodes of the virtual relationship graph of similar patients are constructed according to the similarity weights between the target surgical patient and multiple historical surgical patients, and the nodes with the corresponding similarity weights less than the set threshold are deleted. That is, only the edges with similarity weight greater than the threshold are retained. Patient pairs are sparsely graphed to obtain a virtual relationship graph of similar patients. . Similar patients virtual relationship diagram It can be expressed by the following formula:

[0064] in A set of feature nodes representing different surgical patients, Represents a set of similar relationships between surgical patients. The virtual relationship graph of similar patients can intuitively present the similarity structure of the patient group and provide a basis for subsequent dynamic matching and risk prediction. In order to maintain computational efficiency, edges with low similarity weights can be removed regularly to ensure that each patient node is only connected to the most relevant similar patients, thus constructing a similar patient set .

[0065] In an implementation of the embodiment of the present application, after generating a similar patient virtual relationship graph according to the joint feature representation of the target surgical patient and the joint feature representations of multiple historical surgical patients, the method further includes: When intraoperative data changes, the virtual relationship graph of similar patients is updated according to the updated temporal feature representation.

[0066] Since the intraoperative physiological monitoring characteristics of patients change dynamically, the self-evolving similar patient matching module can use a self-evolving update mechanism to adjust the graph structure of the virtual relationship graph of similar patients in real time. When the intraoperative data of a patient changes, its temporal feature representation will also be updated. Based on the self-evolving update mechanism, this module updates the virtual relationship graph of similar patients accordingly according to the updated temporal feature representation.

[0067] As an example, suppose the target surgical patient is patient , in patients During surgery, when the patient's status changes, the node's features are updated synchronously, as shown in the following formula:

[0068] in, Indicates patient Updated physiological monitoring features, Represents the large time series model obtained after distillation in the low-latency distributed stream processing module. is the fusion weight of historical features and new data, The node features before The node features after .

[0069] After that, the similarity weights need to be recalculated to update the weights of the edges in the virtual relationship graph of similar patients, as shown in the following formula:

[0070] in, Indicates updated patient and patients The similarity weight between Indicates the patient before update and patients The similarity weight between is the weight of the fusion of the old similarity weight and the similarity weight after data update in the virtual relationship graph of similar patients. , then remove the corresponding edge in the virtual relationship graph of similar patients, otherwise add the corresponding edge in the virtual relationship graph of similar patients, thereby completing the update of the virtual relationship graph of similar patients.

[0071] The spatiotemporal risk deduction and warning module described above is based on the spatiotemporal Transformer architecture. It comprehensively analyzes the spatial characteristics of the virtual relationship graph of similar patients and the temporal changes of intraoperative physiological monitoring signals, captures the interaction between the evolution trend of individual patient risks and group patterns, and mines the risk evolution trajectory of individual patients and their interaction in similar patient groups, thereby predicting the evolution trend of cardiovascular and cerebrovascular events and generating graded warning signals.

[0072] In one implementation of the embodiment of the present application, the risk of cardiovascular and cerebrovascular events in a target surgical patient is predicted based on the enhanced feature representation and the time series feature representation, including: (1) The enhanced feature representation and the temporal feature representation are input into a deep learning model that has been trained to assess spatiotemporal risk, and the spatiotemporal risk representation of the target surgical patient at the current moment is obtained; (2) Based on the spatiotemporal risk representation at the current moment, predict the spatiotemporal risk representation of the target surgical patient at multiple moments in the future; (3) Based on the spatiotemporal risk characterization of the target surgical patient at multiple moments in the future, determine the risk level of the target surgical patient for cardiovascular and cerebrovascular events at multiple moments in the future.

[0073] Each surgical patient is not only affected by his own temporal physiological monitoring, but also by the risk patterns of similar patient groups. , the enhanced feature representation of the target surgical patient can be calculated , the enhanced features of the target surgical patient are represented as And time series feature representation The input is processed into a deep learning model that has been trained to assess spatiotemporal risk, and the spatiotemporal risk representation of the target surgical patient at the current moment can be obtained. In actual operation, the deep learning model can be trained based on the spatiotemporal Transformer architecture using time series feature data and enhanced feature data with risk numerical labels, and can learn the relationship between time series features and spatial features. The processing process of the deep learning model can be expressed by the following formula:

[0074] in, represents the spatiotemporal Transformer model, Is a patient The spatiotemporal risk characterization at the current moment can be used for the next step of risk deduction.

[0075] Next, the future risk deduction can be based on the spatiotemporal risk representation at the current moment , gradually calculate the patient The spatial and temporal risk representation at multiple moments in the future. Here we can define the risk state transfer function , this function borrows the idea of ​​"state transfer function" in reinforcement learning and applies it to risk prediction, inferring the future risk assessment value from the current risk assessment value. This function is used for multi-step prediction, as shown in the following formulas:

[0076]

[0077] ︙

[0078] in, Indicates the time of use Spatiotemporal risk characterization , predict the time to obtain Spatiotemporal risk characterization , and so on, is the prediction time window.

[0079] Finally, according to the patient Spatiotemporal risk characterization at multiple moments in the future to identify patients The risk level of cardiovascular and cerebrovascular events at multiple times in the future. Spatiotemporal risk characterization at multiple moments in the future can form a predicted risk trajectory , based on the risk trajectory and the various risk thresholds set , you can determine the patient The risk level of cardiovascular and cerebrovascular events at multiple moments in the future is shown in the following formula:

[0080] in, Indicates patient In the future The risk level of cardiovascular and cerebrovascular events at any time can include low risk, medium risk and high risk.

[0081] It can be seen that through the spatiotemporal risk deduction and early warning module, the future multi-step risk trajectory of surgical patients can be predicted in real time, and real-time intraoperative early warnings for cardiovascular and cerebrovascular events can be dynamically generated, providing medical staff with forward-looking decision-making support, thereby improving the safety of the surgical process.

[0082] The technical solution of the embodiment of the present application pre-constructs a multimodal data map based on the multimodal data of multiple historical surgical patients and the preoperative data of the target surgical patient; when the target surgical patient undergoes surgery, the intraoperative data of the target surgical patient will be obtained in real time, and then the intraoperative data will be input into the trained time series large model for time series feature extraction processing to obtain the time series feature representation of the target surgical patient, and the joint feature representation that can be used to characterize the health status of the target surgical patient is obtained through the multimodal data map; finally, according to the joint feature representation and the time series feature representation, the risk of cardiovascular and cerebrovascular events in the target surgical patient is predicted. The above process uses the multimodal data map to fuse heterogeneous preoperative data and intraoperative data, realizes the deep semantic fusion and dynamic association modeling of multimodal data, can fully explore the correlation between cross-modal data, solve the problem of data feature fragmentation, and thus improve the accuracy of risk prediction of cardiovascular and cerebrovascular events in surgical patients.

[0083] In summary, the embodiment of the present application proposes a real-time intraoperative warning model with a framework of "multimodal data fusion-low-latency computing-pre-training drive-personalized dynamic modeling-space-time risk deduction". By using a multimodal graph fusion representation module, it can solve the problem of multi-source heterogeneous data integration and provide stable and robust multimodal joint information support for risk assessment; by using a low-latency distributed stream processing module, it can realize rapid distributed processing of patient time-series physiological monitoring data to meet the requirements of real-time and high efficiency of warning; by using a lightweight reasoning module of a large time-series model, it can realize efficient extraction and accurate modeling of complex time-series physiological characteristics; by using a self-evolving similar patient matching module, by constructing a virtual relationship graph of similar patients, it can dynamically capture the relationship between the patient's intraoperative status and group characteristics to solve the problem of strong individual differences in surgical patients; by using a space-time risk deduction and warning module, it can jointly analyze the spatial correlation between the time-series physiological data of individual patients and similar patient groups, capture the evolution trend of patient risks based on space-time joint representation, and realize future multi-step risk prediction through a risk state transfer function.

[0084] It should be understood that the size of the serial numbers of the steps in the above embodiments does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present application.

[0085] The above mainly describes a real-time early warning method for cardiovascular and cerebrovascular events during the perioperative period of non-cardiac surgery. The following will describe a real-time early warning device for cardiovascular and cerebrovascular events during the perioperative period of non-cardiac surgery.

[0086] See also Figure 4 , shows a real-time warning device for cardiovascular and cerebrovascular events during the perioperative period of non-cardiac surgery provided by an embodiment of the present application, comprising: An intraoperative data acquisition module 401 is used to acquire intraoperative data of a target surgical patient during the surgical procedure of the target surgical patient; The time series feature extraction module 402 is used to input the intraoperative data into the trained time series large model for time series feature extraction processing to obtain the time series feature representation of the target surgical patient; A joint feature acquisition module 403 is used to acquire a joint feature representation of a target surgical patient through a pre-constructed multimodal data atlas; wherein the multimodal data atlas is constructed based on multimodal data of multiple historical surgical patients and pre-operative data of the target surgical patient, and the joint feature representation of the target surgical patient is used to characterize the health status of the target surgical patient; The risk prediction module 404 is used to predict the risk of cardiovascular and cerebrovascular events in the target surgical patient based on the joint feature representation and the time series feature representation of the target surgical patient.

[0087] In one implementation of the embodiment of the present application, the real-time warning device for cardiovascular and cerebrovascular events during the perioperative period of non-cardiac surgery further includes: A multimodal coding module is used to extract and encode the preoperative data of the target surgical patient to obtain multimodal features; The spatiotemporal correlation module is used to perform spatiotemporal correlation synchronization and alignment processing on multimodal features; The node representation construction module is used to construct the node representation of the target surgical patient in each different modality based on the multi-modal features processed by spatiotemporal correlation synchronization alignment; An association weight determination module is used to determine the association weight of the node representation of the target surgical patient in each different modality according to the similarity between the node representation of multiple historical surgical patients in each different modality and the node representation of the target surgical patient in each different modality; wherein the node representation of multiple historical surgical patients in each different modality is constructed based on the multimodal data of multiple historical surgical patients; The node representation fusion module is used to fuse the node representations of the target surgical patient in different modalities according to the corresponding association weights to obtain the joint feature representation of the target surgical patient.

[0088] In one implementation of the embodiment of the present application, the risk prediction module includes: A virtual relationship graph generating unit, used for generating a similar patient virtual relationship graph according to the joint feature representation of the target surgical patient and the joint feature representation of multiple historical surgical patients, wherein the similar patient virtual relationship graph is used for representing the degree of similarity between the target surgical patient and multiple historical surgical patients; A similar patient set construction unit, used for constructing a similar patient set of a target surgical patient according to a similar patient virtual relationship graph; an enhanced feature representation determination unit, used to determine the enhanced feature representation of the target surgical patient according to the similarity weight between each patient in the similar patient set and the target surgical patient, and the node representation of each patient in the similar patient set under different modalities; The risk prediction unit is used to predict the risk of cardiovascular and cerebrovascular events in target surgical patients based on enhanced feature representation and time series feature representation.

[0089] In one implementation of the embodiment of the present application, the virtual relationship diagram generating unit includes: A similarity weight calculation subunit, used for calculating the similarity weights between the target surgical patient and the multiple historical surgical patients according to the joint feature representation of the target surgical patient and the joint feature representation of the multiple historical surgical patients; The virtual relationship graph generation subunit is used to use the joint feature representation of the target surgical patient and the joint feature representation of multiple historical surgical patients as nodes of the virtual relationship graph of similar patients, construct edges between the nodes of the virtual relationship graph of similar patients according to the similarity weights between the target surgical patient and multiple historical surgical patients, and delete the edges whose corresponding similarity weights are less than a set threshold, so as to obtain the virtual relationship graph of similar patients.

[0090] In one implementation of the embodiment of the present application, the risk prediction unit includes: A moment risk assessment subunit is used to input the enhanced feature representation and the time series feature representation into a trained deep learning model for assessing spatiotemporal risk for processing, so as to obtain the spatiotemporal risk representation of the target surgical patient at the current moment; A future risk prediction subunit is used to predict the spatiotemporal risk representation of the target surgical patient at multiple future moments based on the spatiotemporal risk representation at the current moment; The risk level determination subunit is used to determine the risk level of cardiovascular and cerebrovascular events in the target surgical patient at multiple moments in the future based on the spatiotemporal risk characterization of the target surgical patient at multiple moments in the future.

[0091] In one implementation of the embodiment of the present application, the risk prediction module further includes: The virtual relationship graph updating unit is used to update the virtual relationship graph of similar patients according to the updated time series feature representation when the intraoperative data changes.

[0092] In one implementation of the embodiment of the present application, the time series feature extraction module includes: A distributed stream processing unit is used to process the continuously flowing intraoperative data based on distributed stream computing technology and adopt a sliding window strategy to obtain a physiological signal data stream of a target surgical patient; The timing feature extraction unit is used to input the physiological signal data stream into the timing large model for timing feature extraction processing to obtain the timing feature representation of the target surgical patient.

[0093] An embodiment of the present application also provides a computer-readable storage medium, which stores a computer program. When the computer program is executed by a processor, it implements the real-time warning method for perioperative cardiovascular and cerebrovascular events in non-cardiac surgery described in any of the above embodiments.

[0094] An embodiment of the present application also provides a computer program product. When the computer program product is executed on an electronic device, the electronic device executes the real-time early warning method for perioperative cardiovascular and cerebrovascular events in non-cardiac surgery as described in any of the above embodiments.

[0095] Figure 5is a schematic diagram of an electronic device provided by an embodiment of the present application. Figure 5 As shown, the electronic device 5 of this embodiment includes: a processor 50, a memory 51, and a computer program 52 stored in the memory 51 and executable on the processor 50. When the processor 50 executes the computer program 52, the steps in the above-mentioned embodiments of the method for real-time early warning of cardiovascular and cerebrovascular events during perioperative period of non-cardiac surgery are implemented, for example Figure 1 Alternatively, when the processor 50 executes the computer program 52, the functions of each module / unit in the above-mentioned device embodiments are realized, for example, Figure 4 Functions of modules 401 - 404 of the illustrated apparatus.

[0096] The computer program 52 may be divided into one or more modules / units, which are stored in the memory 51 and executed by the processor 50 to complete the present application. The one or more modules / units may be a series of computer program instruction segments capable of completing specific functions, which are used to describe the execution process of the computer program 52 in the electronic device 5.

[0097] The processor 50 may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or any conventional processor, etc.

[0098] The memory 51 may be an internal storage unit of the electronic device 5, such as a hard disk or memory of the electronic device 5. The memory 51 may also be an external storage device of the electronic device 5, such as a plug-in hard disk, a smart media card (SMC), a secure digital (SD) card, a flash card, etc. equipped on the electronic device 5. Further, the memory 51 may also include both an internal storage unit and an external storage device of the electronic device 5. The memory 51 is used to store the computer program and other programs and data required by the electronic device. The memory 51 may also be used to temporarily store data that has been output or is to be output.

[0099] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the present application implements all or part of the processes in the above-mentioned embodiment method, and can also be completed by instructing the relevant hardware through a computer program. The computer program can be stored in a computer-readable storage medium, and the computer program can implement the steps of the above-mentioned various method embodiments when executed by the processor. Among them, the computer program includes computer program code, and the computer program code can be in source code form, object code form, executable file or some intermediate form. The computer-readable medium may include: any entity or device capable of carrying the computer program code, recording medium, U disk, mobile hard disk, disk, optical disk, computer memory, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), electric carrier signal, telecommunication signal and software distribution medium. It should be noted that the content contained in the computer-readable medium can be appropriately increased or decreased according to the requirements of legislation and patent practice in the jurisdiction. For example, in some jurisdictions, according to legislation and patent practice, computer-readable media do not include electric carrier signals and telecommunication signals.

[0100] The embodiments described above are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, a person skilled in the art should understand that the technical solutions described in the aforementioned embodiments may still be modified, or some of the technical features may be replaced by equivalents. Such modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present application, and should all be included in the protection scope of the present application.

Claims

1. A real-time early warning method for cardiovascular and cerebrovascular events during the perioperative period of non-cardiac surgery, characterized in that: include: During the operation of the target surgical patient, obtaining intraoperative data of the target surgical patient; Inputting the intraoperative data into the trained time series large model to perform time series feature extraction processing to obtain the time series feature representation of the target surgical patient; Obtaining a joint feature representation of the target surgical patient through a pre-constructed multimodal data atlas; wherein the multimodal data atlas is constructed based on multimodal data of multiple historical surgical patients and pre-operative data of the target surgical patient, and the joint feature representation of the target surgical patient is used to characterize the health status of the target surgical patient; The risk of cardiovascular and cerebrovascular events occurring in the target surgical patient is predicted based on the joint feature representation and the time series feature representation of the target surgical patient.

2. The method according to claim 1, characterized in that The process of constructing the multimodal data map includes: Performing feature extraction and encoding processing on the preoperative data of the target surgical patient to obtain multimodal features; Performing spatiotemporal correlation synchronization alignment processing on the multimodal features; Based on the multimodal features processed by spatiotemporal correlation synchronization alignment, constructing node representations of the target surgical patient in each different modality; Determine the association weight of the node representation of the target surgical patient in each different modality according to the similarity between the node representation of the multiple historical surgical patients in each different modality and the node representation of the target surgical patient in each different modality; wherein the node representation of the multiple historical surgical patients in each different modality is constructed based on the multimodal data of the multiple historical surgical patients; The node representations of the target surgical patient in each different modality are fused according to the corresponding association weights to obtain a joint feature representation of the target surgical patient.

3. The method according to claim 2, characterized in that The step of predicting the risk of cardiovascular and cerebrovascular events in the target surgical patient according to the joint feature representation and the time series feature representation of the target surgical patient includes: Generate a similar patient virtual relationship graph according to the joint feature representation of the target surgical patient and the joint feature representation of the plurality of historical surgical patients, wherein the similar patient virtual relationship graph is used to characterize the degree of similarity between the target surgical patient and the plurality of historical surgical patients; Constructing a similar patient set for the target surgical patient according to the similar patient virtual relationship graph; Determine an enhanced feature representation of the target surgical patient according to a similarity weight between each patient in the similar patient set and the target surgical patient, and a node representation of each patient in the similar patient set under different modalities; The risk of cardiovascular and cerebrovascular events in the target surgical patient is predicted based on the enhanced feature representation and the time series feature representation.

4. The method according to claim 3, characterized in that The generating a similar patient virtual relationship graph according to the joint feature representation of the target surgical patient and the joint feature representations of the plurality of historical surgical patients comprises: Calculating similarity weights between the target surgical patient and the plurality of historical surgical patients according to the joint feature representation of the target surgical patient and the joint feature representation of the plurality of historical surgical patients; The joint feature representation of the target surgical patient and the joint feature representation of the multiple historical surgical patients are used as nodes of the similar patient virtual relationship graph. The edges between the nodes of the similar patient virtual relationship graph are constructed according to the similarity weights between the target surgical patient and the multiple historical surgical patients, and the corresponding edges whose similarity weights are less than a set threshold are deleted, thereby obtaining the similar patient virtual relationship graph.

5. The method according to claim 3, characterized in that The step of predicting the risk of cardiovascular and cerebrovascular events in the target surgical patient according to the enhanced feature representation and the time series feature representation includes: Inputting the enhanced feature representation and the temporal feature representation into a trained deep learning model for assessing spatiotemporal risk for processing, to obtain a spatiotemporal risk representation of the target surgical patient at the current moment; Based on the spatiotemporal risk representation at the current moment, predicting the spatiotemporal risk representation of the target surgical patient at multiple moments in the future; The risk level of the target surgical patient for cardiovascular and cerebrovascular events at multiple moments in the future is determined based on the spatiotemporal risk characterization of the target surgical patient at multiple moments in the future.

6. The method according to claim 3, characterized in that After generating a similar patient virtual relationship graph according to the joint feature representation of the target surgical patient and the joint feature representations of the plurality of historical surgical patients, the method further includes: When the intraoperative data changes, the similar patient virtual relationship diagram is updated according to the updated temporal feature representation.

7. The method according to any one of claims 1 to 6, characterized in that: The step of inputting the intraoperative data into a trained time series large model for time series feature extraction processing to obtain a time series feature representation of the target surgical patient includes: Based on the distributed stream computing technology and using the sliding window strategy, the continuously flowing intraoperative data is processed to obtain the physiological signal data stream of the target surgical patient; The physiological signal data stream is input into the large time series model for time series feature extraction processing to obtain the time series feature representation of the target surgical patient.

8. A real-time warning device for cardiovascular and cerebrovascular events during the perioperative period of non-cardiac surgery, characterized in that: include: An intraoperative data acquisition module, used to acquire intraoperative data of a target surgical patient during the surgical procedure of the target surgical patient; A time series feature extraction module, used for inputting the intraoperative data into a trained time series large model for time series feature extraction processing to obtain a time series feature representation of the target surgical patient; A joint feature acquisition module, used to acquire a joint feature representation of the target surgical patient through a pre-constructed multimodal data atlas; wherein the multimodal data atlas is constructed based on multimodal data of multiple historical surgical patients and pre-operative data of the target surgical patient, and the joint feature representation of the target surgical patient is used to characterize the health status of the target surgical patient; The risk prediction module is used to predict the risk of cardiovascular and cerebrovascular events in the target surgical patient based on the joint feature representation and the time series feature representation of the target surgical patient.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the real-time early warning method for perioperative cardiovascular and cerebrovascular events in non-cardiac surgery as described in any one of claims 1 to 7 is implemented.

10. A computer program product, characterized in that When the computer program product is executed on an electronic device, the electronic device executes the real-time early warning method for perioperative cardiovascular and cerebrovascular events in non-cardiac surgery as described in any one of claims 1 to 7.

Images