A multi-modal based medical rescue scene time series prediction method and device

Abstract

This invention belongs to the field of artificial intelligence algorithms. To address the challenges of discrepancies and alignment difficulties in multimodal data and dimensional semantic representation in existing technologies, as well as the contradiction between real-time decision-making requirements and computational complexity in fusion, this invention proposes a multimodal time series prediction method and device for medical emergency scenarios. By acquiring multimodal time series data from medical emergency scenarios, and employing three types of features—digital, visual, and semantic—time series prediction is performed using a multimodal model. The fusion of these three modalities allows for the assessment of patient conditions or vital signs in the emergency scenario, predicting potential cardiac issues such as premature ventricular contractions (PVCs) or other heart problems in the future. This real-time prediction provides support to on-site personnel, enabling proactive preparation.

Description

Technical Field

[0001] This invention belongs to the field of artificial intelligence algorithms, specifically, it relates to a method and device for time series prediction in multimodal medical rescue scenarios. Background Technology

[0002] Time series forecasting in medical emergency scenarios is a crucial means to improve emergency response efficiency and the quality of medical care. Its core objective is to provide data support for rapid decision-making by accurately predicting the trends of patients' vital signs, the risk of disease progression, and the demand for emergency resources. Medical emergency scenario data exhibits multimodal heterogeneity, including image data in two-dimensional or three-dimensional pixel matrix form, physiological data in one-dimensional time series form, and text data such as medical records and symptom descriptions—discrete symbolic sequences. The differences in dimensionality, sampling frequency, and data structure among these different modalities present challenges to multimodal time series forecasting.

[0003] Different modalities describe medical concepts differently. Images rely on visual features to provide lesion and pathological states, text provides diagnostic conclusions through natural language, and vital signs describe physiological dynamics through data amplitude. The inconsistency of modalities further hinders the effective unification of feature space and restricts prediction accuracy.

[0004] In emergency medical scenarios, employing multimodal models for time series forecasting avoids the inaccuracy and lack of reference value of predictions based on single data sources. While medical multimodal models utilize a wide range of data sources, and while they offer diagnostic assistance and clinical decision support capabilities, they lack corresponding support for time series forecasting.

[0005] Current approaches to time series prediction in multimodal medical emergency scenarios include: adaptive fusion mechanisms, hierarchical fusion strategies, improved time series models adapted to the characteristics of medical data, LLM and spatiotemporal knowledge graphs enabling cross-modal reasoning and resource scheduling optimization, and the gradual improvement of data preprocessing techniques and interpretability design, effectively enhancing the accuracy of disease prediction and resource allocation. However, due to differences in multimodal data and dimensional semantic representations, alignment is difficult, and there is a contradiction between real-time decision-making requirements and computational complexity. The scarcity and difficulty in labeling medical emergency data limit the model's generalization ability. Insufficient model interpretability also poses a challenge for practical application in emergency scenarios. Therefore, researching multimodal time series prediction methods to overcome the shortcomings of existing technologies has become an important issue that needs to be addressed in the field of artificial intelligence for medical emergency care. Summary of the Invention

[0006] To address the aforementioned problems, the objective of this invention is to perform time-series prediction using monitoring values ​​derived from medical emergency scenarios, employing three types of features: digital features, visual features, and semantic features, through a multimodal model. The fusion of these three modalities aims to assess the patient's condition or vital signs in emergency scenarios, providing real-time predictions to support on-site care.

[0007] The first aspect of this invention proposes a time series prediction method for medical rescue scenarios based on multimodality, comprising: Obtain multimodal time series of electrocardiograms in medical rescue scenarios, retrieve historical datasets of electrocardiograms from the database, and annotate the historical datasets; The autocorrelation function is used to determine the first period and the first time series within the first period of the historical dataset; key time series segments are determined based on the first period. The historical dataset is vectorized, and the vectorized historical dataset is aligned to obtain the diagnostic names corresponding to the multimodal time series segments. For multimodal time series of electrocardiograms in medical rescue scenarios, the cosine similarity value is calculated using autocorrelation function and cosine similarity. The cosine similarity values ​​are sorted to obtain time series segments of labeled historical datasets. Based on key time series segments, diagnosis names, and time series segments from historical datasets obtained through sorting, determine short-cycle prediction time series; The Informer model method is used to obtain long-period predicted time series for multimodal time series. Based on short-cycle and long-cycle prediction time series, the final predicted time series of medical rescue scenarios is obtained, and heart rate data for a future period is obtained.

[0008] Preferably, determining the first period and the first time series within the first period using an autocorrelation function on a historical dataset specifically includes: The autocorrelation function is used on the time series of the historical dataset to calculate the similarity between the time series and itself at different lag time points, thereby obtaining the periodic value; Lag Similarity is calculated at time points, with the first non-zero significant similarity corresponding to a lag. The time point is the first period, and the first time series within the first period is obtained based on the first period.

[0009] Preferably, the key time series segments are determined based on the first period as follows: For a connection time consisting of multiple first periods, calculate the extreme values ​​of the time series of the historical dataset within the connection time, including the maximum and minimum values, and the time series of the periods containing the maximum and minimum values, and label them as key time series segments.

[0010] Preferably, the alignment process for the vectorized historical dataset includes: Attention scores were calculated using the Bahdanau attention function on the time series data of historical datasets. Calculate attention weights based on attention scores; The attention weights are weighted and averaged to obtain the context vector; the attention score, attention weights, and context vector are input into the Transformer decoder to output the time series. Align the time series output by the Transformer decoder with the historical dataset to obtain the diagnostic name corresponding to the output time series.

[0011] Preferably, determining the short-cycle forecast time series includes the following steps: From the time series segments of the labeled historical dataset obtained by sorting the cosine similarity values, the time series within the period corresponding to the maximum cosine similarity is selected as the first baseline for predicting the time series. Based on the period in which the key time series segment is located, determine the offset of the pathological segment of the electrocardiogram segment in that period, and determine the second prediction time series baseline based on the offset. Calculate the cosine distance between the first and second predicted time series, and select the one with the smallest cosine distance as the short-period predicted time series.

[0012] Preferably, based on the period in which the key time series segment is located, the offset of the pathological segment of the electrocardiogram band in that period is determined, and the second predictive time series baseline is determined based on the offset, specifically: Based on the period in which the key time series segment is located, determine the offset of the pathological segment on the electrocardiogram band of that period; based on the offset, determine the corresponding pathological name; then based on the corresponding pathological name, obtain the time series of the corresponding labeled historical dataset; and use the time series of the corresponding labeled historical dataset as the second reference for the predicted time series.

[0013] Preferably, the short-cycle prediction time series and the long-cycle prediction time series are mixed and weighted to obtain the final predicted time series of medical rescue scenarios.

[0014] A second aspect of the present invention provides a multimodal medical rescue scenario time series prediction device, which is constructed based on the multimodal medical rescue scenario time series prediction method described above, and specifically includes the following modules: The data acquisition module is used to acquire multimodal time series of electrocardiograms in medical rescue scenarios, retrieve historical datasets of electrocardiograms from the database, and annotate the historical datasets; The key time series segment acquisition module determines the first period and the first time series within the first period by applying an autocorrelation function to the historical dataset; and determines the key time series segments based on the first period. The vectorization module performs vectorization processing on the historical dataset, and then aligns the vectorized historical dataset to obtain the diagnostic names corresponding to the multimodal time series segments. The cosine similarity calculation module calculates the cosine similarity value for multimodal time series of electrocardiograms in medical rescue scenarios using autocorrelation function and cosine similarity, sorts the cosine similarity values, and obtains time series segments of labeled historical datasets. The short-cycle prediction time series acquisition module determines the short-cycle prediction time series based on key time series segments, diagnosis names, and time series segments of historical datasets obtained through sorting. The long-term forecast time series acquisition module uses the Informer model method to obtain long-term forecast time series for multimodal time series. The medical rescue scenario time series prediction module, based on short-period and long-period prediction time series, obtains the final predicted medical rescue scenario time series and obtains heart rate data for a future period.

[0015] Compared with the prior art, the beneficial effects of the present invention are: This invention effectively combines short-term and long-term time series prediction. With the support of actual data and labeled text, the obtained time series prediction values ​​have strong reverse tracing value and can be used for various time series analysis tasks.

[0016] This invention utilizes monitoring values ​​derived from medical emergency scenarios, employing three types of features—digital, visual, and semantic—to perform time-series prediction through a multimodal model. The fusion of these three modalities allows for the assessment of patient conditions or vital signs in emergency scenarios, predicting potential cardiac issues such as premature ventricular contractions (PVCs) or other heart problems in the future. This real-time prediction provides support to on-site personnel, enabling proactive preparation. Attached Figure Description

[0017] The accompanying drawings are provided to further illustrate the invention and form part of the specification. They are used together with the embodiments of the invention to explain the invention and do not constitute a limitation thereof.

[0018] In the attached diagram: Appendix Figure 1: Flowchart of the method of the present invention.

[0019] Appendix Figure 2 Time curve of cardiac electrical signal.

[0020] Appendix Figure 3 : Prediction results diagram. Detailed Implementation

[0021] The following is in conjunction with the appendix Figures 1-3 The preferred embodiments of the present invention will be described herein. It should be understood that the preferred embodiments described herein are for illustration and explanation only and are not intended to limit the present invention.

[0022] Example 1: like Figure 1 The diagram shown is a flowchart of the method of this invention. Based on this flowchart, a time series prediction method for medical rescue scenarios based on multimodality is proposed, including: S1. Obtain historical ECG datasets from the database and divide the historical datasets into training and test sets according to a certain ratio. For example, the ratio of training to test sets is 8:2.

[0023] The training and test sets are labeled. Specifically, the diagnostic names in the training and test sets are first labeled according to features (such as voltage increase), and then the labeled diagnostic names are divided according to descriptive terms (voltage) and qualitative terms (increase).

[0024] The multimodal time series Ts of electrocardiograms under medical monitoring scenarios are obtained through medical equipment. The multimodal time series Ts also corresponds to the curve graph Cu and the semantic name of the time series Se.

[0025] S2. Use the autocorrelation function on the historical dataset to determine the first period and the first time series within the first period, and determine the key time series segments based on the first period. Details are as follows: S2.1. Apply the autocorrelation function to the time series of the historical dataset to calculate the similarity between the time series and itself at different lag times, thereby obtaining the periodic value. K .

[0026] The formula for the autocorrelation function is: (1) The similarity between a multimodal time series and itself at different lag times; At a certain point in time The observed values.

[0027] At a certain point in time The observed value, i.e., the lag The value of the period.

[0028] The mean of the entire multimodal time series. : The total length of the multimodal time series.

[0029] Lag Time point (e.g.) )calculate The first non-zero significant The corresponding lag The time point is the first cycle. That is, the period length The first time series within the first period is obtained based on the first period. (Time series within the period) Starting from the time series (The time series is obtained by dividing the length into the corresponding positions).

[0030] S2.2, Connection time for multiple first cycles , n Given a positive integer, calculate the extreme values ​​of the time series of the historical dataset within the connection time, including the maximum and minimum values. The time series of the periods containing the maximum and minimum values ​​are labeled as key time series segments, thus obtaining the key time series segments of the maximum value period and the key time series segments of the minimum value period, as well as the corresponding maximum value period and minimum value period, respectively.

[0031] S3. Vectorize the historical dataset, align the vectorized dataset, and obtain the diagnostic names corresponding to the multimodal time series segments. Details are as follows: S3.1 Vectorize the time series segments and diagnosis names of the historical dataset respectively; S3.2. An attention mechanism is used to align the vectorized multimodal time series segments with their diagnostic names, thereby obtaining the diagnostic names corresponding to the multimodal time series segments. The specific method is as follows: S3.2.1 The time series of the historical dataset is as follows: in, The time series of the historical dataset is in the 1st The hidden state at each time step It is the time series length of the historical dataset.

[0032] S3.2.2 Calculate attention scores using the Bahdanau attention function. : (2) It is the Transformer decoder that generates the first... The hidden state before each word.

[0033] This represents the weight vector for the weighted summation; This represents the weight matrix that projects the hidden state onto a dimension aligned with the attention space; This represents the weight matrix that projects time-series features into the attention space; Indicates attention weight; Indicates transpose; S3.2.3, in generating the first When the word is 1, the time series signal is 1. Attention weights at each time step The calculation formula is: (3) in, Indicates the first The word in the first Attention score of each step Indicates the first The word in the first Attention score of each step Indicates the Transformer encoder position index. This indicates transpose.

[0034] Attention weight Perform a weighted average to obtain the context vector. : (4) in, For the first The hidden state at each time step This indicates transpose.

[0035] S3.2.4, Attention Score Attention weight and context vector The input is fed into the Transformer decoder, and the output of the Transformer decoder is a time series. for: in, This is the time series output from the previous step of the Transformer decoder.

[0036] S3.2.4. Convert the time series output by the Transformer decoder. Align the diagnostic names with those in the historical dataset to obtain the corresponding diagnostic names.

[0037] S4. Obtain the multimodal time series of electrocardiograms under medical monitoring scenarios through medical equipment, and calculate the second period according to the autocorrelation function of S2. And the second cycle Time series within Cosine similarity is used to calculate time series. With S2 time series cosine similarity value : (5) The cosine similarity values ​​are sorted by size to obtain the time series similarity ranking. The ranking is the time series category of the multimodal time series of electrocardiograms in the medical rescue scenario. The first ranked value is the most likely time series category. Once this time series category is clear, the time series segments of the historical dataset labeled S1 are obtained according to the time series category.

[0038] S5. Based on the key time series segments from S2, the diagnostic names obtained in S3, and the time series segments from the historical dataset obtained in S4, determine the short-cycle prediction time series. Specifically, the steps include the following: S5.1 Select the period corresponding to the maximum cosine similarity from the time series segments of the historical dataset obtained from S4. Time series within , serving as the first benchmark for predicting time series.

[0039] S5.2. Based on the period in which the key time series segment is located, determine the offset of the pathological segment of the electrocardiogram band in that period, and determine the second baseline for the predicted time series based on the offset. This specifically includes the following steps: S5.2.1 Determine the offset of the pathological segment on the electrocardiogram band of the period based on the period in which the key time series segment is located.

[0040] For example, the pathological segment on an electrocardiogram (ECG) is the ST segment. The ST segment offset is calculated using a formula. Specifically, the junction between the QRS complex endpoint and the ST segment initiation point is usually defined as the J point. The time to locate the QRS complex endpoint on each lead is... Then the ST band measurement time is : in, This represents the fluctuation value of the patient's physiological time series.

[0041] The voltage at time t is: The baseline voltage is: Then the offset of the ST segment in that lead is : (6) S5.2.2, Based on the offset First, determine the corresponding pathological name (e.g., ST-segment elevation myocardial infarction related to ST elevation). Then, use the corresponding S3 diagnostic name (acute ST-segment elevation myocardial infarction, acute pericarditis, early repolarization syndrome, hyperkalemia, etc.) to obtain the time series of the corresponding labeled historical dataset. Use the time series of the labeled historical dataset as the second reference for predicting the time series.

[0042] S5.3 Calculate the cosine distance between the first and second predicted time series references, and select the one with the smallest cosine distance as the short-period predicted time series. .

[0043] S6. Use the Informer model method to perform long-period prediction on multimodal ECG time series in medical monitoring scenarios, and obtain long-period predicted time series. : (7) The Informer model is input with a multimodal time series of electrocardiograms in a medical monitoring scenario, and the observation window length is [length missing]. L ; Learnable parameters of the Informer model; : The current moment.

[0044] S7. Calculate the short time series forecast values. and long-term forecast time series By performing mixed weighting, the predicted time series of medical rescue scenarios can be obtained. : (8) According to the prediction results This allows us to obtain heart rate data for a potential period of time in the future. The data can span three to five days or longer. In practical use, for medical staff on duty after 9 PM, this allows them to prepare medications, equipment, and treatment plans in advance for patients whose heart rate (predicted) may be abnormal.

[0045] For example: If it is an acute ST-segment elevation myocardial infarction with an elevation amplitude exceeding 5 mV, in addition to the standard STEMI treatment procedure, the following should also be done: (1) High-risk assessment and early warning; (2) Rapid expanded electrocardiogram assessment (3) Enhance hemodynamic monitoring and support preparation (4) Emergency optimization of reperfusion strategy and other tasks.

[0046] Example 2: A time series prediction device for medical rescue scenarios based on multimodality, comprising the following modules: The data acquisition module is used to acquire multimodal time series of electrocardiograms in medical rescue scenarios, retrieve historical datasets of electrocardiograms from the database, and annotate the historical datasets; The key time series segment acquisition module determines the first period and the first time series within the first period by applying an autocorrelation function to the historical dataset; and determines the key time series segments based on the first period. The vectorization module performs vectorization processing on the historical dataset, and then aligns the vectorized historical dataset to obtain the diagnostic names corresponding to the multimodal time series segments. The cosine similarity calculation module calculates the cosine similarity value for multimodal time series of electrocardiograms in medical rescue scenarios using autocorrelation function and cosine similarity, sorts the cosine similarity values, and obtains time series segments of labeled historical datasets. The short-cycle prediction time series acquisition module determines the short-cycle prediction time series based on key time series segments, diagnosis names, and time series segments of historical datasets obtained through sorting. The long-term forecast time series acquisition module uses the Informer model method to obtain long-term forecast time series for multimodal time series. The medical rescue scenario time series prediction module, based on short-period and long-period prediction time series, obtains the final predicted medical rescue scenario time series and obtains heart rate data for a future period.

[0047] The prediction device in this application embodiment can be an electronic device or a component within an electronic device, such as an integrated circuit or a chip. The electronic device can be a terminal or other devices besides a terminal. For example, the electronic device can be a GPU BOX, mobile phone, tablet computer, laptop computer, PDA, in-vehicle electronic device, mobile internet device (MID), augmented reality (AR) / virtual reality (VR) device, robot, wearable device, ultra-mobile personal computer (UMPC), netbook, or personal digital assistant (PDA), etc. It can also be a server, network attached storage (NAS), personal computer (PC), television (TV), ATM, or self-service machine, etc. This application embodiment does not specifically limit the device.

[0048] The prediction device in this application embodiment can be a device with an operating system. This operating system can be Android, Linux, Windows, or other possible operating systems; this application embodiment does not specifically limit it.

[0049] This application provides an electronic device, which includes a processor, a memory, and a computer program stored in the memory and executable on the processor. When the processor executes the program, it implements the steps in the prediction method of the foregoing embodiments.

[0050] This application also provides a computer-readable storage medium storing a computer program / instructions thereon, which, when executed by a processor, implements the steps in the prediction method disclosed in this application.

[0051] This application also provides a computer program product that, when run on an electronic device, causes a processor to execute the steps in the prediction method disclosed in this application.

[0052] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on the differences from other embodiments. The same or similar parts between the various embodiments can be referred to each other.

[0053] This application describes embodiments with reference to flowchart illustrations and / or block diagrams of methods, apparatuses, electronic devices, and computer program products according to embodiments of this application. It should be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing terminal device to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing terminal device, generate instructions for implementing the flowchart... Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0054] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing terminal device to operate in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0055] These computer program instructions can also be loaded onto a computer or other programmable data processing terminal equipment, causing a series of operational steps to be performed on the computer or other programmable terminal equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable terminal equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0056] Simulation verification: Validation was performed using the PTB-XL dataset, targeting incomplete right bundle branch block (IRBBB, i.e., generating cardiac electrical signals with slight delays or partial interruptions during right bundle branch conduction, but not complete blockage). The data curve in channel 1 is shown below. Figure 2 As shown.

[0057] After the time series data of channel 1 of the above curve is input into the Transformer model, the corresponding embedding vector is obtained. The vector format is as follows: (0.077872, -0.017309, -0.011340, -0.041217, -0.045323, 0.007201, 0.000209, -0.061472, ... -0.081433). The vector is then input into a two-level classifier. For example, the first level uses SVM for high-speed screening, and the second level uses deep feature classification. Multiple pathological names and corresponding cosine similarities may be obtained. Based on the corresponding cosine similarity, the preferred result is obtained, such as IRBBB.

[0058] Find IRBBB data in existing time series databases to obtain existing data. Then, weight the short-term and long-term forecast values ​​together to obtain the forecast result, such as... Figure 3 As shown, this condition may present with other cardiac responses similar to premature ventricular contractions in the future, requiring advance preparation.

[0059] The foregoing has shown and described the basic principles, main features, and advantages of the present invention. Those skilled in the art should understand that the present invention is not limited to the above embodiments. The embodiments and descriptions in the specification are merely illustrative of the principles of the invention. Various changes and modifications can be made to the invention without departing from its spirit and scope, and all such changes and modifications fall within the scope of the present invention as claimed. The scope of protection of this invention is defined by the appended claims and their equivalents.

Claims

1. A time series prediction method for medical rescue scenarios based on multimodality, characterized in that, include: Obtain multimodal time series of electrocardiograms in medical rescue scenarios, retrieve historical datasets of electrocardiograms from the database, and annotate the historical datasets; The autocorrelation function is used to determine the first period and the first time series within the first period of the historical dataset; key time series segments are determined based on the first period. The historical dataset is vectorized, and the vectorized historical dataset is aligned to obtain the diagnostic names corresponding to the multimodal time series segments. For multimodal time series of electrocardiograms in medical rescue scenarios, the cosine similarity value is calculated using autocorrelation function and cosine similarity. The cosine similarity values ​​are sorted to obtain time series segments of labeled historical datasets. Based on key time series segments, diagnosis names, and time series segments from historical datasets obtained through sorting, determine short-cycle prediction time series; The Informer model method is used to obtain long-period predicted time series for multimodal time series. Based on short-cycle and long-cycle prediction time series, the final predicted time series of medical rescue scenarios is obtained, and heart rate data for a future period is obtained.

2. The method for time series prediction in a multimodal medical rescue scenario according to claim 1, characterized in that, Determining the first period and the first time series within the first period using the autocorrelation function on historical datasets specifically includes: The autocorrelation function is used on the time series of the historical dataset to calculate the similarity between the time series and itself at different lag time points, thereby obtaining the periodic value; Lag Similarity is calculated at time points, with the first non-zero significant similarity corresponding to a lag. The time point is the first period, and the first time series within the first period is obtained based on the first period.

3. The method for time series prediction in a multimodal medical rescue scenario according to claim 2, characterized in that, The key time series segments determined based on the first period are as follows: For a connection time consisting of multiple first periods, calculate the extreme values ​​of the time series of the historical dataset within the connection time, including the maximum and minimum values, and the time series of the periods containing the maximum and minimum values, and label them as key time series segments.

4. The method for time series prediction in a multimodal medical rescue scenario according to claim 3, characterized in that, Alignment processing of the vectorized historical dataset includes: Attention scores were calculated using the Bahdanau attention function on the time series data of historical datasets. Calculate attention weights based on attention scores; The attention weights are weighted and averaged to obtain the context vector; the attention score, attention weights, and context vector are input into the Transformer decoder to output the time series. Align the time series output by the Transformer decoder with the historical dataset to obtain the diagnostic name corresponding to the output time series.

5. The method for time series prediction in a multimodal medical rescue scenario according to claim 4, characterized in that, Determining short-cycle forecast time series involves the following steps: From the time series segments of the labeled historical dataset obtained by sorting the cosine similarity values, the time series within the period corresponding to the maximum cosine similarity is selected as the first baseline for predicting the time series. Based on the period in which the key time series segment is located, determine the offset of the pathological segment of the electrocardiogram segment in that period, and determine the second prediction time series baseline based on the offset. Calculate the cosine distance between the first and second predicted time series, and select the one with the smallest cosine distance as the short-period predicted time series.

6. The method for time series prediction in a multimodal medical rescue scenario according to claim 5, characterized in that, Based on the period in which the key time series segment is located, determine the offset of the pathological segment of the electrocardiogram band in that period. Based on this offset, determine the second baseline for the predicted time series, specifically: Based on the period in which the key time series segment is located, determine the offset of the pathological segment on the electrocardiogram band of that period; based on the offset, determine the corresponding pathological name; then based on the corresponding pathological name, obtain the time series of the corresponding labeled historical dataset; and use the time series of the corresponding labeled historical dataset as the second reference for the predicted time series.

7. The method for time series prediction in a multimodal medical rescue scenario according to claim 6, characterized in that, The short-cycle and long-cycle predicted time series are mixed and weighted to obtain the final predicted time series for medical rescue scenarios.

8. A time series prediction device for medical rescue scenarios based on multimodality, characterized in that, Includes the following modules: The data acquisition module is used to acquire multimodal time series of electrocardiograms in medical rescue scenarios, retrieve historical datasets of electrocardiograms from the database, and annotate the historical datasets; The key time series segment acquisition module determines the first period and the first time series within the first period by applying an autocorrelation function to the historical dataset; and determines the key time series segments based on the first period. The vectorization module performs vectorization processing on the historical dataset, and then aligns the vectorized historical dataset to obtain the diagnostic names corresponding to the multimodal time series segments. The cosine similarity calculation module calculates the cosine similarity value for multimodal time series of electrocardiograms in medical rescue scenarios using autocorrelation function and cosine similarity, sorts the cosine similarity values, and obtains time series segments of labeled historical datasets. The short-cycle prediction time series acquisition module determines the short-cycle prediction time series based on key time series segments, diagnosis names, and time series segments of historical datasets obtained through sorting. The long-term forecast time series acquisition module uses the Informer model method to obtain long-term forecast time series for multimodal time series. The medical rescue scenario time series prediction module, based on short-period and long-period prediction time series, obtains the final predicted medical rescue scenario time series and obtains heart rate data for a future period.

Images