Cardiovascular acute event early warning method based on reinforcement learning

Abstract

The invention discloses a cardiovascular acute event early warning method based on reinforcement learning, and aims to solve the problems that an early warning scheme is difficult to consider individualized trigger opportunity and evaluation frequency and is difficult to balance false alarm and missed alarm. The method comprises the following steps of: outputting risks, trends and uncertainty of current and future time windows by using multiple heads of an uncertainty perception deep network, constructing a state in combination with historical alarm records, individualized risk thresholds and evaluation time intervals, and inputting the state into a security constraint reinforcement learning network; in the same forward reasoning, whether early warning, threshold value adjustment amount and next evaluation time interval are determined in a combined manner, and a constraint weight updating strategy is adaptively adjusted based on early warning income, false alarm and missed alarm indexes and prediction uncertainty after a real label is obtained. The technical effects of reducing the false alarm rate, optimizing the early warning advance and the evaluation frequency and improving the early warning accuracy and reliability of the cardiovascular acute event on the premise that the missed alarm risk is controllable are achieved.

Description

Technical Field

[0001] This invention relates to the field of medical monitoring and medical information processing, and in particular to a method for early warning of acute cardiovascular events based on reinforcement learning. Background Technology

[0002] Acute cardiovascular events are characterized by rapid onset and rapid progression. Clinically, continuous monitoring methods such as electrocardiogram, blood pressure, and blood oxygen are commonly used for risk assessment and early warning.

[0003] In recent years, with the development of intensive care systems and wearable devices, cardiovascular event prediction algorithms based on physiological signals have gradually evolved from traditional rule scoring and statistical models to intelligent early warning systems that utilize deep learning to extract features and predict risks from multi-channel time-series signals.

[0004] In existing technologies, some solutions can output the probability of cardiovascular events based on the physiological characteristics of the current time window and trigger an alarm when the probability exceeds a fixed threshold. Some also introduce time series models to improve the sensitivity and lead time of prediction.

[0005] However, existing technologies still have the following shortcomings in terms of early warning strategies:

[0006] 1. Most methods only output a single risk score for the current moment, or roughly predict the probability of occurrence within a certain fixed time range. They lack joint modeling of multi-dimensional information such as risk levels and risk change trends in different future time windows, making it difficult to provide sufficient support for the refined design of "when to start the warning and how far in advance the warning should be".

[0007] 2. Existing early warning systems generally use manually set fixed risk thresholds and fixed risk assessment time intervals, which do not fully consider individual differences and the dynamic changes of risk over time. They are difficult to adaptively adjust the thresholds and assessment frequency according to the patient's current status and historical alarms, which can easily lead to problems such as a high false alarm rate or an increased risk of missed alarms, as well as low efficiency in the use of computing and communication resources.

[0008] 3. In the high-risk application scenario of early warning of acute cardiovascular events, the costs of false alarms and missed alarms are significantly asymmetrical. However, existing methods mostly use fixed loss weights or simple threshold trade-offs, lacking explicit modeling of prediction uncertainty and its adaptive adjustment mechanism in the constraints of false alarms and missed alarms. They cannot dynamically adjust the focus of safety constraints according to the model confidence and the occurrence of recent real events, making it difficult to effectively suppress false alarms while ensuring that the risk of missed alarms is controllable, thus affecting the clinical usability and reliability of the early warning system.

[0009] Therefore, a method for early warning of acute cardiovascular events that can overcome the shortcomings of the existing technology is a problem that needs to be solved by those skilled in the art. Summary of the Invention

[0010] One objective of this invention is to propose an early warning method for acute cardiovascular events based on reinforcement learning. Addressing the limitations of existing technologies in balancing individualized warning triggering timing and assessment frequency, and the difficulty in dynamically balancing false alarms and missed alarms, this invention proposes a technical solution combining an uncertainty-aware deep network with safety-constrained reinforcement learning. Specifically: physiological data such as ECG, blood pressure, and blood oxygen are collected, preprocessed, and input into the uncertainty-aware deep network. This network outputs risk scores, risk change trends, and predictive uncertainty for the current and future time windows. The multi-dimensional risk information, along with historical alarm records, individualized risk thresholds, and historical assessment time intervals, forms a strategy state, which is then input into a safety-constrained reinforcement learning strategy network. Through multi-headed actions, a joint decision is made on whether to trigger a warning, the risk threshold adjustment amount, and the next risk assessment time interval. After obtaining accurate annotations, the warning benefit and evaluation indicators for false alarms and missed alarms are calculated. The constraint weights for false alarms and missed alarms are adaptively adjusted using predictive uncertainty, and the strategy network is iteratively updated. This invention achieves the technical effects of reducing the false alarm rate, increasing the early warning time and individualization level, optimizing assessment frequency and resource utilization, and thus improving the accuracy and reliability of early warnings for acute cardiovascular events, all while ensuring controllable missed alarm risk.

[0011] According to an embodiment of the present invention, a method for early warning of acute cardiovascular events based on reinforcement learning includes:

[0012] S1. Collect at least one physiological monitoring data from electrocardiogram, blood pressure and blood oxygen signals, preprocess it and input it as a physiological feature sequence into the uncertainty perception deep network to obtain the risk score of acute cardiovascular events at the current time point, the risk value and risk change trend index of occurrence in at least one preset future time window, and the corresponding prediction uncertainty index.

[0013] S2. The risk score, the risk value within at least one preset future time window, the risk change trend indicator and the prediction uncertainty indicator constitute the first type of state parameters. The historical alarm records related to the monitored object, the current individualized risk threshold and the historical risk assessment time interval are read from the local storage to constitute the second type of state parameters. These are combined to obtain multiple strategy state parameters at the current time point.

[0014] S3. Input multiple policy state parameters into the security constraint reinforcement learning policy network, perform policy reasoning, and obtain the first action parameter, the second action parameter, and the third action parameter.

[0015] S4. Based on the second and third action parameters, update the current individualized risk threshold and the time interval for the next risk assessment. When the first action parameter triggers a cardiovascular acute event warning signal, record the warning behavior and its time information as the warning behavior record result.

[0016] S5. After obtaining the real-time labeling information on whether an acute cardiovascular event has occurred, the results of the early warning behavior record are correlated to obtain the early warning benefit index, false alarm evaluation index, and missed alarm evaluation index. The false alarm evaluation index and missed alarm evaluation index are weighted according to the prediction uncertainty index to obtain the constraint weights used for safety constraint reinforcement learning.

[0017] S6. Input multiple policy state parameters, early warning benefit indicators, false alarm evaluation indicators, missed alarm evaluation indicators, and constraint weights into the security constraint reinforcement learning policy network. With the early warning benefit indicator as the optimization objective, and provided that the false alarm evaluation indicators and missed alarm evaluation indicators meet the preset constraints, adjust the relative weights of false alarms and missed alarms according to the constraint weights to update the parameters for the next execution.

[0018] Optionally, step S1 specifically includes:

[0019] Collect at least one physiological monitoring data from the monitored subject, including electrocardiogram, blood pressure, and blood oxygen signals. Perform bandpass filtering to remove noise, baseline drift correction, and amplitude normalization on the collected physiological monitoring data from each channel to obtain preprocessed physiological monitoring data.

[0020] Based on the preset time window length and sliding step size, physiological monitoring data within the time window corresponding to the current time point is extracted from the preprocessed physiological monitoring data and arranged in the order of sampling time to obtain the physiological feature sequence of the current time window;

[0021] The physiological feature sequence of the current time window is input into the temporal feature extraction subnetwork of the uncertainty perception deep network, and the temporal feature extraction subnetwork is used to extract features to obtain the temporal feature vector corresponding to the current time window.

[0022] The temporal feature vector is input into the multi-head output layer of the uncertainty-aware deep network, and the multi-head output layer simultaneously outputs multiple prediction results for the current time window. The multiple prediction results include at least a cardiovascular acute event risk score at the current time point, a cardiovascular acute event occurrence risk value within at least one preset future time window, a cardiovascular acute event risk change trend index, and a corresponding prediction uncertainty index. The prediction uncertainty index includes at least the prediction uncertainty for the cardiovascular acute event risk score, the prediction uncertainty for the cardiovascular acute event occurrence risk value within the at least one preset future time window, and the prediction uncertainty for the cardiovascular acute event risk change trend index.

[0023] Terminology definition:

[0024] The physiological monitoring data is time-series data obtained by continuously collecting at least one physiological parameter among the electrocardiogram signal, blood pressure signal, and blood oxygen signal of the monitored object, and can be a single-channel or multi-channel discrete sampling sequence.

[0025] The preprocessing refers to the steps of improving the signal quality and unifying the scale of the collected physiological monitoring data, which may include one or more of the following: bandpass filtering, baseline drift correction, amplitude normalization, and other conventional signal preprocessing operations.

[0026] The current time point is the reference time corresponding to the risk assessment and strategy decision-making when the method of the present invention is executed, which is usually the end of the current time window or the corresponding sampling time;

[0027] The preset time window length is the duration of time or the corresponding number of sampling points used when extracting the current time window from the physiological monitoring data, and is used to limit the duration of historical data input to the uncertainty perception deep network each time.

[0028] The sliding step size is the time interval or sampling point interval between the start times of two adjacent intercept time windows, used to control the degree of overlap of time windows and the time resolution of risk assessment.

[0029] The current time window is a continuous time range extracted from the physiological monitoring data with the current time point as a reference and according to the preset time window length, corresponding to the physiological monitoring data within this time range;

[0030] The physiological feature sequence is a multidimensional time series composed of one or more physiological monitoring data channels arranged in the order of sampling time within the current time window, and is used as the input feature of the uncertainty-aware deep network.

[0031] The uncertainty-aware deep network is a deep neural network model that can output the prediction results of cardiovascular acute event-related risks and provide corresponding prediction uncertainty indicators. It can achieve the prediction mean and its uncertainty estimation through Bayesian neural networks, model ensemble, Monte Carlo Dropout, distributed output layers and other methods.

[0032] The preset future time window is a pre-defined time range that is shifted backward relative to the current time point. Each future time window may have the same or different lengths and starting positions, which are used to characterize the risk of cardiovascular acute events under different lead times.

[0033] The aforementioned acute cardiovascular events are acute abnormal events that occur within a short period of time, are related to the cardiovascular system, and may be life-threatening or require emergency medical intervention, including but not limited to acute myocardial infarction, severe arrhythmia, cardiogenic shock, and exacerbation of acute heart failure.

[0034] The cardiovascular acute event risk score is a numerical indicator output by the uncertainty perception deep network at the current time point to characterize the probability of a cardiovascular acute event occurring near the current time point. The larger the value, the higher the current risk level.

[0035] The cardiovascular acute event risk value is a numerical indicator output by the uncertainty-aware deep network for at least one preset future time window, which characterizes the probability of an cardiovascular acute event occurring within the corresponding future time window.

[0036] The cardiovascular acute event risk change trend index is a characteristic index calculated based on the cardiovascular acute event risk score and / or occurrence risk value at the current time point and at least one preset future time window. It is used to characterize the direction and magnitude of risk change over time and can reflect the trend of risk increase, decrease or basically stable.

[0037] The prediction uncertainty index is a numerical indicator that quantifies the reliability of the cardiovascular acute event risk score, occurrence risk value, and risk change trend index output by the uncertainty-aware deep network. It can be the variance of the prediction distribution, the confidence interval width, the entropy value, the dispersion between samples, and / or other measures that can reflect the prediction confidence.

[0038] Optionally, step S2 specifically includes:

[0039] The cardiovascular acute event risk score at the current time point, the cardiovascular acute event occurrence risk value within the at least one preset future time window, the cardiovascular acute event risk change trend index, and the corresponding prediction uncertainty index are arranged in a preset order and subjected to numerical normalization to obtain the first type of state parameters.

[0040] Historical alarm records, current individualized risk thresholds, and historical risk assessment time intervals related to the monitored object are read from local storage. The historical alarm records are parsed to extract alarm statistics such as the previous warning time and the number of warnings within the preset historical time range. The time interval between the previous warning time and the current time point, the number of warnings, the current individualized risk threshold, and the historical risk assessment time intervals are numerically normalized to obtain the second type of state parameters.

[0041] The first type of state parameters and the second type of state parameters are concatenated along the feature dimension. The concatenated features are then input into a policy state encoding network. The policy state encoding network is used to perform a nonlinear transformation to obtain multiple policy state parameters at the current time point. These multiple policy state parameters are used to characterize the risk level, risk change trend, prediction uncertainty, and the comprehensive impact of historical alarm behavior on subsequent policy decisions at the current time point.

[0042] Terminology definition:

[0043] The first type of state parameters is a set of parameters obtained by arranging the cardiovascular acute event risk score, the cardiovascular acute event risk value within at least one preset future time window, the cardiovascular acute event risk change trend index and its corresponding prediction uncertainty index in a preset order at the current time point and then performing numerical normalization processing. It is used to characterize the risk and uncertainty information directly related to the current physiological state.

[0044] The second type of status parameters is a set of parameters formed by parsing, extracting statistics, and normalizing the historical alarm record information, current individualized risk threshold, and historical risk assessment time interval information read from local storage. It is used to characterize the status information related to historical warning behavior and individualized strategies.

[0045] The local storage is a storage medium or storage module in the monitoring device or server where the method of the present invention is deployed, used to store data related to the monitored object, including but not limited to memory, disk, flash memory or database, for persistently storing information such as historical alarm records, individualized risk thresholds and risk assessment time intervals;

[0046] The historical alarm record information is the recorded data of cardiovascular acute event warning signals and their occurrence time output by the early warning system within a preset historical time range, which is used to reflect the past early warning triggering situation of the monitored object;

[0047] The current individualized risk threshold is a risk assessment threshold set for the current state of the monitored object to determine whether a cardiovascular acute event warning is triggered. This threshold can be dynamically adjusted based on historical monitoring results and strategy network output, and can be different for different monitored objects.

[0048] The risk assessment time interval is the time interval between two consecutive cardiovascular acute event risk assessment steps or the corresponding sampling interval, used to determine the frequency or cycle of risk assessment.

[0049] The historical risk assessment time interval information is a record of one or more risk assessment time intervals actually used within a preset historical time range and their changes, used to reflect the frequency or periodic characteristics of risk assessment over a past period of time.

[0050] The preset historical time range is a time window used to select historical alarm record information and historical risk assessment time interval information when constructing the second type of state parameters. Its start and end times are preset relative to the current time point, and can be a fixed duration or a duration configured according to the application scenario.

[0051] The previous warning time is the time when the warning signal for the most recent cardiovascular acute event was triggered within the preset historical time range;

[0052] The number of warnings is the cumulative number of times the system triggers a warning signal for an acute cardiovascular event within the preset historical time range;

[0053] The alarm statistics are one or more statistical features calculated based on historical alarm record information within a preset historical time range, including but not limited to the previous warning time, the number of warnings, and other statistical indicators derived from multiple alarm times.

[0054] The time interval between the previous warning time and the current time is the time difference between the current time and the most recent warning trigger time within the preset historical time range, which is used to reflect the distance between the most recent warning and the current time.

[0055] The numerical normalization process is a process of mapping the original numerical features to a preset numerical range through linear transformation, nonlinear transformation or other scaling methods. It is used to eliminate dimensional differences and facilitate joint modeling of different types of features in the same network.

[0056] The policy state encoding network is a neural network model used to receive the feature vector after concatenating the first type of state parameters and the second type of state parameters in the feature dimension, and to convert it into policy state parameters for reinforcement learning decision through one or more nonlinear mapping layers.

[0057] The multiple policy state parameters are a set of vectorized features output by the policy state encoding network at the current time point, which are used as the state input of the security constraint reinforcement learning policy network to comprehensively characterize the impact of the current risk level, risk change trend, prediction uncertainty, and historical alarm behavior on subsequent policy decisions.

[0058] Optionally, step S3 specifically includes:

[0059] Multiple policy state parameters are input into the state encoding subnetwork of the security constraint reinforcement learning policy network. The state encoding subnetwork is used to perform a nonlinear transformation on the multiple policy state parameters to obtain the policy encoding vector corresponding to the current time point.

[0060] The policy encoding vector is input into the action output subnetwork of the safety constraint reinforcement learning policy network. The action output subnetwork includes at least three output heads that do not share parameters. In the same forward inference process, the action output subnetwork generates, based on the policy encoding vector, a first action parameter indicating whether a cardiovascular acute event warning is triggered at the current time point, a second action parameter indicating the threshold adjustment amount for increasing or decreasing the current individualized risk threshold, and a third action parameter indicating the risk assessment time interval to be used when executing step S1 next.

[0061] Terminology definition:

[0062] The security constraint reinforcement learning strategy network is a strategy network model that uses a reinforcement learning framework to model the early warning strategy and explicitly considers security-related constraints such as false alarms and missed alarms while optimizing the early warning benefits. It is used to output early warning-related action parameters based on the current strategy state parameters.

[0063] The policy reasoning is the process of obtaining the corresponding action parameters through forward computation of the network, given multiple policy state parameters and pre-trained security constraint reinforcement learning policy network parameters, without updating the network parameters.

[0064] The state encoding subnetwork is part of the security constraint reinforcement learning policy network. It is a neural network submodule that receives multiple policy state parameters and converts them into lower-dimensional or structured policy encoding vectors through one or more nonlinear mapping layers.

[0065] The policy encoding vector is a vectorized representation of the output of the state encoding subnetwork at the current time point based on multiple policy state parameters. It is used to centrally represent the comprehensive state information related to the current risk level, trend, uncertainty and historical alarm behavior, and serves as the input of the subsequent action output subnetwork.

[0066] The action output subnetwork is part of the safety constraint reinforcement learning policy network. It is used to generate a neural network submodule with first action parameters, second action parameters and third action parameters respectively through multiple output heads that do not share parameters, given a policy encoding vector.

[0067] The output head is an independent output branch in the action output sub-network that is set for a certain type of action parameter. Each output head has its own independent network parameters and is used to output values ​​or probability distributions related to the corresponding action type.

[0068] The forward inference process is a process of calculating the input features sequentially through each layer of the security constraint reinforcement learning policy network and obtaining the output results without updating the network parameters.

[0069] The first action parameter is an action parameter generated by the output head of the action output sub-network for decision-making alarm behavior based on the policy encoding vector, which is used to indicate whether to trigger a cardiovascular acute event warning at the current time point;

[0070] The cardiovascular acute event early warning is an early warning issued to medical staff, monitored subjects, or monitoring systems regarding the possible occurrence of cardiovascular acute events, based on the assessment results of the risk of cardiovascular acute events in the current and future time windows, to prompt timely intervention or further diagnosis.

[0071] The second action parameter is an action parameter generated by the output head of the action output subnetwork for threshold adjustment based on the policy encoding vector, which is used to indicate whether to increase or decrease the current individualized risk threshold;

[0072] The threshold adjustment amount is the magnitude of the change in the value of the current individualized risk threshold, represented by the second action parameter. Its value can be positive, negative, or zero, corresponding to increasing, decreasing, or keeping the current individualized risk threshold unchanged.

[0073] The third action parameter is an action parameter generated by the output head of the action output sub-network used to evaluate frequency control based on the policy encoding vector, which is used to indicate the risk assessment time interval to be used when executing step S1 next.

[0074] Optionally, step S4 specifically includes:

[0075] According to the second action parameter, the current individualized risk threshold is read from the local storage, the threshold adjustment amount indicated by the second action parameter is numerically calculated with the current individualized risk threshold to obtain the updated individualized risk threshold, and the updated individualized risk threshold is written to the local storage to replace the current individualized risk threshold.

[0076] According to the third action parameter, the risk assessment time interval indicated by the third action parameter is set as the risk assessment time interval for the next execution of step S1 in the scheduling module, and the risk assessment time interval is written as the latest risk assessment time interval information into the local storage.

[0077] According to the first action parameter, when the first action parameter indicates that an early warning is triggered, the control terminal device outputs a cardiovascular acute event early warning signal at the current time point, and records the early warning behavior and its corresponding current time point together as the early warning behavior recording result.

[0078] Terminology definition:

[0079] The updated individualized risk threshold is a new risk judgment threshold obtained by numerically calculating the current individualized risk threshold according to the threshold adjustment amount indicated by the second action parameter. It is used to replace the original current individualized risk threshold in subsequent early warning decisions.

[0080] The numerical calculation is an addition, subtraction or other pre-set arithmetic operation between the current individualized risk threshold and the threshold adjustment amount, used to obtain the updated individualized risk threshold;

[0081] The scheduling module is a task scheduling unit used to manage and trigger the execution time of each step in the method of the present invention. It can be a software module or hardware logic running on a monitoring device or server, and is used to set the risk assessment time interval for the next execution of step S1 according to the third action parameter.

[0082] The risk assessment time interval information is time interval-related data used to characterize the frequency of risk assessment execution, including the currently used risk assessment time interval and historical risk assessment time intervals;

[0083] The terminal device is a human-computer interaction device or display device used to display or output early warning signals of acute cardiovascular events to medical staff, monitored subjects or monitoring systems. It can be a bedside monitor, a nurse station monitoring terminal, a mobile terminal or other device with alarm prompting function.

[0084] The cardiovascular acute event warning signal is an alarm signal output by the terminal device in a predetermined form to indicate that a cardiovascular acute event may occur. It may include audible and visual alarms, interface pop-ups, vibration reminders, message pushes, or combinations thereof.

[0085] The warning behavior is the alarm action actually executed by the warning system when the first action parameter indicates that the warning is triggered, including outputting a cardiovascular acute event warning signal and its corresponding time information;

[0086] The warning behavior recording result is a record of one or more warning behaviors that occur within a preset time range and their corresponding time points, which is used for time alignment and statistical analysis with the real annotation information in subsequent steps.

[0087] Optionally, step S5 specifically includes:

[0088] After obtaining the true annotation information regarding whether the monitored object has experienced an acute cardiovascular event, the true annotation information includes at least whether an acute cardiovascular event occurred within a preset observation time range and the actual occurrence time of the acute cardiovascular event. The true annotation information is aligned and matched with the early warning behavior record results by time to determine the actual event result corresponding to each early warning behavior. Based on the actual event result and the early warning behavior record results, the early warning benefit index, false alarm evaluation index, and missed alarm evaluation index for the current time point are calculated according to preset calculation rules. The early warning benefit index is used to characterize the successful triggering of an early warning within a preset advance time range before the actual occurrence of an acute cardiovascular event, with higher benefits for larger advance amounts. The false alarm evaluation index is used to characterize the number or proportion of early warnings triggered even though no acute cardiovascular event occurred within the preset observation time range. The missed alarm evaluation index is used to characterize the number or proportion of effective early warnings not triggered within the target advance time range even though an acute cardiovascular event occurred within the preset observation time range.

[0089] The prediction uncertainty index corresponding to the cardiovascular acute event risk score, the cardiovascular acute event occurrence risk value within the at least one preset future time window, and the cardiovascular acute event risk change trend index is read from the prediction uncertainty index for the current time point. Weighting coefficients for false alarm evaluation index and missed alarm evaluation index are determined based on the prediction uncertainty index. This increases the weight of the missed alarm evaluation index in the constraints when prediction uncertainty is high, and increases the weight of the false alarm evaluation index in the constraints when prediction uncertainty is low and no real cardiovascular acute events have occurred for a long period. The false alarm evaluation index and missed alarm evaluation index are then weighted and normalized based on the weighting coefficients to obtain the constraint weights for safety constraint reinforcement learning.

[0090] Terminology definition:

[0091] The actual annotation information is the annotation data obtained by marking whether the monitored subject has experienced an acute cardiovascular event and the time of its occurrence based on medical records, annotations by medical staff, diagnosis and treatment operation records and / or other reliable information sources. It is used as an objective reference for evaluating the early warning effect.

[0092] The preset observation time range is a time interval selected relative to the current time point or a certain reference time point when evaluating the benefits of early warning, false alarms and missed alarms. It is used to statistically analyze the actual occurrence of acute cardiovascular events and the triggering of early warnings within this time interval.

[0093] The actual occurrence time is the time point at which a certain acute cardiovascular event begins or is confirmed to have occurred, as recorded in the real annotation information. It is used to compare with the trigger time of the warning behavior to calculate the lead time and the number of missed warnings.

[0094] The time alignment and matching is a process of associating the occurrence time of acute cardiovascular events recorded in the real annotation information with the trigger times of each warning recorded in the warning behavior record results based on timestamps or other time identifiers, in order to determine the real event result corresponding to each warning behavior;

[0095] The real event result refers to the result of judging whether each warning behavior corresponds to a real acute cardiovascular event within a preset observation time range after time alignment and matching are completed. It may include information such as true positive warning, false positive warning, and missed warning.

[0096] The preset calculation rules are calculation methods or algorithms pre-set for calculating early warning benefit indicators, false alarm evaluation indicators and missed alarm evaluation indicators from real event results and early warning behavior record results. They may include specific formulas, statistical methods, weighting methods and their parameter settings.

[0097] The preset advance time range is the time interval used to determine "whether it is an effective advance warning" when evaluating the benefits of the warning. That is, it is a preset time range that is pushed forward relative to the actual occurrence time of the acute cardiovascular event. Triggering a warning within this time range is considered as an advance warning for the event.

[0098] The target advance time range is the time interval used to determine whether the warning meets the expected advance requirement when assessing missed warnings. It is usually a target time range that is pushed forward relative to the actual occurrence time of the acute cardiovascular event. Events that do not trigger an effective warning outside this time range can be regarded as missed warnings.

[0099] The weighting coefficient is a numerical weight determined based on the prediction uncertainty index for the current time point, used to adjust the relative importance of the false alarm evaluation index and the missed alarm evaluation index, and is used to amplify or reduce the impact of false alarms or missed alarms on the constraints in subsequent calculations.

[0100] The constraint weights are comprehensive weight parameters obtained by weighting and normalizing the false alarm evaluation index and the missed alarm evaluation index in combination with weighting coefficients, and are used for safety constraint reinforcement learning. They are used to adjust the relative weights of the constraint loss components related to false alarms and the constraint loss components related to missed alarms in the total loss during the reinforcement learning training process.

[0101] The weighting and normalization process involves performing a weighted product operation on the false alarm evaluation index and the missed alarm evaluation index according to the weighting coefficients, and scaling or normalizing the results to make them meet the preset numerical range or sum constraints. This process is used to construct numerically stable and comparable constraint weights.

[0102] Optionally, step S6 specifically includes:

[0103] Multiple policy state parameters, as well as early warning benefit indicators, false alarm evaluation indicators, missed alarm evaluation indicators, and constraint weights, are input into the training module of the security constraint reinforcement learning policy network. Combined with multiple action parameters output at the corresponding time point in step S3, an instant reward signal containing early warning benefit information, false alarm penalty information, and missed alarm penalty information is constructed. The instant reward signal is used to characterize the comprehensive policy performance obtained under the current policy state parameters and the multiple action parameters.

[0104] The instantaneous reward signal is compared with the policy value predicted by the safety constraint reinforcement learning policy network based on the multiple policy state parameters. The policy evaluation error is calculated, and the constraint loss components related to the false alarm evaluation index and the missed alarm evaluation index are weighted according to the constraint weights to form a total loss function that simultaneously includes the early warning benefit index, the false alarm evaluation index, and the missed alarm evaluation index. This increases the weight of the missed alarm-related constraint loss in the total loss function when the prediction uncertainty is high, and increases the weight of the false alarm-related constraint loss in the total loss function when the prediction uncertainty is low and no real acute cardiovascular events have occurred for a long time.

[0105] Based on the total loss function, a gradient descent-type optimization algorithm is used to jointly update the parameters of the state encoding subnetwork and the action output subnetwork in the safety constraint reinforcement learning policy network to obtain the updated safety constraint reinforcement learning policy network. In the next execution of step S3, the updated safety constraint reinforcement learning policy network is used to perform policy inference on the new multiple policy state parameters.

[0106] Terminology definition:

[0107] The preset constraints are the allowable ranges or upper limits of constraints for false alarm evaluation indicators and missed alarm evaluation indicators, which are set in advance according to clinical safety needs or system design requirements before strategy optimization. They are used to ensure that the false alarm rate and missed alarm rate remain within a safe and acceptable level while optimizing the early warning benefit indicators.

[0108] The optimization objective is the performance metric that is prioritized when training or updating a security-constrained reinforcement learning policy network. In this invention, it is to maximize or nearly maximize the early warning benefit metric.

[0109] The training module is a functional module in the safety-constrained reinforcement learning policy network used to perform training or update operations. It can be a software-implemented training program or a hardware / software combined training unit, used to receive input data and update network parameters according to the loss function.

[0110] The early warning benefit information is benefit-related information extracted or transformed from the early warning benefit index to form an instant reward signal, which is used to reflect the positive benefit obtained by early warning under the current strategy state parameters and action parameters.

[0111] The false alarm penalty information is penalty-related information extracted or transformed from the false alarm evaluation index to form an immediate reward signal, and is used to impose a negative reward on the behavior that falsely triggers the warning when no real acute cardiovascular event has occurred.

[0112] The missed alarm penalty information is penalty-related information extracted or transformed from the missed alarm evaluation index to form an immediate reward signal. It is used to impose negative rewards on behaviors that fail to trigger an effective warning within the target advance time range when a real acute cardiovascular event occurs.

[0113] The instantaneous reward signal is a single-step reward value or reward vector calculated in a preset manner based on early warning benefit information, false alarm penalty information and missed alarm penalty information, given multiple current strategy state parameters and corresponding action parameters, and is used to characterize the comprehensive strategy performance obtained under the state-action pair.

[0114] The comprehensive strategy is the overall effect obtained by comprehensively considering the early warning benefits, false alarm penalties, and missed alarm penalties under the current strategy state parameters and corresponding multiple action parameters, and is used to evaluate the merits of the current strategy decision.

[0115] The policy value is the expected cumulative return or value assessment result of the state predicted by the security constraint reinforcement learning policy network based on multiple policy state parameters. It is used to estimate the long-term comprehensive benefit that can be obtained by continuing to execute the existing policy in the current state.

[0116] The strategy evaluation error is a measure of the difference between the immediate return signal and the strategy value. It is used to reflect the degree of inconsistency between the current strategy value prediction and the actual return. It can be measured by squared error, absolute error or other forms.

[0117] The constraint loss component is the loss part corresponding to the false alarm evaluation index or the missed alarm evaluation index when constructing the total loss function. It is used to measure the degree of violation or deviation of the current strategy in terms of false alarm constraints and missed alarm constraints.

[0118] The total loss function is an overall optimization objective function constructed by comprehensively considering the early warning benefit index, false alarm evaluation index and missed alarm evaluation index when training the safety constraint reinforcement learning policy network. It is composed of the main loss term used to improve the early warning benefit and the constraint loss components related to false alarm and missed alarm according to the constraint weights.

[0119] The gradient descent-type optimization algorithm is an optimization method that iteratively updates network parameters based on the gradient information of the total loss function. It includes stochastic gradient descent, momentum method, adaptive learning rate algorithm and its variants, which are used to gradually reduce the value of the total loss function.

[0120] The joint update is a method of simultaneously updating the parameters of the state encoding subnetwork and the action output subnetwork in the safety-constrained reinforcement learning policy network during the same backpropagation and gradient calculation process based on the total loss function, so that the parameters of the two are optimized in a coordinated manner.

[0121] The updated safety constraint reinforcement learning policy network is a new network model obtained after jointly updating the parameters of the state encoding subnetwork and the action output subnetwork using a gradient descent optimization algorithm. It is used to perform policy reasoning on the new multiple policy state parameters in the next execution of step S3.

[0122] The beneficial effects of this invention are:

[0123] 1. By using an uncertainty-aware deep network, multi-head joint predictions can be made on the risk values, risk change trends, and corresponding prediction uncertainties of acute cardiovascular events in the current time window and multiple future time windows. This can more comprehensively depict the event occurrence process and development trend, provide rich information for determining reasonable early warning time and dynamically adjusting risk thresholds, and help improve early warning capabilities and risk assessment accuracy.

[0124] 2. By combining risk scores, future risks, trends, uncertainties, historical alarm records, current individualized risk thresholds, and historical risk assessment time intervals to form the strategy state, and using the multi-head action output of a safety-constrained reinforcement learning strategy network, the system simultaneously obtains whether to trigger an alarm, the threshold adjustment amount, and the next risk assessment time interval in a single strategy inference. Compared with alarm strategies with fixed thresholds and fixed assessment frequencies, this system can achieve individual-oriented threshold adaptation and assessment frequency adaptation, which is beneficial to reduce the false alarm rate and improve the utilization efficiency of computing and monitoring resources while ensuring that the risk of missed alarms is controllable.

[0125] 3. By simultaneously considering early warning benefit indicators, false alarm evaluation indicators, and missed alarm evaluation indicators during the training phase, and by using prediction uncertainty to weight the constraint losses related to false alarms and missed alarms, the relative weights of false alarms and missed alarms in the total loss function are dynamically adjusted. This strengthens the constraint on missed alarms when prediction uncertainty is high, and strengthens the constraint on false alarms when prediction uncertainty is low and no real events have occurred for a long time. Compared with reinforcement learning methods that use fixed loss weights, this approach can adaptively balance safety and sensitivity based on model confidence and actual clinical scenarios, which is beneficial to improving the stability and clinical usability of the cardiovascular acute event early warning system. Attached Figure Description

[0126] The accompanying drawings are provided to further illustrate the invention and form part of the specification. They are used in conjunction with embodiments of the invention to explain the invention and do not constitute a limitation thereof. In the drawings:

[0127] Figure 1 This is a flowchart of a method for early warning of acute cardiovascular events based on reinforcement learning, as proposed in this invention. Detailed Implementation

[0128] The present invention will now be described in further detail with reference to the accompanying drawings. These drawings are simplified schematic diagrams, illustrating only the basic structure of the invention, and therefore only show the components relevant to the invention.

[0129] refer to Figure 1 A reinforcement learning-based early warning method for acute cardiovascular events includes:

[0130] S1. Collect at least one physiological monitoring data from electrocardiogram, blood pressure and blood oxygen signals, preprocess it and input it as a physiological feature sequence into the uncertainty perception deep network to obtain the risk score of acute cardiovascular events at the current time point, the risk value and risk change trend index of occurrence in at least one preset future time window, and the corresponding prediction uncertainty index.

[0131] S2. The risk score, the risk value within at least one preset future time window, the risk change trend indicator and the prediction uncertainty indicator constitute the first type of state parameters. The historical alarm records related to the monitored object, the current individualized risk threshold and the historical risk assessment time interval are read from the local storage to constitute the second type of state parameters. These are combined to obtain multiple strategy state parameters at the current time point.

[0132] S3. Input multiple policy state parameters into the security constraint reinforcement learning policy network, perform policy reasoning, and obtain the first action parameter, the second action parameter, and the third action parameter.

[0133] S4. Based on the second and third action parameters, update the current individualized risk threshold and the time interval for the next risk assessment. When the first action parameter triggers a cardiovascular acute event warning signal, record the warning behavior and its time information as the warning behavior record result.

[0134] S5. After obtaining the real-time labeling information on whether an acute cardiovascular event has occurred, the results of the early warning behavior record are correlated to obtain the early warning benefit index, false alarm evaluation index, and missed alarm evaluation index. The false alarm evaluation index and missed alarm evaluation index are weighted according to the prediction uncertainty index to obtain the constraint weights used for safety constraint reinforcement learning.

[0135] S6. Input multiple policy state parameters, early warning benefit indicators, false alarm evaluation indicators, missed alarm evaluation indicators, and constraint weights into the security constraint reinforcement learning policy network. With the early warning benefit indicator as the optimization objective, and provided that the false alarm evaluation indicators and missed alarm evaluation indicators meet the preset constraints, adjust the relative weights of false alarms and missed alarms according to the constraint weights to update the parameters for the next execution.

[0136] In this specific embodiment, S1 specifically refers to:

[0137] The monitoring system uses a sampling frequency At least one physiological parameter among the electrocardiogram (ECG), blood pressure, and blood oxygen saturation signals of the monitored subject is continuously sampled to obtain multi-channel physiological monitoring data. ,in This indicates the fixed sampling frequency of the physiological monitoring equipment. Indicates in the channel Upper The original physiological signal amplitudes collected at each sampling point This is a channel index used to distinguish different physiological channels. The index of discrete sampling points is ordered sequentially over time.

[0138] Then, the raw physiological monitoring data for each channel... Preprocessing operations, including bandpass filtering, baseline drift correction, and artifact suppression, are performed sequentially to obtain the intermediate signal after noise suppression and baseline correction. ,in Indicates in the channel Upper The preprocessed signal amplitude at each sampling point;

[0139] To eliminate the dimensional and amplitude scale differences between different channels, further... Amplitude normalization was performed to obtain normalized physiological monitoring data. Its calculation form can be expressed as ,in Indicates in the channel Upper The dimensionless amplitude of each sampling point after normalization This indicates the channel within the preset statistical time range. The signal mean, Indicates channels within the same statistical time range The standard deviation of the signal;

[0140] After preprocessing and normalizing the data from each channel, the time window length is pre-set according to the monitoring application's requirements for risk assessment time resolution and historical information length. and sliding step size And according to the sampling frequency Convert it to the corresponding number of sampling points and ,satisfy ,in This indicates the physical time length covered by the current time window used in a single risk assessment. Indicates the physical time interval between two consecutive risk assessments. This indicates the number of sampling points contained within each time window. This indicates the sliding step size of the time window on the sampling point sequence;

[0141] For the first time during continuous monitoring There are 1 time windows to be evaluated, and the starting sampling point index of this time window in the sampling sequence is defined as . And according to the sliding step relationship To determine, among which The time window index is a non-negative integer used to identify the first... A time window for evaluation This indicates the starting position of the corresponding time window in the global sampling sequence. The sampling interval between the starting positions of adjacent time windows;

[0142] Within each time window, index from the sampling point on each channel. Begin by taking samples of length 1 in chronological order. Normalized physiological monitoring data, i.e., in the channel Select from arrive of They are then spliced ​​together and arranged by time along the channel dimension to form the physiological feature sequence of the current time window. This physiological characteristic sequence can be formally represented as ,in Indicates the first Multi-channel time series features organized within a time window This represents the total number of physiological monitoring channels involved in the modeling. For the normalized physiological monitoring data defined above, This refers to the index of the sampling points being traversed within the current window;

[0143] At the same time, the physical time at the end of the time window is recorded as the current time point. ,in Used to identify and the first The actual monitoring time corresponding to the end of each time window is used as a reference for aligning the subsequent risk assessment results with the actual event time.

[0144] Subsequently, the physiological characteristic sequence of the current time window is... A time-series feature extraction subnetwork with non-shared parameters is input to an uncertainty-aware deep network. This time-series feature extraction subnetwork can be abstractly represented as a function. Through the Perform a series of temporal convolutions, recurrent units, or self-attention operations to output the temporal feature vector corresponding to the current time window. The mapping relationship can be expressed as: ,in Indicates As input, with parameter vector For temporal feature extraction mapping of trainable parameters, This represents the feature vector after comprehensively encoding multi-channel physiological dynamic information within the current time window;

[0145] In obtaining time-series feature vectors Then, it is fed into the multi-head output layer of the uncertainty-aware deep network, which can be abstractly represented as a function. In the same forward inference process, it simultaneously outputs risk prediction results and their uncertainty indicators for the current time point and multiple preset future time windows. The output relationship can be summarized as follows: ,in Indicates As input, with parameter vector For multi-head output mapping with trainable parameters, Indicates the current time point Risk score for nearby acute cardiovascular events Relative to the current time point The A preset risk value for the occurrence of acute cardiovascular events within a future time window. This indicates the total number of preset future time windows. This refers to an indicator of the trend of cardiovascular acute event risk, calculated based on the current risk score and risk values ​​over multiple future time windows. It is used to characterize the upward, downward, or stable trend of risk over time. This indicates the risk score at the current point in time. The predictive uncertainty index Indicates that for the first Preset future time window risk value The predictive uncertainty index Indicators representing trends in risk changes The predictive uncertainty index, through the above processing, at the current time point This allows us to simultaneously obtain the current risk score, the risk value occurring within at least one preset future time window, the risk change trend indicator, and the corresponding prediction uncertainty indicator, all of which are used to construct the first type of state parameters.

[0146] In this specific embodiment, S2 specifically refers to:

[0147] Regarding the current time point Obtained cardiovascular acute event risk score Risk values ​​for acute cardiovascular events in multiple preset future time windows Trend indicators of changes in the risk of acute cardiovascular events and the corresponding forecast uncertainty indicators The scalars related to risk level, future risk and risk change trend and their prediction uncertainty are arranged in a preset order and input into the numerical normalization function. The first type of state parameter vector is obtained. The calculation relationship can be summarized as follows: ,in This is a time window index used to identify the current time window. A time window, For the current time point corresponding to the end of this time window, Indicates the current time point Risk score for nearby acute cardiovascular events Relative to the current time point The A preset risk value for the occurrence of acute cardiovascular events within a future time window. To preset the total number of future time windows, This indicator represents the trend of cardiovascular acute event risk changes calculated based on the current risk score and risk values ​​for each future time window. It is used to characterize the trend of risk increasing, decreasing, or remaining relatively stable over time. This indicates the current risk score. The predictive uncertainty index Indicates that for the first Preset future time window risk value The predictive uncertainty index Indicators representing trends in risk changes The predictive uncertainty index This indicates mapping the input features to a numerical normalization function within a uniform numerical range according to a predetermined rule. [] indicates concatenating multiple scalar features into a one-dimensional vector along the feature dimension in a preset order. Indicates the current time point The first type of state parameter vector is composed of risk score, risk value within a preset future time window, risk change trend and corresponding prediction uncertainty after normalization.

[0148] Secondly, it reads historical alarm records, current individualized risk thresholds, and historical risk assessment time intervals related to the monitored object within a preset historical time range from local storage, parses the historical alarm records, and then... Determine the time of the most recent triggering of an acute cardiovascular event warning. and the number of warnings within that time frame. And based on the current time point Compared to the time of the most recent warning trigger Calculate the time interval between the previous warning time and the current time. The calculation relationship can be expressed as follows: ,in Relative to the current time point The preset historical time range for backward reference. Indicates the time interval The time of the most recent triggering of a cardiovascular acute event warning. This indicates the cumulative number of times a warning has been triggered within that time interval. This indicates the time interval between the previous warning time and the current time.

[0149] At the same time, read the current individualized risk threshold from local storage. and the characteristics of risk assessment time intervals obtained from statistical analysis of historical risk assessment time interval information. ,in Indicates the current time point The current individualized risk thresholds used for the monitored subjects, Indicates within the preset historical time range The representative risk assessment time interval characteristics are calculated based on historical risk assessment time interval information.

[0150] Then, and Arrange the data in a preset order and input the numerical normalization function. The second type of state parameter vector is obtained. The calculation relationship can be summarized as follows: ,in Indicates the current time point The second type of state parameter vector is composed of the time interval between the previous warning time and the current time, the number of warnings, the current individualized risk threshold, and the characteristics of the historical risk assessment time interval after normalization.

[0151] Subsequently, the first type of state parameter vector With the second type of state parameter vector The features are concatenated to obtain the comprehensive policy state input vector. Its form can be expressed as ,in Indicates the current time point The input vector below is a comprehensive strategy status that simultaneously includes risk-related information, historical alerts, and individualized strategy information;

[0152] Finally, the integrated policy state input vector Input policy state coding network Multiple policy state parameters at the current time point can be obtained through one or more nonlinear mappings. The mapping relationship can be expressed as ,in Indicates For input, with For a policy state encoding network with trainable parameter vectors, Indicates the current time point The policy state parameters output by the policy state encoding network are used as state inputs to the security constraint reinforcement learning policy network in subsequent steps to comprehensively reflect the current risk level, risk change trend, prediction uncertainty, and the impact of historical alarm behavior and individualized policies on subsequent early warning decisions.

[0153] In this specific embodiment, S3 specifically refers to:

[0154] At the current time point Below, based on multiple policy state parameter vectors The state input is used to construct a security-constrained reinforcement learning policy network, where For time windows or decision step indexes, used to identify the current time step. At each decision-making moment, In order to be with the first The current time point corresponding to the end of each time window Indicates the current time point The following multiple policy state parameter vectors output by the policy state coding network comprehensively represent the current risk level, risk change trend, prediction uncertainty, historical alarm behavior, and individualized policy information;

[0155] Then, the policy state parameter vector The state-encoding subnetwork in the input security-constrained reinforcement learning policy network can be abstractly represented as a mapping function. Through one or more nonlinear transformations Compress or reconstruct the policy encoding vector for the current time point. The mapping relationship can be simply expressed as follows: ,in Indicates As input, with parameter vector For a state-encoded subnetwork with trainable parameters, Indicates the current time point The policy encoding vector output by the state encoding subnetwork is used to centrally represent the high-dimensional policy state information related to early warning decision-making. This represents the set of network parameters, such as weights and biases, for each layer within the state-coded subnetwork.

[0156] After obtaining the policy encoding vector After that, The action-output subnetwork of the input safety-constrained reinforcement learning policy network can be abstractly represented as a mapping function. In the same forward inference process, the first action parameter, the second action parameter, and the third action parameter are generated respectively through at least three output heads that do not share parameters. The overall output relationship can be summarized as follows: ,in Represents a policy-encoded vector As input, with parameter vector For trainable parameters, output subnetwork for actions. This represents the set of network parameters that are shared or not shared among the output heads within the action output sub-network. Indicates the current time point The first action parameter, generated by the output head used for alarm decision-making, indicates whether to trigger a cardiovascular acute event warning signal at the current time point. Indicates the current time point The second action parameter, generated by the output head used for threshold adjustment, indicates the amount of threshold adjustment to increase, decrease, or remain unchanged the current individualized risk threshold. Indicates the current time point The third action parameter, generated by the output head used to evaluate frequency control, indicates the risk assessment time interval to be used in the next execution of step S1. Through the cascading of the aforementioned state encoding subnetwork and action output subnetwork, the current policy state parameter vector can be used for forward inference without changing the network parameters. By combining the three action parameters—whether to trigger an early warning, how to adjust the individualized risk threshold, and how to set the time interval for the next risk assessment—a decision-making basis is provided for subsequent steps such as executing early warning actions, updating thresholds, and scheduling assessments.

[0157] In this specific embodiment, S4 specifically refers to:

[0158] At the current time point Next, according to the second action parameter Adjust the current individualized risk threshold value and read the current individualized risk threshold from local storage. And calculate the updated individualized risk threshold according to the additive update rule. ,in Indicates the relationship with the first The current time point corresponding to the end of each time window Indicates a point in time The second action parameter, generated by the threshold adjustment output head of the safety constraint reinforcement learning policy network, is used to characterize the amount of adjustment to the individualized risk threshold in this instance. Indicates a point in time The current individualized risk threshold previously stored locally. Indicates based on the second action parameter The updated individualized risk threshold is obtained;

[0159] The monitoring system is completed After the calculation, it is written to local storage to replace the original. This makes it possible at subsequent time points When performing steps S2 and S3, both are done with It serves as the current individualized risk threshold for state construction and strategy reasoning;

[0160] Secondly, based on the third action parameter output in step S3 The monitoring system can set the interval for the next risk assessment. This is directly interpreted as the risk assessment time interval value used in the next execution of step S1, and denoted as... ,Right now ,in Indicates a point in time The third action parameter, generated by the evaluation frequency control output head of the safety constraint reinforcement learning policy network, indicates the time interval for the next risk assessment. Indicates from the current time point The scheduling module is used to arrange the risk assessment time interval for the next execution of step S1.

[0161] The scheduling module receives the updated risk assessment time interval. Then, according to this time interval, the trigger time for the next execution of step S1 is scheduled on the system clock, and... The latest risk assessment time interval information is written to local storage so that it can be used as part of the historical risk assessment time interval information in the subsequent step S2 to construct the second type of state parameters.

[0162] Next, based on the first action parameter output in step S3 Decide whether to do so at the current time. Triggering an acute cardiovascular event warning, when the first action parameter When the value of or the discretization result of the value indicates that an early warning needs to be triggered, the monitoring system control terminal device will, at the current time point... It outputs early warning signals for acute cardiovascular events, such as alerts to medical staff or monitored individuals via audible and visual alarms, pop-up windows, or push notifications. A corresponding early warning behavior record is generated locally, storing the early warning behavior and its trigger time in the early warning behavior record results. Indicates a point in time The first action parameter generated by the alarm decision output head of the safety constraint reinforcement learning policy network is used to indicate whether a cardiovascular acute event warning is triggered at this time point. The warning behavior record result is used to perform time alignment and matching with the real labeled information in the subsequent step S5 to calculate the warning benefit index, false alarm evaluation index and missed alarm evaluation index.

[0163] In this specific embodiment, S5 specifically includes:

[0164] During the training or offline assessment phase, once accurate information regarding whether the monitored subject experienced an acute cardiovascular event and the actual timing of each event is obtained, the monitoring system uses the current time point... For reference, within a preset observation time range extending backward from this time point, the actual event occurrence time is aligned and matched with the warning behavior record results obtained in step S4 on the time axis. Based on whether a real event exists and whether the warning falls within the preset advance time range, each warning behavior is determined to be either a valid advance warning before the real event, a false alarm that does not correspond to a real event within the observation time range, or a missed alarm corresponding to a real event that was not triggered within the target advance time range. Based on this, according to preset statistical or scoring rules, the warning benefit index, false alarm evaluation index, and missed alarm evaluation index for the current time point are calculated, where the warning benefit index is marked as... This is used to quantify the degree to which a false alarm is successfully triggered before a real acute cardiovascular event, with greater lead time resulting in higher benefits. The false alarm rating is denoted as... This is used to quantify the number or proportion of times an alert is triggered even though no real event has occurred within a preset observation period. The missed alert evaluation indicator is marked as... This is used to quantify the number or proportion of real events that occur within a preset observation time range but fail to trigger an effective warning within the target advance time range. These factors collectively reflect the time point... The overall performance of the surrounding early warning strategy in detecting real events in advance, avoiding invalid alarms, and reducing missed alarms;

[0165] Subsequently, in order to incorporate prediction uncertainty into the trade-off between false alarms and missed alarms, the system, starting from step S1, targets the current time point. The output forecast uncertainty index includes the uncertainty quantities corresponding to the current risk score, the risk values ​​for each preset future time window, and the risk change trend index. Furthermore, the multidimensional uncertainty is compressed into a single overall prediction uncertainty level through an uncertainty aggregation function. Their relationship can be summarized as follows: ,in This represents an indicator of the predictive uncertainty of the risk score at the current point in time. Indicates that for the first A predictive uncertainty index for a pre-defined future time window risk value. This indicates the total number of preset future time windows. This indicates the uncertainty in predicting trends of risk. Indicates a point in time The following is an overall characterization of the confidence level in current and future risk predictions. This is a pre-defined, uncertain aggregation function;

[0166] At the same time, relative to the current point in time Within a preset historical timeframe, the occurrence time of the most recent real acute cardiovascular event is statistically analyzed based on real-world annotation information. And calculate the time interval that has elapsed since the actual event occurred. ,in This indicates the time of the most recent actual acute cardiovascular event prior to the current time. Used to reflect whether real acute cardiovascular events have occurred frequently recently;

[0167] Given the overall forecast uncertainty Time interval between actual events Subsequently, the system uses a weighting mapping function to assign weighting coefficients to the false alarm evaluation index and the missed alarm evaluation index, respectively. and Their relationship can be abstractly written as ,in Indicates a point in time False alarm evaluation indicators The assigned weighting coefficients, Indicates a point in time Evaluation indicators for missed alarms The assigned weighting coefficients, To adaptively adjust the mapping function for the relative importance of false alarms and missed alarms based on model prediction uncertainty and recent real-world event occurrences, its design enables... At higher levels, it tends to increase To strengthen the constraints related to missed alarms, lower and Larger, meaning it tends to increase when no real event has occurred for a long time. To strengthen constraints related to false alarms;

[0168] Finally, the system multiplies the aforementioned weighting coefficients by their corresponding evaluation metrics and performs normalization to construct a constraint weight vector for security constraint reinforcement learning. Its form can be expressed as ,in Indicates a point in time The following is a comprehensive constraint weight used to balance the relative weights of the loss components related to false alarms and those related to missed alarms. This represents a numerical normalization function that scales or normalizes the input quantity to a preset range. In step S6 and Together, they serve as an important basis for constructing the total loss function and updating the security-constrained reinforcement learning policy network.

[0169] In this specific embodiment, S6 specifically refers to:

[0170] At the current time point Next, the training module of the security constraint reinforcement learning policy network reads the corresponding policy state parameter vectors from steps S2 and S3. And the three action parameters output at that time point Simultaneously, the early warning benefit index calculated at the same time point is read from step S5. False alarm evaluation indicators Missed alarm evaluation indicators and constraint weight vector ,in Indicates the first The current time point corresponding to each decision moment. For discrete decision step indexes, Indicates a point in time The policy state parameters output by the policy state coding network are used to comprehensively characterize the current risk level, risk change trends, prediction uncertainty, historical alarms, and individualized policy information. This indicates the first action parameter related to alarm decision-making. This represents the second action parameter related to threshold adjustment. This indicates the third action parameter related to risk assessment time interval control. Indicates a point in time The early warning benefits calculated based on real annotations and early warning records within the adjacent observation time range, This indicates the level of a false alarm, where no actual event occurred within the preset observation period but an alert was triggered. This indicates the degree of missed alerts, where a real event occurred within the preset observation time range but failed to trigger an effective early warning within the target lead time range. This represents the constraint weight vector calculated by combining the uncertainty of prediction and the occurrence of recent real events;

[0171] Secondly, the training module constructs real-time feedback signals based on early warning benefit indicators and false alarm / missed alarm evaluation indicators. Used to reflect the state and actions The overall strategy performance under the given conditions, and the relationship between the immediate return and the three evaluation metrics, can be briefly expressed as follows: ,in Indicates a point in time Instant rewards This represents a pre-designed reward constructor whose design principle is to make... right It makes a positive contribution, and and right It has a punitive effect;

[0172] Then, the policy value evaluation subnetwork obtains a higher cumulative reward based on the policy state parameters; simultaneously, the training module uses the false alarm evaluation index... and missed alarm evaluation indicators Construct false alarm constraint loss respectively and loss of alarm constraint ,in Used to characterize the extent to which the current strategy is violated in the face of preset constraints by the virtual police. These two losses are used to characterize the extent to which the current strategy is violated by the police when faced with pre-set constraints; they vary with... and Increase and increase;

[0173] Next, the training module starts from the constraint weight vector Extract the weight components for false alarm constraints and missed alarm constraints, and denote them as follows: and ,in Indicates a point in time Constraints related to false alarms The assigned weights Indicates a point in time Constraints related to missed alarms The assigned weights are adaptively determined by the prediction uncertainty and the actual event occurrence in step S5, and tend to increase when the prediction uncertainty is high or there are many recent actual events. It tends to increase when the forecast uncertainty is low and no real events have occurred for a long period of time. ;

[0174] In gaining value loss False alarm constraint loss Loss due to missed alarms and corresponding weights Then, the training module generates the total loss function according to the preset combination rules. This total loss unifies the goal of maximizing early warning benefits with the safety constraints of false alarms and missed alarms into the same optimization framework, which can be summarized as follows: ,in Indicates a point in time The overall loss value used to guide parameter updates;

[0175] Finally, the safety-constrained reinforcement learning policy network uses a gradient descent-based optimization algorithm to jointly update the parameters of its internal sub-networks. Let the set of all trainable parameters be denoted as . The learning rate is The gradient of the total loss function with respect to the parameters is Then a parameter update can be represented as ,in This represents all trainable parameters, including the policy state encoding network, the state encoding sub-network, the action output sub-network, and the policy value evaluation sub-network. This represents the step size taken along the negative gradient direction during each update. Indicates a point in time The gradient of the total loss with respect to the parameter set is used by reusing it across multiple decision steps. and constraint weights By constructing immediate rewards and total losses and performing the gradient updates described above, the updated safety-constrained reinforcement learning policy network can output better action parameters when executing step S3 in the next iteration, while satisfying the preset constraints on false alarms and missed alarms, thereby continuously improving the overall performance of early warning for acute cardiovascular events.

[0176] The above description is only a preferred embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any equivalent substitutions or modifications made by those skilled in the art within the scope of the technology disclosed in the present invention, based on the technical solution and inventive concept of the present invention, should be covered within the scope of protection of the present invention.

[0177] This invention achieves a closed loop between "risk assessment" and "early warning strategy optimization" by co-designing an uncertainty-aware deep network and a safety-constrained reinforcement learning strategy network. The front-end deep network simultaneously outputs the current risk, risks in different future time windows, risk trends, and corresponding uncertainties within the same time window, providing the strategy layer with rich state information about the event's progress and model confidence. The strategy network, considering the aforementioned multi-dimensional risk information, historical alarm behavior, individualized thresholds, and historical assessment time intervals, jointly determines whether to trigger an early warning, how to fine-tune the risk threshold, and the time interval for the next risk assessment in a single inference process through multi-head actions. After obtaining accurate annotations, it iteratively optimizes the strategy using early warning benefits, false alarm and missed alarm indicators, and constraint weights adjusted by uncertainty. Therefore, this invention achieves dynamic and adaptive control of the early warning triggering timing and risk assessment frequency in the specific physiological signal acquisition and processing flow. While ensuring controllable missed alarm risks, it effectively reduces the false alarm rate, increases early warning lead time and individualization, thereby improving the overall technical effectiveness of early warning for acute cardiovascular events.

[0178] To address the technical challenge of balancing security and resource utilization efficiency in early warning triggering timing and frequency strategies, this invention makes several targeted improvements to the algorithm structure:

[0179] First, a multi-head prediction structure is adopted in the output layer of the deep network. It not only outputs a single risk score, but also jointly outputs the future window risk and risk trend and its uncertainty. This makes the state vector input to the reinforcement learning policy network more comprehensive and more conducive to making more precise judgments on "when to start the warning" and "how far in advance the warning should be issued".

[0180] Second, a multi-head action output structure is introduced at the policy network end, which extracts "whether to issue an alarm", "threshold adjustment amount" and "next evaluation interval" from the previously fragmented manual rules, and makes unified decisions by the same policy coding vector, so that threshold adjustment and evaluation frequency scheduling can be closely linked and optimized around the current risk situation and historical alarm performance.

[0181] Third, in terms of training mechanism, the weights of the constraint losses related to false alarms and missed alarms are adaptively adjusted by predictive uncertainty, so that the strategy can automatically strengthen the suppression of missed alarms when uncertainty is high, and strengthen the suppression of false alarms when uncertainty is low and no real events have been seen for a long time.

[0182] The algorithm's structure has been enhanced to adapt to high-risk medical scenarios, achieving a better balance between security and sensitivity.

Claims

1. A method for early warning of acute cardiovascular events based on reinforcement learning, characterized in that, include: S1. Collect at least one physiological monitoring data from electrocardiogram, blood pressure, and blood oxygen signals. After preprocessing, input the physiological feature sequence into the uncertainty perception deep network to obtain the risk score of acute cardiovascular events at the current time point, the risk value and risk change trend index of occurrence within at least one preset future time window, and the corresponding prediction uncertainty index. S2. The risk score, the risk value within at least one preset future time window, the risk change trend indicator and the prediction uncertainty indicator constitute the first type of state parameters. The historical alarm records related to the monitored object, the current individualized risk threshold and the historical risk assessment time interval are read from the local storage to constitute the second type of state parameters. These are combined to obtain multiple strategy state parameters at the current time point. S3. Input multiple policy state parameters into the security constraint reinforcement learning policy network, perform policy reasoning, and obtain the first action parameter, the second action parameter, and the third action parameter. S4. Based on the second and third action parameters, update the current individualized risk threshold and the time interval for the next risk assessment. When the first action parameter triggers a cardiovascular acute event warning signal, record the warning behavior and its time information as the warning behavior record result. S5. After obtaining the real-time labeling information on whether an acute cardiovascular event has occurred, the results of the early warning behavior record are correlated to obtain the early warning benefit index, false alarm evaluation index, and missed alarm evaluation index. The false alarm evaluation index and missed alarm evaluation index are weighted according to the prediction uncertainty index to obtain the constraint weights used for safety constraint reinforcement learning. S6. Input multiple policy state parameters, early warning benefit indicators, false alarm evaluation indicators, missed alarm evaluation indicators, and constraint weights into the security constraint reinforcement learning policy network. With the early warning benefit indicator as the optimization objective, and provided that the false alarm evaluation indicators and missed alarm evaluation indicators meet the preset constraints, adjust the relative weights of false alarms and missed alarms according to the constraint weights to update the parameters for the next execution.

2. The method for early warning of acute cardiovascular events based on reinforcement learning according to claim 1, characterized in that, S1 specifically refers to: Collect at least one physiological monitoring data from the monitored subject, including electrocardiogram, blood pressure, and blood oxygen signals. Perform bandpass filtering to remove noise, baseline drift correction, and amplitude normalization on the collected physiological monitoring data from each channel to obtain preprocessed physiological monitoring data. Based on the preset time window length and sliding step size, physiological monitoring data within the time window corresponding to the current time point is extracted from the preprocessed physiological monitoring data and arranged in the order of sampling time to obtain the physiological feature sequence of the current time window; The physiological feature sequence of the current time window is input into the temporal feature extraction subnetwork of the uncertainty perception deep network, and the temporal feature extraction subnetwork is used to extract features to obtain the temporal feature vector corresponding to the current time window. The temporal feature vector is input into the multi-head output layer of the uncertainty-aware deep network, and the multi-head output layer simultaneously outputs multiple prediction results for the current time window. The multiple prediction results include at least a cardiovascular acute event risk score at the current time point, a cardiovascular acute event occurrence risk value within at least one preset future time window, a cardiovascular acute event risk change trend index, and a corresponding prediction uncertainty index. The prediction uncertainty index includes at least the prediction uncertainty for the cardiovascular acute event risk score, the prediction uncertainty for the cardiovascular acute event occurrence risk value within the at least one preset future time window, and the prediction uncertainty for the cardiovascular acute event risk change trend index.

3. The method for early warning of acute cardiovascular events based on reinforcement learning according to claim 1, characterized in that, S2 specifically refers to: The cardiovascular acute event risk score at the current time point, the cardiovascular acute event risk value within the at least one preset future time window, the cardiovascular acute event risk change trend index, and the corresponding prediction uncertainty index are arranged in a preset order and subjected to numerical normalization to obtain the first type of state parameters. Historical alarm records, current individualized risk thresholds, and historical risk assessment time intervals related to the monitored object are read from local storage. The historical alarm records are parsed to extract alarm statistics such as the previous warning time and the number of warnings within the preset historical time range. The time interval between the previous warning time and the current time point, the number of warnings, the current individualized risk threshold, and the historical risk assessment time intervals are numerically normalized to obtain the second type of state parameters. The first type of state parameters and the second type of state parameters are concatenated along the feature dimension. The concatenated features are then input into a policy state encoding network. The policy state encoding network is used to perform a nonlinear transformation to obtain multiple policy state parameters at the current time point. These multiple policy state parameters are used to characterize the risk level, risk change trend, prediction uncertainty, and the comprehensive impact of historical alarm behavior on subsequent policy decisions at the current time point.

4. The method for early warning of acute cardiovascular events based on reinforcement learning according to claim 1, characterized in that, S3 specifically refers to: Multiple policy state parameters are input into the state encoding subnetwork of the security constraint reinforcement learning policy network. The state encoding subnetwork is used to perform a nonlinear transformation on the multiple policy state parameters to obtain the policy encoding vector corresponding to the current time point. The policy encoding vector is input into the action output subnetwork of the safety constraint reinforcement learning policy network. The action output subnetwork includes at least three output heads that do not share parameters. In the same forward inference process, the action output subnetwork generates, based on the policy encoding vector, a first action parameter indicating whether a cardiovascular acute event warning is triggered at the current time point, a second action parameter indicating the threshold adjustment amount for increasing or decreasing the current individualized risk threshold, and a third action parameter indicating the risk assessment time interval to be used when executing step S1 next.

5. The method for early warning of acute cardiovascular events based on reinforcement learning according to claim 1, characterized in that, S4 specifically refers to: According to the second action parameter, the current individualized risk threshold is read from the local storage, the threshold adjustment amount indicated by the second action parameter is numerically calculated with the current individualized risk threshold to obtain the updated individualized risk threshold, and the updated individualized risk threshold is written to the local storage to replace the current individualized risk threshold. According to the third action parameter, the risk assessment time interval indicated by the third action parameter is set as the risk assessment time interval for the next execution of step S1 in the scheduling module, and the risk assessment time interval is written as the latest risk assessment time interval information into the local storage. According to the first action parameter, when the first action parameter indicates that an early warning is triggered, the control terminal device outputs a cardiovascular acute event early warning signal at the current time point, and records the early warning behavior and its corresponding current time point together as the early warning behavior recording result.

6. The method for early warning of acute cardiovascular events based on reinforcement learning according to claim 1, characterized in that, S5 specifically refers to: After obtaining the true annotation information regarding whether the monitored object has experienced an acute cardiovascular event, the true annotation information includes at least whether an acute cardiovascular event occurred within a preset observation time range and the actual occurrence time of the acute cardiovascular event. The true annotation information is aligned and matched with the early warning behavior record results by time to determine the actual event result corresponding to each early warning behavior. Based on the actual event result and the early warning behavior record results, the early warning benefit index, false alarm evaluation index, and missed alarm evaluation index for the current time point are calculated according to preset calculation rules. The early warning benefit index is used to characterize the successful triggering of an early warning within a preset advance time range before the actual occurrence of an acute cardiovascular event, with higher benefits for larger advance amounts. The false alarm evaluation index is used to characterize the number or proportion of early warnings triggered even though no acute cardiovascular event occurred within the preset observation time range. The missed alarm evaluation index is used to characterize the number or proportion of effective early warnings not triggered within the target advance time range even though an acute cardiovascular event occurred within the preset observation time range. The prediction uncertainty index corresponding to the cardiovascular acute event risk score, the cardiovascular acute event occurrence risk value within the at least one preset future time window, and the cardiovascular acute event risk change trend index is read from the prediction uncertainty index for the current time point. Weighting coefficients for false alarm evaluation index and missed alarm evaluation index are determined based on the prediction uncertainty index. This increases the weight of the missed alarm evaluation index in the constraints when prediction uncertainty is high, and increases the weight of the false alarm evaluation index in the constraints when prediction uncertainty is low and no real cardiovascular acute events have occurred for a long period. The false alarm evaluation index and missed alarm evaluation index are then weighted and normalized based on the weighting coefficients to obtain the constraint weights for safety constraint reinforcement learning.

7. The method for early warning of acute cardiovascular events based on reinforcement learning according to claim 1, characterized in that, S6 specifically refers to: Multiple policy state parameters, as well as early warning benefit indicators, false alarm evaluation indicators, missed alarm evaluation indicators, and constraint weights, are input into the training module of the security constraint reinforcement learning policy network. Combined with multiple action parameters output at the corresponding time point in step S3, an instant reward signal containing early warning benefit information, false alarm penalty information, and missed alarm penalty information is constructed. The instant reward signal is used to characterize the comprehensive policy performance obtained under the current policy state parameters and the multiple action parameters. The instantaneous reward signal is compared with the policy value predicted by the safety constraint reinforcement learning policy network based on the multiple policy state parameters. The policy evaluation error is calculated, and the constraint loss components related to the false alarm evaluation index and the missed alarm evaluation index are weighted according to the constraint weights to form a total loss function that simultaneously includes the early warning benefit index, the false alarm evaluation index, and the missed alarm evaluation index. This increases the weight of the missed alarm-related constraint loss in the total loss function when the prediction uncertainty is high, and increases the weight of the false alarm-related constraint loss in the total loss function when the prediction uncertainty is low and no real acute cardiovascular events have occurred for a long time. Based on the total loss function, a gradient descent-type optimization algorithm is used to jointly update the parameters of the state encoding subnetwork and the action output subnetwork in the safety constraint reinforcement learning policy network to obtain the updated safety constraint reinforcement learning policy network. In the next execution of step S3, the updated safety constraint reinforcement learning policy network is used to perform policy inference on the new multiple policy state parameters.

Images