A critical illness dynamic intervention decision support method, device, equipment and medium

Abstract

The application discloses a critical illness dynamic intervention decision support method, device, equipment and medium, and the method comprises the following steps: collecting multi-source longitudinal clinical data of a target critical illness patient and performing compliance management, and constructing a patient-level longitudinal time series dataset; taking the time of diagnosis or onset as the time starting point, performing time alignment on the longitudinal time series dataset according to three follow-up stages, and generating three-stage trajectory samples; constructing a staged state representation vector and a mixed clinical action space based on the three-stage trajectory samples, and on this basis, constructing a four-network fusion architecture composed of a strategy generation network, a value evaluation network, an online value estimation network and a target value estimation network; using the trained networks to infer the real-time state of each follow-up stage of the target patient, and obtaining a staged intervention decision suggestion for the target patient. The application can ensure that the intervention decision conforms to the diagnosis and treatment priority of each stage, and realizes the maximization of long-term clinical benefits of the patient.

Description

Technical Field

[0001] This application relates to the fields of artificial intelligence and clinical decision support technology, specifically to a method, device, equipment, and medium for dynamic intervention decision support for critical illnesses. Background Technology

[0002] Critical illnesses such as acute kidney injury, severe pneumonia, and acute respiratory distress syndrome are characterized by rapid onset, rapid progression, highly variable prognosis, and complex disease trajectories. If critically ill patients do not receive timely intervention, their condition can deteriorate rapidly in the short term, continue to progress in the medium term, and significantly increase the risk of developing chronic diseases or even death in the long term, posing a significant challenge to clinical diagnosis and treatment.

[0003] Currently, clinical intervention decisions mainly rely on the experience and judgment of healthcare professionals, or on predictive models built using traditional statistical methods and conventional machine learning. However, existing methods generally have the following limitations: First, most of them are single-point static prediction models, which cannot effectively integrate longitudinal time-series follow-up data and are difficult to reflect the dynamic trend of the patient's condition over time, thus limiting the timeliness and clinical applicability of the models.

[0004] Second, although reinforcement learning has shown potential in the field of dynamic decision-making, the conventional DQN (Deep Q-Network) architecture has a significant problem of Q-value overestimation. When dealing with high-dimensional, sparse and noisy clinical time-series data, it is prone to prediction bias, which leads to unstable training.

[0005] Third, most existing models only support discrete action outputs. For continuous intervention parameters that are prevalent in clinical practice, such as drug dosage and flow rate, coarse discretization processing is often required, resulting in insufficient intervention precision and an inability to meet the needs of refined clinical control.

[0006] Fourth, the course of severe illness has significant stage-specific characteristics. Existing methods lack dedicated reinforcement learning mechanisms designed for key clinical features at each stage, and have not established a tiered reward mechanism that aligns with clinical treatment priorities. This results in insufficient sensitivity in identifying high-risk outcomes, making it difficult to maximize long-term benefits.

[0007] Therefore, building an intelligent model that can integrate multi-stage longitudinal follow-up information, has dynamic updating and real-time evaluation capabilities, and can accurately output continuous intervention parameters and make personalized medical decisions remains a technical problem that urgently needs to be solved in this field. Summary of the Invention

[0008] To address the aforementioned technical problems, embodiments of this application provide a method, apparatus, device, and medium for dynamic intervention decision support for critical illnesses.

[0009] In a first aspect, embodiments of this application provide a dynamic intervention decision support method for critical illnesses, the method comprising: Collect multi-source clinical longitudinal data of patients with target severe illnesses and conduct compliance governance to construct a patient-level longitudinal time series dataset; Using the time of diagnosis or onset of illness as the starting point, the longitudinal time series dataset is time-aligned according to three follow-up stages: short-term, medium-term, and long-term, to generate three-stage trajectory samples. Based on the three-stage trajectory samples, a phased state representation vector and a hybrid clinical action space are constructed. On the basis of the phased state representation vector and the hybrid clinical action space, a four-network fusion architecture consisting of a policy generation network, a value assessment network, an online value estimation network, and a target value estimation network is constructed. A hierarchical composite reward function is constructed based on the clinical goals of each follow-up stage. The strategy generation network, the value assessment network, the online value estimation network, and the target value estimation network are then collaboratively trained with their respective optimization goals using a loss function with a regularized penalty term. The trained networks are then used to infer the real-time status of the target patient at each follow-up stage to obtain short-term, medium-term, and long-term phased intervention decision recommendations for the target patient. In some embodiments of this application, the collection of multi-source clinical longitudinal data from patients with target severe illnesses and the implementation of compliance governance to construct a patient-level longitudinal time-series dataset includes: Longitudinal clinical data from multiple sources were collected, including at least core laboratory indicators, vital signs, clinical diagnosis, history of underlying diseases, surgical or interventional records, medication regimens, and follow-up outcome data. The multi-source clinical longitudinal data were sequentially subjected to de-identification, standardization, and quality control processing. Using patient identifiers as the primary key and timestamps as the index, a patient-level longitudinal time-series dataset with short-term, medium-term, and long-term stage annotations is constructed.

[0010] In some embodiments of this application, the step of aligning the longitudinal time-series dataset according to three follow-up stages—short-term, medium-term, and long-term—using the time of diagnosis or onset as the starting point to generate three-stage trajectory samples includes: The longitudinal time series dataset is divided into stages based on the time of diagnosis or onset of illness, according to short-term, medium-term and long-term follow-up time windows. The time window for each follow-up stage is a configurable parameter. Data from each patient's follow-up phases were aggregated and aligned. The aggregated data from each stage is organized into a quadruple containing the current patient status, clinical intervention actions, stage rewards, and the patient status for the next stage, thus obtaining the three-stage trajectory sample.

[0011] In some embodiments of this application, the construction of a phased state representation vector and a hybrid clinical action space based on the three-stage trajectory samples includes: Based on the three-stage trajectory samples, the weights and time window lengths of each feature subset are configured differently according to the short-term, medium-term and long-term follow-up objectives. The corresponding features are extracted from each configured feature subset, normalized and then concatenated to obtain the staged state representation vector. Based on the clinical constraint rule base, a set of discrete intervention actions matching the clinical goals of each follow-up stage is generated according to the diagnosis and treatment guidelines of each follow-up stage. The discrete intervention actions include at least one or more of the following: intervention category selection, protocol switching, monitoring intensity adjustment, and manual review triggering. Based on the clinical constraint rule base, an action mask matrix is ​​generated in real time according to the rule mapping relationship between each clinical contraindication condition and the intervention action to shield contraindicated interventions, and boundary constraints are set for the continuous intervention parameters bound to the discrete intervention action according to the action space configuration parameters. The discrete set of intervention actions and the corresponding continuous intervention parameters together constitute a hybrid clinical action space.

[0012] In some embodiments of this application, the construction of a four-network fusion architecture consisting of a policy generation network, a value assessment network, an online value estimation network, and a target value estimation network based on the phased state representation vector and the hybrid clinical action space includes: A strategy generation network is constructed, taking the phased state representation vector as input, outputting the selection probability distribution of each discrete intervention action through the discrete action output branch, and outputting the continuous intervention parameter values ​​through the continuous parameter output branch and applying truncation constraints. A value assessment network is constructed, with the phased state representation vector and action pair as joint inputs. The state value function and the state action value function are fitted simultaneously. The advantage estimate formed by the difference between the two is used as the gradient signal to guide the update of the policy generation network parameters. The policy loss is minimized with the goal of maximizing the expected action value output of the value assessment network. The parameter update also introduces a clinical safety regularization term to impose constraints and prune invalid actions and out-of-bounds continuous parameters. Construct an online value estimation network and a target value estimation network with independent parameters. The online value estimation network estimates the value of all candidate actions in the current state and selects the optimal action. The target value estimation network receives the optimal action and performs independent value estimation. The state action value output of the value evaluation network is corrected by multiplying the decoupled estimation results of the two networks by a value calibration coefficient. A four-network fusion architecture is obtained, consisting of the policy generation network, the value assessment network, the online value estimation network, and the target value estimation network. The collaborative training of the four-network fusion architecture uses a differential discount factor in the follow-up phase to calculate the target value, and the discount factor is configured to increase in the short-term, medium-term, and long-term phases.

[0013] In some embodiments of this application, the step of correcting the state-action value output of the value assessment network by multiplying the decoupling estimation results of the two networks by a value calibration coefficient includes: The action corresponding to the highest value estimated by the online value estimation network for all candidate actions in the current state is taken as the benchmark action; The baseline action is input into the target value estimation network to obtain the target value; The difference between the target value and the current output of the value evaluation network is multiplied by the value calibration coefficient and then added to the output of the value evaluation network. The corrected state-action value is also used as the basis for calculating the advantage estimate, so as to synergistically improve the accuracy of intervention strategy selection and training stability.

[0014] In some embodiments of this application, the construction of a hierarchical composite reward function guided by clinical goals at each follow-up stage includes: With the goal of early reversal or control in the short term, the short-term efficacy reward component is calculated by weighting the positive improvement of the baseline reward value and core clinical indicators relative to the short-term stage judgment threshold, and deducting the abnormal increase in the amount of penalty. The short-term safety risk penalty component and resource cost penalty component are also integrated. With the goal of stabilizing the condition and preventing its deterioration in the medium term, the mid-term efficacy reward component is calculated using the same magnitude weighting calculation logic as the short term, and the mid-term safety risk penalty component is also incorporated. With the goal of preventing adverse outcomes in the long term, the long-term efficacy reward component is calculated using the same magnitude weighting calculation logic as the short-term component, and the long-term safety risk penalty component is also incorporated. The threshold values, magnitude weighting coefficients, and reward / penalty weights for each stage are uniformly managed by the reward function configuration parameters, resulting in a hierarchical composite reward function.

[0015] In some embodiments of this application, the loss function with regularization penalty includes at least a temporal difference mean square error term, a sparsity regularization term, a weight decay regularization term, a value estimation upper limit penalty term, and a policy generation network loss weighting term; the step of co-training the policy generation network, the value evaluation network, and the online value estimation network with the loss function with regularization penalty under their respective optimization objectives includes: The value assessment network updates its parameters with the goal of minimizing the mean square error of the time-series difference. The policy generation network updates its parameters with the optimization objective of minimizing the loss weighting term of the policy generation network. The loss weighting term of the policy generation network is optimized with the expected value of the state-action output by the value evaluation network, and its contribution ratio in the total loss is controlled by the weighting coefficient. The online value estimation network updates its parameters with the optimization objective of minimizing the total loss function containing all five terms. The value estimation upper limit penalty term applies a penalty when the output of the online value estimation network exceeds a preset value upper limit. The target value estimation network periodically synchronizes its parameters from the online value estimation network using a soft update method. The soft update coefficient is a configurable parameter, resulting in the networks that have completed the collaborative training.

[0016] In some embodiments of this application, the step of using the trained networks to infer the real-time status of the target patient at each follow-up stage to obtain short-term, medium-term, and long-term phased intervention decision recommendations for the target patient includes: The strategy is used to generate a network to infer the real-time status of the target patient at each follow-up stage and output a phased recommendation plan that includes discrete intervention actions and continuous intervention parameters. Uncertainty assessment and out-of-distribution detection are performed on the inference output. When the uncertainty exceeds the limit or the out-of-distribution detection is triggered, a conservative action set is output or manual review is triggered, and the triggering reason, patient identifier and triggering time are recorded. The results of intervention execution and follow-up outcomes at each stage are collected and added to the three-stage trajectory sample. Closed-loop incremental fine-tuning is performed periodically based on the loss function with regularized penalty term. The masking rules and boundary constraints of the hybrid clinical action space and the judgment thresholds of each stage of the hierarchical composite reward function are updated simultaneously to obtain short-term, medium-term and long-term phased intervention decision suggestions for target patients.

[0017] A second aspect of this application provides a dynamic intervention decision support device for critical illnesses, the device comprising: The data acquisition module is used to collect multi-source clinical longitudinal data of patients with target severe diseases and to perform compliance governance to build a patient-level longitudinal time series dataset. The alignment module is used to perform time-series alignment of the longitudinal time-series dataset according to three follow-up stages: short-term, medium-term, and long-term, with the time of diagnosis or onset of illness as the starting point, and generate three-stage trajectory samples. The construction module is used to construct a phased state representation vector and a hybrid clinical action space based on the three-stage trajectory samples, and to construct a four-network fusion architecture consisting of a policy generation network, a value assessment network, an online value estimation network and a target value estimation network based on the phased state representation vector and the hybrid clinical action space. The decision-making module is used to construct a hierarchical composite reward function guided by the clinical goals of each follow-up stage. The strategy generation network, the value assessment network, the online value estimation network, and the target value estimation network are jointly trained with their respective optimization goals using a loss function with a regularized penalty term. The trained networks are then used to infer the real-time status of the target patient at each follow-up stage to obtain short-term, medium-term, and long-term phased intervention decision recommendations for the target patient.

[0018] A third aspect of this application provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the dynamic intervention decision support method for critical illnesses described in the embodiments of this application.

[0019] A fourth aspect of this application provides a computer-readable storage medium having a computer program stored thereon, the computer program being executed by a processor to implement the dynamic intervention decision support method for critical illnesses described in the embodiments of this application.

[0020] The dynamic intervention decision support method for critical illnesses provided in this application collects multi-source clinical longitudinal data from patients with target critical illnesses and performs compliance governance to construct a patient-level longitudinal time-series dataset. Using the time of diagnosis or onset as the starting point, the longitudinal time-series dataset is time-aligned according to three follow-up stages: short-term, medium-term, and long-term, generating three-stage trajectory samples. This achieves dynamic modeling of the entire course of critical illness, solving the problem that traditional static models cannot capture the evolution trend of the disease. Based on the three-stage trajectory samples, a phased state representation vector and a hybrid clinical action space are constructed. Furthermore, based on the phased state representation vector and the hybrid clinical action space, a strategy generation network, a value assessment network, and an online price evaluation network are constructed. A four-network fusion architecture consisting of a value estimation network and a target value estimation network is adopted to balance the refined output of continuous intervention parameters and the suppression of Q-value overestimation, thereby improving the stability and clinical suitability of the intervention strategy. Then, a hierarchical composite reward function is constructed with the clinical goals of each follow-up stage as the guide. The strategy generation network, the value assessment network, and the online value estimation network are co-trained with their respective optimization goals using a loss function with a regularized penalty term. The trained networks are used to infer the real-time status of the target patient at each follow-up stage to obtain short-term, medium-term, and long-term phased intervention decision suggestions for the target patient, thereby ensuring that the intervention decision is in line with the treatment priority of each stage and maximizing the long-term clinical benefit of the patient.

[0021] The above description is merely an overview of the technical solutions of the embodiments of this application. In order to better understand the technical means of this application and to implement it in accordance with the contents of the specification, and to make the above and other objects, features and advantages of the embodiments of this application more obvious and understandable, specific implementation methods of this application are described below. Attached Figure Description

[0022] Various other advantages and benefits will become apparent to those skilled in the art upon reading the detailed description of the preferred embodiments below. The accompanying drawings are for illustrative purposes only and are not intended to limit the scope of this application. Furthermore, the same reference numerals denote the same parts throughout the drawings. In the drawings: Figure 1 This is a schematic diagram illustrating the steps of the dynamic intervention decision support method for critical illnesses provided in some embodiments of this application; Figure 2 This application provides a schematic diagram illustrating the steps for constructing a phased state representation vector and a hybrid clinical action space based on the three-stage trajectory samples in some embodiments. Figure 3 A schematic diagram of a four-network converged architecture provided in some embodiments of this application; Figure 4 This is a schematic diagram of the structure of a dynamic intervention decision support device for critical illnesses provided in an embodiment of this application; Figure 5 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Detailed Implementation

[0023] The embodiments of the technical solution of this application will now be described in detail with reference to the accompanying drawings. These embodiments are only used to more clearly illustrate the technical solution of this application and are therefore merely examples, and should not be used to limit the scope of protection of this application.

[0024] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which embodiments of this application pertain; the terminology used herein is for the purpose of describing particular embodiments only and is not intended to limit the embodiments of this application; the terms “comprising” and “having”, and any variations thereof, in the specification, claims, and foregoing description of the drawings of this application are intended to cover non-exclusive inclusion.

[0025] In the description of the embodiments of this application, "multiple" means two or more, unless otherwise explicitly specified.

[0026] In this document, the term "embodiment" means that a particular feature, structure, or characteristic described in connection with an embodiment may be included in at least one embodiment of this application. The appearance of this phrase in various places throughout the specification does not necessarily refer to the same embodiment, nor is it a separate or alternative embodiment mutually exclusive with other embodiments. It will be explicitly and implicitly understood by those skilled in the art that the embodiments described herein can be combined with other embodiments.

[0027] In the description of the embodiments in this application, the term "and / or" is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, and B existing alone. Additionally, the character " / " in this document generally indicates that the preceding and following related objects have an "or" relationship.

[0028] Critical illnesses such as acute kidney injury, severe pneumonia, and sepsis are complex clinical syndromes with insidious or abrupt onset, non-linear disease progression, and significant individual prognostic differences. Without precise, time-series intervention, these patients often face the risk of short-term acute deterioration, persistent organ dysfunction in the medium term, and long-term chronicity or death, placing extremely high demands on the timeliness and accuracy of clinical decision-making. Existing decision support methods for critical illnesses mainly rely on traditional statistical models or conventional machine learning algorithms. These methods are often limited to static risk assessment at a single time point, essentially representing a "retrospective" fit. They lack the ability to dynamically integrate longitudinal follow-up data across short-, medium-, and long-term stages, making it difficult to capture the continuous trajectory and inflection points of disease progression. This limits the generalization ability and practicality of the models in real-world clinical scenarios.

[0029] With the development of artificial intelligence technology, deep reinforcement learning has provided a new paradigm for solving the aforementioned dynamic decision-making problems. In particular, the DQN (Deep Q-Network) series of algorithms has demonstrated its advantages in sequential decision-making in fields such as game theory. However, when applied to critical care medicine scenarios, it faces severe technical adaptability issues: First, the problem of Q-value overestimation. Conventional DQN architectures are prone to overly optimistic estimations of action value in high-dimensional, sparse, and noisy clinical time-series data, leading to intervention strategies deviating from the clinically optimal solution and making training stability difficult to guarantee; Second, rigid action space. Existing models mostly use a single discrete action space, which cannot adapt to the numerous continuous intervention parameters present in clinical practice, such as drug dosage and flow rate regulation, limiting the precision of intervention; Third, the lack of a phased mechanism. The course of critical illness has obvious phased characteristics, and the clinical goals and focus indicators for different phases, such as acute phase control, plateau phase stabilization, and recovery phase relapse prevention, are drastically different. Existing technologies have failed to design differentiated state representation and reward feedback mechanisms for these stage characteristics, resulting in insufficient sensitivity of the model in identifying high-risk outcomes and failing to maximize clinical benefits for patients throughout their entire life cycle.

[0030] It is evident that overcoming the overestimation defect of conventional DQN, constructing a hybrid action space adapted to the characteristics of critical care medicine, and utilizing multi-stage temporal information to achieve accurate and stable dynamic intervention decisions are technical bottlenecks that urgently need to be overcome in this field. Therefore, embodiments of this application provide a method for dynamic intervention decision support in critical care diseases, such as... Figure 1 As shown, the method includes: S1. Collect multi-source clinical longitudinal data of patients with target severe diseases and conduct compliance governance to construct a patient-level longitudinal time series dataset.

[0031] In one specific implementation, multi-source clinical longitudinal data, including at least core laboratory indicators, vital signs, basic medical history, surgical or interventional records, medication regimens, and follow-up outcome data, is collected from a multi-source clinical system. The multi-source clinical longitudinal data is then subjected to de-identification, standardization, and quality control processing in sequence. A patient-level longitudinal time-series dataset with short-term, medium-term, and long-term stage annotations is constructed using the patient identifier as the primary key and the timestamp as the index.

[0032] Taking acute kidney injury (AKI) as an example, comprehensive longitudinal clinical data of AKI patients were collected from a multi-source clinical system. The core laboratory indicators corresponded to 12 features in the feature subset, including serum creatinine, blood urea nitrogen, cystatin, hourly urine output, serum potassium, serum sodium, serum chloride, serum calcium, white blood cell count, hemoglobin, platelets, and C-reactive protein (CRP). Vital signs data corresponded to 6 features in the feature subset, including heart rate, systolic blood pressure, diastolic blood pressure, mean arterial pressure, body temperature, and percutaneous oxygen saturation. Clinical diagnostic data included AKI staging and history of underlying diseases. In addition, surgical or interventional records, medication records, and follow-up outcome data were also included.

[0033] Furthermore, the aforementioned data underwent the following compliance governance processes in sequence: de-identification and anonymization, replacing direct identifiers such as patient names and ID numbers while retaining unique anonymous patient IDs; standardization processing, normalizing test indicators, aligning timestamps to a unified format, and uniformly mapping disease and drug codes; quality control processing, removing duplicate records, outliers exceeding medically reasonable ranges, and samples with missing values ​​exceeding a preset threshold, and employing forward imputation or mean imputation strategies for a small number of missing values. Finally, using patient IDs as the primary key and timestamps as the index, a patient-level longitudinal time-series dataset with short-term, medium-term, and long-term stage annotations was constructed.

[0034] As an alternative implementation, a federated learning framework can be introduced to support multi-center joint training; data can be accessed from multi-source systems through the HL7FHIR standard interface; and the collection scope can be adjusted according to the clinical characteristics of other critical diseases without changing the core technical architecture.

[0035] S2. Using the time of diagnosis or onset of illness as the starting point, the longitudinal time series dataset is time-aligned according to three follow-up stages: short-term, medium-term, and long-term, to generate three-stage trajectory samples.

[0036] In one specific implementation, the data in the longitudinal time series dataset is divided into stages according to short-term, medium-term, and long-term follow-up time windows, with each follow-up stage time window being a configurable parameter. The data of each patient in each follow-up stage are aggregated and aligned. The aggregated data of each stage is organized into a quadruple containing the current stage patient status, clinical intervention action, stage reward, and the next stage patient status to obtain the three-stage trajectory sample.

[0037] Taking AKI as an example, with the time of diagnosis or onset as the time anchor, the data for each stage is divided and aggregated according to three follow-up time windows: short-term (0-72 hours), medium-term (3-7 days), and long-term (7-90 days). All time windows are configurable parameters. The aggregated data for each stage is organized into quadruple trajectory samples (s). t , a t , r t , st+1 ), where s t Let a be the phased state representation vector at the current moment. t For clinical intervention actions, r t The stage reward value is calculated using a hierarchical composite reward function; s t+1 This serves as the state representation vector for the next stage. When data is missing for a patient at a certain follow-up stage, strategies such as interpolation completion or weighting of missing state markers can be used to address this.

[0038] S3. Based on the three-stage trajectory samples, construct a phased state representation vector and a hybrid clinical action space. On the basis of the phased state representation vector and the hybrid clinical action space, construct a four-network fusion architecture consisting of a policy generation network, a value assessment network, an online value estimation network, and a target value estimation network.

[0039] In one possible implementation, refer to Figure 2 Based on the three-stage trajectory samples, a phased state representation vector and a hybrid clinical action space are constructed, including S31-S34: S31. Based on the three-stage trajectory samples, weights and time window lengths are configured differently for each feature subset according to short-term, medium-term, and long-term follow-up goals. Corresponding features are extracted from each configured feature subset, normalized, and then concatenated to obtain a phased state representation vector. This phased state representation vector includes at least two or more categories from the vital signs feature subset, the test indicator feature subset, the disease-specific feature subset, and the longitudinal time-series feature subset. The weights and time window lengths of each feature subset are configured differently according to short-term, medium-term, and long-term stages. The vital signs feature subset includes at least heart rate, blood pressure, and blood oxygen saturation; the test indicator feature subset includes at least core indicators of target organ function and electrolyte indicators; the disease-specific feature subset includes at least the target disease stage, history of underlying diseases, and relevant drug exposure markers; and the longitudinal time-series feature subset includes at least the changing trend and fluctuation amplitude of core indicators within the time-series window.

[0040] The weights of each feature subset are allocated differently according to a phased strategy. For example, in the short term, the key weights are allocated to early warning indicators such as serum creatinine and urine output; in the medium term, the weights are evenly distributed; and in the long term, the key weights are allocated to indicators of renal function recovery and time-series trend features. For other severe diseases, the feature subsets can be expanded by modifying the state space configuration parameters without changing the model structure.

[0041] S32. Based on the clinical constraint rule base, generate a set of discrete intervention actions that match the clinical goals of each follow-up stage according to the diagnosis and treatment guidelines of each follow-up stage. The discrete intervention actions include at least one or more of the following: intervention category selection, protocol switching, monitoring intensity adjustment, and manual review triggering.

[0042] For example, based on the treatment guidelines for each follow-up stage, a set of discrete intervention actions matching the clinical goals of each stage can be generated. Taking AKI as an example: in the short term (0 to 72 hours), discrete intervention actions include volume management, reduction or discontinuation of nephrotoxic drugs, initiation of initial renal protection strategies, adjustment of monitoring intensity, and conservative maintenance or manual review triggering; in the medium term (3 to 7 days), these include optimization of renal protection drug dosage, complication management strategies, adjustment of monitoring frequency, and conservative maintenance or manual review triggering; in the long term (7 to 90 days), these include the development of long-term renal protection strategies, setting of chronic disease management goals, development of follow-up plans, and conservative follow-up or manual review triggering.

[0043] S33. Based on the clinical constraint rule base, an action mask matrix is ​​generated in real time according to the rule mapping relationship between each clinical contraindication condition and the intervention action to shield contraindicated interventions, and boundary constraints are set for the continuous intervention parameters bound to the discrete intervention action according to the action space configuration parameters.

[0044] Specifically, the action mask matrix is ​​generated in real time based on a clinical constraint rule base, mapping each clinical contraindication condition to its corresponding intervention action. Taking AKI as an example, potassium supplementation-related discrete intervention actions are masked when serum potassium exceeds 5.5 mmol / L; the maximum dose level of diuretics is masked when urine output is not less than 0.5 ml / kg / h; and anticoagulation-related intervention actions are masked when active bleeding is present. The action mask rule list is uniformly maintained by action space configuration parameters and supports runtime hot updates, physically preventing the possibility of contraindicated interventions being output.

[0045] The boundary constraints of continuous intervention parameters are configured by the continuous action constraint boundary parameters in the action space configuration parameters. Taking AKI as an example, the continuous intervention parameters and their constraint boundaries are updated as follows: the continuous infusion rate of isotonic crystalloid is 0.5–3.0 mL·kg⁻¹. - ¹·h - ¹; In cases of volume overload, the continuous intravenous infusion rate of furosemide is 0.05–0.3 mg / kg. - ¹·h - ¹, the upper limit of furosemide per day is ≤600 mg / d; the ultrafiltration rate is usually 20–25 mL·kg. - ¹·h - ¹, for active bleeding, 5–15 mL / kg - ¹·h - ¹; Systemic heparin anticoagulation 5–15 U / kg when there is no bleeding. - ¹·h - ¹; Outpatient follow-up intervals are 7 to 90 days. Continuous parameters are subject to a truncation operation after the output layer of the policy generation network to enforce boundary constraints, ensuring that the output parameters remain within clinically safe ranges.

[0046] S34. The discrete intervention action set and the corresponding continuous intervention parameters together constitute a hybrid clinical action space.

[0047] The set of discrete intervention actions and the corresponding set of continuous intervention parameters together constitute a hybrid clinical action space. ,in It is a set of discrete actions, including enumerated decisions such as intervention category selection, plan switching, conservative maintenance, and manual review triggering; This is a set of continuous parameters bound to discrete actions, including numerical parameters such as dose, rate, frequency, and interval. Compared to the limitations of traditional DQN, which can only handle discrete action spaces, the hybrid action space design eliminates the need for discretization of continuous intervention parameters, fundamentally solving the problem of accuracy loss.

[0048] In another specific implementation, based on the phased state representation vector and the hybrid clinical action space, when constructing a four-network fusion architecture consisting of a policy generation network, a value assessment network, an online value estimation network and a target value estimation network, the policy generation network is first constructed. Taking the phased state representation vector as input, it outputs the selection probability distribution of each discrete intervention action through the discrete action output branch, and outputs the continuous intervention parameter values ​​and applies truncation constraints through the continuous parameter output branch.

[0049] Based on the aforementioned phased state representation vector and hybrid clinical action space, this application's embodiments construct a four-network fusion architecture based on Double DQN and Actor-Critic, such as... Figure 3 As shown, the architecture comprises a policy generation network, a value evaluation network, an online value estimation network, and a target value estimation network. The core decoupling relationship of this architecture is manifested in the collaborative decoupling between the Double DQN and Actor-Critic mechanisms: policy optimization is handled by Actor-Critic, while Q-value estimation is handled by Double DQN. The two work independently and collaboratively, meaning that the decoupling described above is an architectural-level decoupling between Double DQN and Actor-Critic as a whole, rather than a parameter-level decoupling between the online network and the target network within Double DQN.

[0050] Specifically, the policy generation network uses the phased state representation vector s tThe input layer uses a fully connected neural network architecture with a default of 3 layers. The hidden layer dimensions are 256, 128, and 64 respectively, and the activation function is ReLU. The output layer has two branches: the discrete action output branch is activated by Softmax and outputs the probability distribution of the selection of each discrete intervention action; the continuous parameter output branch is transformed by linear transformation to output the mean of the continuous intervention parameters, which is then truncated and restricted to the boundary range specified by the action space configuration parameters, and directly used as the output of the continuous intervention parameters. The output of the discrete action branch of the policy generation network is filtered by the action mask matrix to determine the final discrete intervention action selection, which, together with the output of the continuous parameter branch, forms a complete hybrid intervention action. This enables the coordinated output of intervention categories and corresponding parameters.

[0051] Furthermore, a value assessment network is constructed with staged state representation vectors and action pairs as joint inputs. Simultaneously, the state value function and the state-action value function are fitted. The advantage estimate formed by the difference between the two is used as the gradient signal to guide the update of the policy generation network parameters. The optimization objective is to minimize the policy loss by maximizing the expected action value output of the value assessment network. At the same time, a clinical safety regularization term is introduced to impose constraints and prune invalid actions and out-of-bounds continuous parameters during parameter updates.

[0052] The value assessment network here uses a phased state representation vector s. t With action a t The concatenated vector serves as the joint input, employing a fully connected network with the same architecture as the policy generation network. It simultaneously fits the state value function V and the state-action value function Q, where V represents the expected value of all actions in the current state. The difference between these two functions constitutes the advantage function, and the advantage estimate serves as the gradient signal. This advantage-weighted policy gradient algorithm guides the parameter updates of the policy generation network. The optimization objective of the policy generation network is to maximize the expected action value output by the value assessment network, i.e., minimize the policy loss. Parameter updates also incorporate a clinical safety regularization term, imposing constraints on the probability of invalid actions masked by the trigger action mask matrix and on continuous intervention parameters exceeding boundary constraints. This ensures clinical safety at the gradient level during policy optimization.

[0053] Furthermore, an online value estimation network and a target value estimation network with mutually independent parameters are constructed. The online value estimation network estimates the value of all candidate actions in the current state and selects the optimal action. The target value estimation network receives the optimal action and performs independent value estimation. The state action value output of the value evaluation network is corrected by multiplying the decoupled estimation results of the two networks by the value calibration coefficient.

[0054] It should be noted that the online value estimation network and the target value estimation network have the same structure but completely independent parameters and do not share them. The online value estimation network is responsible for estimating the value of all candidate actions separately and selecting the optimal action; the target value estimation network receives the optimal action and performs independent value estimation to obtain the target Q value.

[0055] Finally, a four-network fusion architecture was obtained, consisting of a policy generation network, a value assessment network, an online value estimation network, and a target value estimation network. The collaborative training of this architecture employs a differentiated discount factor calculated based on follow-up phases, with the discount factor increasing progressively for short-term, medium-term, and long-term phases. Specifically, the state-action value output of the value assessment network is corrected by multiplying the decoupled estimation results of the online value estimation network and the target value estimation network by a value calibration coefficient. In practice, the action with the highest value estimated by the online value estimation network for all candidate actions in the current state is used as the baseline action. This baseline action is then input into the target value estimation network to obtain the target value. The difference between the target value and the current output of the value assessment network is multiplied by the value calibration coefficient and added to the output of the value assessment network. This corrected state-action value also serves as the basis for calculating the advantage estimate, synergistically improving the accuracy of intervention strategy selection and training stability.

[0056] S4. Construct a hierarchical composite reward function guided by the clinical goals of each follow-up stage. Use a loss function with regularization penalty term to collaboratively train the strategy generation network, the value evaluation network, and the online value estimation network with their respective optimization goals. Use the trained networks to infer the real-time status of the target patient at each follow-up stage to obtain short-term, medium-term, and long-term phased intervention decision suggestions for the target patient.

[0057] In one possible implementation, a hierarchical composite reward function is constructed based on the clinical goals of each follow-up stage. This includes: For short-term early reversal or control, a short-term efficacy reward component is calculated by weighting the positive improvement of the baseline reward value and core clinical indicators relative to the short-term stage judgment threshold, and deducting a weighted penalty for abnormal increases, while also incorporating short-term safety risk penalty and resource cost penalty components; For medium-term disease stabilization and deterioration prevention, a medium-term efficacy reward component is calculated using the same weighted calculation logic as the short-term approach, while also incorporating the medium-term safety risk penalty component; For long-term prevention of adverse outcomes, a long-term efficacy reward component is calculated using the same weighted calculation logic as the short-term approach, while also incorporating the long-term safety risk penalty component; The judgment thresholds, weighted coefficients, and reward / penalty weights for each stage are uniformly managed by the reward function configuration parameters, resulting in the hierarchical composite reward function.

[0058] The therapeutic effect reward for each of the above stages is uniformly calculated using the following weighted formula: + Δx- Δy, where, These serve as the benchmark values ​​for rewards and penalties at each stage. To reward the gradient coefficient, The penalty gradient coefficient is Δx, where Δx is the core clinical indicator relative to the stage determination threshold α. threshold The positive improvement is denoted as follows: in the short term, Δx represents the degree of improvement in early reversal or control of severe illness; in the medium term, Δx represents the degree of improvement in disease stability; and in the long term, Δx represents the degree of improvement in the prevention and control of adverse outcomes. Δy is the core clinical indicator relative to α. threshold The abnormal increase in Δy is categorized into three levels: short-term Δy-side weighting of the degree of unreversible or uncontrolled severe illness, medium-term Δy-side weighting of the risk of disease deterioration, and long-term Δy-side weighting of the risk of adverse outcomes. The reward interval, judgment threshold, and reward / penalty values ​​for each stage are adaptively configured by the reward function configuration parameters in the global configuration set α.

[0059] This hierarchical composite reward function is designed to explicitly distinguish the treatment priorities at each stage during model training. It ensures that short-term priority is reversal or control, medium-term focus is on maintaining stability, and long-term goal is to prevent adverse outcomes, thereby maximizing long-term clinical benefits for patients. The decision thresholds, magnitude weighting coefficients, and reward / penalty weights for each stage are uniformly managed by the reward function configuration parameters, resulting in the hierarchical composite reward function.

[0060] In this embodiment, the loss function with regularization penalty includes at least a temporal difference mean squared error term, a sparsity regularization term, a weight decay regularization term, a value estimation upper bound penalty term, and a policy generation network loss weighting term; the loss function with regularization penalty is:

[0061] In the formula: is the TD mean squared error loss term, where N is the number of training samples in the batch; Divided into L1 regularization terms, where The L1 regularization coefficient is... The L1 norm of all trainable parameters of the model; For L2 regularization terms, where The L2 regularization coefficient is... The square of the L2 norm of all trainable parameters of the model; This is a penalty term for the upper limit of Q value, where The penalty coefficient is truncated for the Q value. This is the upper limit of the preset reasonable range for the Q value; For Actor network loss weight coefficients, The policy loss of the Actor network is expressed as the action value evaluation value output by the Critic network, with the expression being... ,in The action value assessment value output by the Critic network.

[0062] The policy generation network, value evaluation network, and online value estimation network are co-trained using a loss function with regularization penalties, each with its own optimization objective. Specifically: the value evaluation network updates its parameters to minimize the temporal difference mean squared error; the policy generation network updates its parameters to minimize the policy generation network loss weighting term, with the expected value of the state-action output of the value evaluation network as its optimization objective, and its contribution to the total loss controlled by weight coefficients; the online value estimation network updates its parameters to minimize the total loss function containing all five terms, with a value estimation cap penalty applied when the output of the online value estimation network exceeds a preset value cap; and the target value estimation network periodically synchronizes its parameters from the online value estimation network using a soft update method, with the soft update coefficient being a configurable parameter. The target network employs a soft update strategy, and the update formula is as follows: θ represents the online network parameters, θ' represents the target network parameters, and τ represents the soft update coefficients. The Actor network and the Critic network are updated synchronously. The Critic network minimizes the TD error ( By optimizing parameters, the Actor network minimizes the Actor loss term. The parameters are optimized by the Double DQN online network through the total loss function, and the three are updated collaboratively to obtain the networks that have been trained collaboratively.

[0063] In another possible implementation, the trained networks are used to infer the real-time status of the target patient at each follow-up stage to obtain short-term, medium-term, and long-term phased intervention decision recommendations for the target patient. This includes: using a policy generation network to infer the real-time status of the target patient at each follow-up stage and outputting a phased recommendation plan containing discrete intervention actions and continuous intervention parameters; performing uncertainty assessment and out-of-distribution detection on the inference output; when uncertainty exceeds the limit or out-of-distribution detection is triggered, outputting a conservative action set or triggering manual review, and recording the triggering reason, patient identifier, and triggering time; collecting intervention execution results and follow-up outcomes at each stage to supplement the three-stage trajectory sample; performing closed-loop incremental fine-tuning periodically based on a loss function with regularization penalty terms; and simultaneously updating the masking rules and boundary constraints of the hybrid clinical action space and the decision thresholds of the hierarchical composite reward function at each stage to obtain short-term, medium-term, and long-term phased intervention decision recommendations for the target patient.

[0064] Specifically, the clinical execution results of the model-recommended actions and the follow-up outcomes of patients at each stage are collected to form a new (s t ,a t ,r t ,s t+1 Trajectory samples are supplemented to the historical trajectory dataset; incremental fine-tuning is performed at preset intervals based on a loss function with regularized penalty terms, maintaining the phased configuration logic of each sub-parameter of the loss function during the fine-tuning process; the masking rules and boundary constraints of the hybrid clinical action space and the judgment thresholds of each stage of the hierarchical composite reward function are updated synchronously to adapt to updates in clinical guidelines or changes in patient population distribution. Through the above steps, phased intervention decision recommendations for target patients are finally output. The short-term (0-72 hours) includes the optimal discrete intervention plan and corresponding continuous parameter values; the medium-term (3-7 days) includes the optimal drug dosage optimization plan and complication management strategies; and the long-term (7-90 days) includes long-term renal protection strategies, chronic disease management target values, and outpatient follow-up intervals. These phased intervention decision recommendations are output in a structured form and can be directly integrated into the hospital information system to assist clinicians in developing targeted treatment plans at each follow-up node. In practical implementation, this method also introduces a global configuration set α to uniformly and adaptively configure the entire process of the method, supporting zero-code adaptation and deployment for multiple diseases and multiple centers. Operation and maintenance personnel only need to modify the corresponding parameters in the global configuration set α to complete the rapid adaptation of this application to different severe diseases, different hospital departments, or different clinical guideline versions, without requiring any changes to the core modules of the method. The global configuration set α includes the following types of configuration parameters: determining the weight vector and time series window length of each feature subset, configuring the normalization type and missing value handling strategy, and driving the phased state representation vector s. t The system dynamically adapts to different scenarios. Specifically, it determines the action mask rule list, continuous action constraint boundaries, and clinical safety regularization coefficients to ensure the contraindications of discrete intervention actions and the boundary safety of continuous intervention parameters. It configures the weighted coefficients of efficacy reward magnitude, basic reward values, and core clinical indicator judgment thresholds for each stage to achieve parameterized management of the hierarchical composite reward function for each stage. It configures the differential discount factor, upper limit of value estimation cutoff, value calibration coefficient, and soft update coefficient for the follow-up stage to uniformly manage the training hyperparameters of the four-network fusion architecture. It configures the uncertainty confidence threshold, out-of-distribution detection strategy, and manual review triggering conditions for each stage to ensure the clinical safety of the inference output.

[0065] In some embodiments of this application, a dynamic intervention decision support device for critical illness is also provided, such as... Figure 4 As shown, the device includes: The acquisition module 401 is used to collect multi-source clinical longitudinal data of patients with target severe diseases and perform compliance governance to build a patient-level longitudinal time series dataset. Alignment module 402 is used to perform time-series alignment of the longitudinal time-series dataset according to three follow-up stages—short-term, medium-term, and long-term—with the time of diagnosis or onset of illness as the starting point, and generate three-stage trajectory samples. The construction module 403 is used to construct a phased state representation vector and a hybrid clinical action space based on the three-stage trajectory samples, and to construct a four-network fusion architecture consisting of a policy generation network, a value assessment network, an online value estimation network and a target value estimation network based on the phased state representation vector and the hybrid clinical action space. The decision module 404 is used to construct a hierarchical composite reward function guided by the clinical goals of each follow-up stage. It uses a loss function with a regularized penalty term to collaboratively train the policy generation network, the value assessment network, and the online value estimation network, each with its own optimization objective. The target value estimation network periodically synchronizes parameters using a soft update method. The trained networks are then used to infer the real-time status of the target patient at each follow-up stage, obtaining short-term, medium-term, and long-term phased intervention decision recommendations for the target patient. Furthermore, the decision module 404 supports closed-loop incremental fine-tuning based on newly added trajectory samples, i.e., iteratively optimizing and updating the model after each round of follow-up outcome collection to continuously improve decision quality.

[0066] The device provided in this application embodiment includes an acquisition module that enables standardized and compliant governance of multi-source clinical data, an alignment module that completes three-stage temporal alignment and generates structured trajectory samples, overcoming the limitation of traditional methods in integrating longitudinal multi-stage data, a construction module that simultaneously constructs staged state representation vectors, a hybrid clinical action space, and a four-network fusion architecture, balancing the accurate output of continuous intervention parameters with the suppression of Q-value overestimation, and a decision module that drives the collaborative training of the three networks with a hierarchical composite reward function, combining uncertainty assessment, out-of-distribution detection, and closed-loop incremental fine-tuning to continuously output intervention decision suggestions that meet the treatment priorities of short-term, medium-term, and long-term stages, thereby maximizing the long-term clinical benefits for patients.

[0067] In some embodiments of this application, an electronic device is also provided to execute the above-described dynamic intervention decision support method for severe diseases. Please refer to... Figure 5 This illustrates a schematic diagram of an electronic device provided by some embodiments of this application. For example... Figure 5 As shown, the electronic device 9 includes: a processor 901, a memory 902, a bus 903, and a communication interface 904. The processor 901, the communication interface 904, and the memory 902 are connected via the bus 903. The memory 902 stores a computer program that can run on the processor 901. When the processor 901 runs the computer program, it executes the critical illness dynamic intervention decision support method provided in any of the foregoing embodiments of this application.

[0068] The memory 902 may include high-speed random access memory (RAM) or non-volatile memory, such as at least one disk storage device. Communication between this device network element and at least one other network element is achieved through at least one communication interface 904 (which can be wired or wireless), such as the Internet, wide area network, local area network, metropolitan area network, etc.

[0069] Bus 903 can be an ISA bus, PCI bus, or EISA bus, etc. Buses can be divided into address buses, data buses, control buses, etc. Memory 902 is used to store programs. After receiving execution instructions, processor 901 executes the programs. The dynamic intervention decision support method for critical illnesses disclosed in any of the aforementioned embodiments of this application can be applied to processor 901, or implemented by processor 901.

[0070] The processor 901 may be an integrated circuit chip with signal processing capabilities. In implementation, each step of the above method can be completed by the integrated logic circuitry in the hardware of the processor 901 or by instructions in software form. The processor 901 can be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it can also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), an off-the-shelf programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. It can implement or execute the methods, steps, and logic block diagrams disclosed in the embodiments of this application. The general-purpose processor can be a microprocessor or any conventional processor. The steps of the methods disclosed in the embodiments of this application can be directly embodied in the execution of a hardware decoding processor, or can be executed by a combination of hardware and software modules in the decoding processor. The software modules can be located in random access memory, flash memory, read-only memory, programmable read-only memory, electrically erasable programmable memory, registers, or other mature storage media in the art. The storage medium is located in memory 902. Processor 901 reads the information in memory 902 and, in conjunction with its hardware, completes the steps of the above method.

[0071] The electronic device provided in this application embodiment and the critical illness dynamic intervention decision support method provided in this application embodiment are based on the same inventive concept and have the same beneficial effects as the methods they adopt, operate or implement.

[0072] This application also provides a computer-readable storage medium corresponding to the critical illness dynamic intervention decision support method provided in the foregoing embodiments, which stores a computer program (i.e., a program product) thereon. When the computer program is run by a processor, it executes the critical illness dynamic intervention decision support method provided in any of the foregoing embodiments.

[0073] It should be noted that computer-readable storage media may include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory, optical disc or other optical and magnetic storage media, etc., which will not be elaborated here.

[0074] The computer-readable storage medium provided in this application embodiment and the critical illness dynamic intervention decision support method provided in this application embodiment are based on the same inventive concept and have the same beneficial effects as the methods adopted, run or implemented by the application stored therein.

[0075] This application also provides a computer program product corresponding to the critical illness dynamic intervention decision support method provided in the foregoing embodiments, including a computer program that is executed by a processor to implement the above-described critical illness dynamic intervention decision support method.

[0076] The computer program product provided in this application embodiment is based on the same inventive concept as the critical disease dynamic intervention decision support method provided in this application embodiment, and has the same beneficial effects as the method implemented by the computer program being executed by a processor.

[0077] It is understood that the description of the various embodiments above tends to emphasize the differences between the various embodiments, and the similarities or similarities can be referred to each other. For the sake of brevity, these will not be repeated here.

[0078] Those skilled in the art will understand that, in the methods described above in specific embodiments, the order in which the steps are written does not imply a strict execution order and does not constitute any limitation on the implementation process. The specific execution order of each step should be determined based on its function and possible internal logic.

[0079] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of this application, and not to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features therein. These modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of this application, and they should all be covered within the scope of the claims and specification of this application. In particular, as long as there is no structural conflict, the various technical features mentioned in the embodiments can be combined in any way. This application is not limited to the specific embodiments disclosed herein, but includes all technical solutions falling within the scope of the claims.

Claims

1. A dynamic intervention decision support method for critical illnesses, characterized in that, The method includes: Collect multi-source clinical longitudinal data of patients with target severe illnesses and conduct compliance governance to construct a patient-level longitudinal time series dataset; Using the time of diagnosis or onset of illness as the starting point, the longitudinal time series dataset is time-aligned according to three follow-up stages: short-term, medium-term, and long-term, to generate three-stage trajectory samples. Based on the three-stage trajectory samples, a phased state representation vector and a hybrid clinical action space are constructed. On the basis of the phased state representation vector and the hybrid clinical action space, a four-network fusion architecture consisting of a policy generation network, a value assessment network, an online value estimation network, and a target value estimation network is constructed. A hierarchical composite reward function is constructed based on the clinical goals of each follow-up stage. The strategy generation network, the value assessment network, the online value estimation network, and the target value estimation network are trained collaboratively with their respective optimization goals using a loss function with a regularized penalty term. The trained networks are then used to infer the real-time status of the target patient at each follow-up stage to obtain short-term, medium-term, and long-term phased intervention decision recommendations for the target patient. The construction of phased state representation vectors and a hybrid clinical action space based on the three-stage trajectory samples includes: Based on the three-stage trajectory samples, the weights and time window lengths of each feature subset are configured differently according to the short-term, medium-term and long-term follow-up objectives. The corresponding features are extracted from each configured feature subset, normalized and then concatenated to obtain the staged state representation vector. Based on the clinical constraint rule base, a set of discrete intervention actions matching the clinical goals of each follow-up stage is generated according to the diagnosis and treatment guidelines of each follow-up stage. The discrete intervention actions include at least one or more of the following: intervention category selection, protocol switching, monitoring intensity adjustment, and manual review triggering. Based on the clinical constraint rule base, an action mask matrix is ​​generated in real time according to the rule mapping relationship between each clinical contraindication condition and the intervention action to shield contraindicated interventions, and boundary constraints are set for the continuous intervention parameters bound to the discrete intervention action according to the action space configuration parameters. The discrete set of intervention actions and the corresponding continuous intervention parameters together constitute a hybrid clinical action space.

2. The method as described in claim 1, characterized in that, The process involves collecting multi-source clinical longitudinal data from patients with target severe illnesses and performing compliance governance to construct a patient-level longitudinal time-series dataset, including: Longitudinal clinical data from multiple sources were collected, including at least core laboratory indicators, vital signs, history of underlying diseases, surgical or interventional records, medication regimens, and follow-up outcome data. The multi-source clinical longitudinal data were sequentially subjected to de-identification, standardization, and quality control processing. Using patient identifiers as the primary key and timestamps as the index, a patient-level longitudinal time-series dataset with short-term, medium-term, and long-term stage annotations is constructed.

3. The method as described in claim 1, characterized in that, The process involves using the moment of diagnosis or onset of illness as the starting point, aligning the longitudinal time-series dataset according to three follow-up stages: short-term, medium-term, and long-term, to generate three-stage trajectory samples, including: The longitudinal time series dataset is divided into stages based on the time of diagnosis or onset of illness, according to short-term, medium-term and long-term follow-up time windows. The time windows for each follow-up stage are configurable parameters. Data from each patient's follow-up phases were aggregated and aligned. The aggregated data from each stage is organized into a quadruple containing the current patient status, clinical intervention actions, stage rewards, and the patient status for the next stage, thus obtaining the three-stage trajectory sample.

4. The method as described in claim 1, characterized in that, Based on the phased state representation vector and the hybrid clinical action space, a four-network fusion architecture is constructed, consisting of a policy generation network, a value assessment network, an online value estimation network, and a target value estimation network, including: A strategy generation network is constructed, taking the phased state representation vector as input, outputting the selection probability distribution of each discrete intervention action through the discrete action output branch, and outputting the continuous intervention parameter values ​​through the continuous parameter output branch and applying truncation constraints. A value assessment network is constructed, with the phased state representation vector and action pair as joint inputs. The state value function and the state action value function are fitted simultaneously. The advantage estimate formed by the difference between the two is used as the gradient signal to guide the update of the policy generation network parameters. The policy loss is minimized with the goal of maximizing the expected action value output of the value assessment network. The parameter update also introduces a clinical safety regularization term to impose constraints and prune invalid actions and out-of-bounds continuous parameters. Construct an online value estimation network and a target value estimation network with independent parameters. The online value estimation network estimates the value of all candidate actions in the current state and selects the optimal action. The target value estimation network receives the optimal action and performs independent value estimation. The state action value output of the value evaluation network is corrected by multiplying the decoupled estimation results of the two networks by a value calibration coefficient. A four-network fusion architecture is obtained, consisting of the policy generation network, the value assessment network, the online value estimation network, and the target value estimation network. The collaborative training of the four-network fusion architecture uses a differential discount factor in the follow-up phase to calculate the target value, and the discount factor is configured to increase in the short-term, medium-term, and long-term phases.

5. The method as described in claim 4, characterized in that, The step of correcting the state-action value output of the value assessment network by multiplying the decoupling estimation results of the two networks by a value calibration coefficient includes: The action corresponding to the highest value estimated by the online value estimation network for all candidate actions in the current state is taken as the benchmark action; The baseline action is input into the target value estimation network to obtain the target value; The difference between the target value and the current output of the value evaluation network is multiplied by the value calibration coefficient and then added to the output of the value evaluation network. The corrected state-action value is also used as the basis for calculating the advantage estimate, so as to synergistically improve the accuracy of intervention strategy selection and training stability.

6. The method as described in claim 1, characterized in that, The hierarchical composite reward function, constructed based on clinical goals at each follow-up stage, includes: With the goal of early reversal or control in the short term, the short-term efficacy reward component is calculated by weighting the positive improvement of the baseline reward value and core clinical indicators relative to the short-term stage judgment threshold, and deducting the abnormal increase in the amount of penalty. The short-term safety risk penalty component and resource cost penalty component are also integrated. With the goal of stabilizing the condition and preventing its deterioration in the medium term, the mid-term efficacy reward component is calculated using the same magnitude weighting calculation logic as the short term, and the mid-term safety risk penalty component is also incorporated. With the goal of preventing adverse outcomes in the long term, the long-term efficacy reward component is calculated using the same magnitude weighting calculation logic as the short-term component, and the long-term safety risk penalty component is also incorporated. The threshold values, magnitude weighting coefficients, and reward / penalty weights for each stage are uniformly managed by the reward function configuration parameters, resulting in a hierarchical composite reward function.

7. The method as described in claim 1, characterized in that, The loss function with regularization penalty includes at least a temporal difference mean squared error term, a sparsity regularization term, a weight decay regularization term, a value estimation upper limit penalty term, and a policy generation network loss weighting term; the step of co-training the policy generation network, the value evaluation network, and the online value estimation network with the loss function with regularization penalty under their respective optimization objectives includes: The value assessment network updates its parameters with the goal of minimizing the mean square error of the time-series difference. The policy generation network updates its parameters with the optimization objective of minimizing the loss weighting term of the policy generation network. The loss weighting term of the policy generation network is optimized with the expected value of the state-action output by the value evaluation network, and its contribution ratio in the total loss is controlled by the weighting coefficient. The online value estimation network updates its parameters with the optimization objective of minimizing the total loss function containing all five terms. The value estimation upper limit penalty term applies a penalty when the output of the online value estimation network exceeds a preset value upper limit. The target value estimation network periodically synchronizes its parameters from the online value estimation network using a soft update method. The soft update coefficient is a configurable parameter, resulting in the networks that have completed the collaborative training.

8. The method as described in claim 1, characterized in that, The trained networks are used to infer the real-time status of the target patient at each follow-up stage, obtaining short-term, medium-term, and long-term phased intervention decision recommendations for the target patient, including: The strategy is used to generate a network to infer the real-time status of the target patient at each follow-up stage and output a phased recommendation plan that includes discrete intervention actions and continuous intervention parameters. Uncertainty assessment and out-of-distribution detection are performed on the output of the inference. When the uncertainty exceeds the limit or the out-of-distribution detection is triggered, a conservative action set is output or manual review is triggered, and the triggering reason, patient identification and triggering time are recorded. The results of intervention execution and follow-up outcomes at each stage are collected and added to the three-stage trajectory sample. Closed-loop incremental fine-tuning is performed periodically based on the loss function with regularized penalty term. The masking rules and boundary constraints of the hybrid clinical action space and the judgment thresholds of each stage of the hierarchical composite reward function are updated simultaneously to obtain short-term, medium-term and long-term phased intervention decision suggestions for target patients.

9. A dynamic intervention decision support device for critical illnesses, characterized in that, The device includes: The data acquisition module is used to collect multi-source clinical longitudinal data of patients with target severe diseases and to perform compliance governance to build a patient-level longitudinal time series dataset. The alignment module is used to perform time-series alignment of the longitudinal time-series dataset according to three follow-up stages: short-term, medium-term, and long-term, with the time of diagnosis or onset of illness as the starting point, and generate three-stage trajectory samples. The construction module is used to construct a phased state representation vector and a hybrid clinical action space based on the three-stage trajectory samples, and to construct a four-network fusion architecture consisting of a policy generation network, a value assessment network, an online value estimation network and a target value estimation network based on the phased state representation vector and the hybrid clinical action space. The decision-making module is used to construct a hierarchical composite reward function guided by the clinical goals of each follow-up stage. The strategy generation network, the value assessment network, the online value estimation network, and the target value estimation network are trained collaboratively with their respective optimization goals using a loss function with a regularized penalty term. The trained networks are used to infer the real-time status of the target patient at each follow-up stage to obtain short-term, medium-term, and long-term phased intervention decision suggestions for the target patient. The construction of phased state representation vectors and a hybrid clinical action space based on the three-stage trajectory samples includes: Based on the three-stage trajectory samples, the weights and time window lengths of each feature subset are configured differently according to the short-term, medium-term and long-term follow-up objectives. The corresponding features are extracted from each configured feature subset, normalized and then concatenated to obtain the staged state representation vector. Based on the clinical constraint rule base, a set of discrete intervention actions matching the clinical goals of each follow-up stage is generated according to the diagnosis and treatment guidelines of each follow-up stage. The discrete intervention actions include at least one or more of the following: intervention category selection, protocol switching, monitoring intensity adjustment, and manual review triggering. Based on the clinical constraint rule base, an action mask matrix is ​​generated in real time according to the rule mapping relationship between each clinical contraindication condition and the intervention action to shield contraindicated interventions, and boundary constraints are set for the continuous intervention parameters bound to the discrete intervention action according to the action space configuration parameters. The discrete set of intervention actions and the corresponding continuous intervention parameters together constitute a hybrid clinical action space.

10. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, The processor runs the computer program to implement the dynamic intervention decision support method for critical illnesses as described in any one of claims 1-8.

11. A computer-readable storage medium having a computer program stored thereon, characterized in that, The computer program is executed by a processor to implement the dynamic intervention decision support method for critical illnesses as described in any one of claims 1-8.

Images