A hospital operation cost dynamic optimization method and system based on reinforcement learning

Abstract

The application relates to a hospital operation cost dynamic optimization method and system based on reinforcement learning, which comprises the following steps: collecting Internet of Things sensor energy consumption, equipment state, patient flow, management database and external public health alarm data, generating a state vector through dimension reduction and standardization; analyzing alarm data to extract real-time event type identification. Based on historical state vectors, a reinforcement learning model is trained, the weight coefficient is dynamically adjusted according to the event identification, and an attention mechanism is integrated to generate a key variable weight distribution. The current state vector is input to output an optimized action, which is sent to a hospital building management system after being verified by an ethical constraint rule; the actual reward is calculated by collecting the executed data, the deviation between the predicted reward and the actual reward is compared, the model is updated when the deviation exceeds a threshold, and the cost saving rate, the resource utilization rate and the patient satisfaction are cooperatively optimized. The application realizes multi-objective cooperative optimization of hospital operation cost under the premise of safety and controllability.

Description

Technical Field

[0001] This application relates to the fields of smart healthcare and lean hospital management technology, and in particular to a method and system for dynamic optimization of hospital operating costs based on reinforcement learning. Background Technology

[0002] With the deepening of smart hospital construction, IoT sensing devices, hospital information systems, and building automation systems have been widely applied in hospitals, accumulating massive amounts of real-time energy consumption data, equipment operation logs, patient flow information, and human resource data. This data makes it possible for hospital operations to shift from a traditional experience-driven model to a data-driven model. At the same time, artificial intelligence technologies such as reinforcement learning have shown great potential in handling sequential decision-making problems, providing new tools for automatically finding optimal strategies in complex and dynamic environments. In practice, some explorations have attempted to use historical data to build predictive models or set fixed rules to automate the control of subsystems such as air conditioning and lighting, aiming to save energy and improve the efficiency of resource utilization.

[0003] However, existing hospital operation and management methods often focus on static optimization of single or a few objectives. When faced with highly complex, multi-objective conflicting, and uncertain real-world environments, they lack a comprehensive framework that can systematically integrate external event perception and internal state analysis. This makes it difficult for the system to adapt to the impact of emergencies such as infectious disease outbreaks or extreme weather, and it cannot dynamically adjust the priority of optimization objectives. Furthermore, due to the extremely high requirements for safety and interpretability in the medical setting, purely data-driven, black-box decision-making is difficult to gain the trust of clinical management personnel and carries the risk of violating medical safety guidelines. In addition, existing operating systems cannot evolve with changes in hospital business models, making it difficult to achieve long-term, sustainable, and lean operations. Summary of the Invention

[0004] To address the aforementioned technical issues, this application provides a method and system for dynamic optimization of hospital operating costs based on reinforcement learning.

[0005] Firstly, this application provides a method for dynamically optimizing hospital operating costs based on reinforcement learning, employing the following technical solution: Collect real-time energy consumption data and equipment status data from IoT sensors, patient flow data, management database data, and external public health event alarm data. Preprocess the collected data and generate standardized state vectors through feature dimensionality reduction. Extract real-time event type identifiers from the external public health event alert data; A reinforcement learning model is trained based on a historical state vector dataset. A reward function is designed using a proximal policy optimization algorithm. The weight coefficients of the reward function are dynamically adjusted according to the real-time event type identifier. An attention mechanism is integrated to generate the weight distribution of key variables. The standardized state vector at the current moment is input into the trained reinforcement learning model, which outputs an optimized decision action. At the same time, an interpretability report of the decision is generated based on the weight distribution of the key variables, and the pre-configured set of ethical constraint rules is called to verify the action. The optimized decision action, which has passed the action verification, is sent to the hospital building management system through the application programming interface. After the action is executed, the equipment status update data and patient flow change data are collected. The actual status and actual reward value are calculated based on the reward function. The deviation between the predicted reward value output by the reinforcement learning model and the actual reward value is compared. When the deviation exceeds a set threshold, a parameter update instruction for the reinforcement learning model is triggered. Based on the actual state, the synergistic optimization results of cost saving rate, resource utilization rate and patient satisfaction index are evaluated to obtain the degree of achievement of the optimization goal.

[0006] By adopting the above technical solutions, deeply integrating multi-source IoT data, extracting real-time event type identifiers, and transforming external environmental shocks into decision-making context labels, the weights of the reinforcement learning reward function are dynamically adjusted based on these identifiers. This enables dynamic switching of multi-objective priorities according to the context, solving the problem of strategy adaptation under emergencies. By calling pre-configured ethical constraint rule sets to verify actions, the system ensures that decisions are transparent and comply with medical safety bottom lines. By collecting actual data after execution, the system continuously adapts to changes in hospital operations, ultimately achieving multi-objective collaborative optimization of operating costs in a dynamic environment. This improves resource utilization efficiency and decision credibility, supporting long-term lean operations.

[0007] Secondly, this application provides a hospital operation cost dynamic optimization system based on reinforcement learning, which adopts the following technical solution: The multi-source sensing and state modeling module is used to collect real-time energy consumption data and equipment status data from IoT sensors, patient flow data, management database data, and external public health event alarm data. It preprocesses the collected data and generates standardized state vectors through feature dimensionality reduction. The event type parsing module is used to extract real-time event type identifiers based on the external public health event alert data; The adaptive policy training module is used to train a reinforcement learning model based on a historical state vector dataset. It designs a reward function using a proximal policy optimization algorithm, dynamically adjusts the weight coefficients of the reward function according to the real-time event type identifier, and integrates an attention mechanism to generate the weight distribution of key variables. The intelligent decision-making and verification module is used to input the standardized state vector at the current moment into the trained reinforcement learning model, output optimized decision actions, generate a decision interpretability report based on the weight distribution of the key variables, and call the pre-configured set of ethical constraint rules to verify the actions. The execution feedback and evaluation module is used to send the optimized decision action that has passed the action verification to the hospital building management system through the application programming interface, collect equipment status update data and patient flow change data after the action is executed, and calculate the actual status and actual reward value based on the reward function. The model iteration and collaborative optimization module is used to compare the deviation between the predicted reward value output by the reinforcement learning model and the actual reward value. When the deviation exceeds a set threshold, a reinforcement learning model parameter update instruction is triggered, and the collaborative optimization results of cost saving rate, resource utilization rate and patient satisfaction index are evaluated based on the actual state.

[0008] Thirdly, this application provides a computer-readable storage medium, which adopts the following technical solution: A computer-readable storage medium storing a computer program that can be loaded by a processor and executed as in any of the methods in the first aspect.

[0009] In summary, this application achieves at least one of the following beneficial technical effects: It enables dynamic optimization of hospital operating costs, adjusts strategies under different external event scenarios, balances multiple objectives, and the model self-corrects through real-time feedback to adapt to business changes. Compared to traditional static rules and manual experience, it improves resource utilization efficiency and decision-making transparency, safeguards the bottom line of medical safety, and provides hospitals with a data-driven, sustainable operating cost optimization method. Attached Figure Description

[0010] Figure 1 This is a schematic diagram of the first process of a method for dynamic optimization of hospital operating costs based on reinforcement learning, which is one embodiment of this application.

[0011] Figure 2 This is a schematic diagram of the second process of a reinforcement learning-based dynamic optimization method for hospital operating costs, which is one embodiment of this application.

[0012] Figure 3 This is a schematic diagram of the third process of a reinforcement learning-based dynamic optimization method for hospital operating costs, which is one embodiment of this application.

[0013] Figure 4 This is a schematic diagram of the fourth process of a reinforcement learning-based dynamic optimization method for hospital operating costs, which is one embodiment of this application.

[0014] Figure 5This is a schematic diagram of the fifth process of a reinforcement learning-based dynamic optimization method for hospital operating costs, which is one embodiment of this application.

[0015] Figure 6 This is a schematic diagram of the sixth process of a reinforcement learning-based dynamic optimization method for hospital operating costs, which is one embodiment of this application.

[0016] Figure 7 This is a schematic diagram of the seventh process of a reinforcement learning-based dynamic optimization method for hospital operating costs, which is one embodiment of this application.

[0017] Figure 8 This is a schematic diagram of the eighth process of a reinforcement learning-based dynamic optimization method for hospital operating costs, according to one embodiment of this application. Detailed Implementation

[0018] To make the purpose, technical solution, and advantages of this application clearer, the following description is provided in conjunction with the appendix. Figures 1-8 The present application will be further described in detail below with reference to embodiments. It should be understood that the specific embodiments described herein are for illustrative purposes only and are not intended to limit the scope of the application.

[0019] This application discloses a method for dynamic optimization of hospital operating costs based on reinforcement learning.

[0020] Reference Figure 1 A method for dynamic optimization of hospital operating costs based on reinforcement learning, specifically including: Step S101: Collect real-time energy consumption data and equipment status data, patient flow data, management database data and external public health event alarm data from IoT sensors; preprocess the collected data and generate standardized state vectors through feature dimensionality reduction. Specifically, collecting real-time data from IoT sensors (such as power, water consumption, and temperature sensors) captures the "physiological signs" of hospital infrastructure; obtaining patient flow data from the electronic medical record system (such as registration, waiting, and admission numbers) reflects the "pulse of demand" for medical services; retrieving manpower scheduling and inventory records from the management database provides an understanding of the "resource base" of operations; and introducing external public health event alert data (such as infectious disease warnings and severe weather notifications) injects a "forward-looking vision" of environmental risk perception into the system.

[0021] It should be noted that these heterogeneous, high-dimensional raw data inevitably contain noise (such as sensor instantaneous errors) and missing values. Therefore, preprocessing is necessary, including using filtering algorithms to smooth noise and performing reasonable missing value imputation based on time series or related variables to ensure data quality.

[0022] Subsequently, feature dimensionality reduction techniques (such as principal component analysis, PCA) are employed to reconstruct a set of orthogonal, variance-sorted new features (principal components) through linear transformation. Principal components whose cumulative variance contribution rate reaches a set threshold (e.g., 85%) are retained. This significantly compresses data dimensionality and eliminates redundancy while preserving the original information to the maximum extent.

[0023] Finally, the features of each dimension are standardized (such as Z-score standardization) to eliminate the differences in dimensions and map all features to the same scale. The resulting standardized state vector is a mathematical object that can represent the complex operation of the hospital in real time and concisely, providing a unified observation window for the reinforcement learning model.

[0024] Step S102: Extract real-time event type identifiers based on external public health event alert data; The logic behind this step is to transform discrete, sudden events in the external environment into identifiable and manageable decision-making context labels within the system.

[0025] In this embodiment of the application, external public health event alert data typically contains structured event descriptions and classification codes. This step parses these codes and maps them to a pre-defined, finite set of event type enumeration values ​​(e.g., local infectious disease warnings, regional extreme weather, large-scale traffic accidents, etc.).

[0026] Understandably, this real-time event type identifier is not simply a data field, but a crucial context switch. Its existence signifies that the system acknowledges that hospital operational optimization is not in a static or stable environment; different external shocks can fundamentally alter the relationships and importance ranking of various optimization variables within the system (such as patient demographics and resource scarcity). For example, during an infectious disease outbreak, infection control-related energy consumption (such as enhanced ventilation) may take precedence over routine energy conservation goals. Therefore, this identifier provides a contextual basis for the subsequent dynamic adjustment of the reward function, enabling the system to evolve from a "one-size-fits-all" optimization model to possessing preliminary intelligence capable of "contextualized decision-making."

[0027] Step S103: Train a reinforcement learning model based on the historical state vector dataset, design a reward function using a proximal policy optimization algorithm, dynamically adjust the weight coefficients of the reward function according to the real-time event type identifier, and integrate an attention mechanism to generate the weight distribution of key variables. Specifically, the training data is a standardized sequence of state vectors from historical periods, recording a complete closed loop of states, decisions, state transitions, and outcomes. The model employs the Proximal Policy Optimization (PPO) algorithm, whose core principle is to introduce a probability ratio pruning mechanism when updating the decision policy to constrain the difference between the old and new policies, thereby ensuring the stability of the training process and avoiding policy performance collapse due to excessively large single updates.

[0028] In this embodiment, the reward function is not fixed, but dynamically adjusts the weight coefficients of its internal components (such as cost savings, patient waiting time, and equipment safety redundancy) based on the extracted real-time event type identifier. For example, when the event is identified as "emergency peak," the algorithm automatically reduces the penalty weight of the patient waiting time item (i.e., allows for a temporary increase in cost to shorten the waiting time), while when it is identified as "abnormal energy consumption," the reward weight of the cost savings item is increased. This dynamism allows the optimization objective to switch flexibly according to the external environment.

[0029] Furthermore, the model integrates an attention mechanism, which allows the model to automatically calculate and focus on the feature dimensions most critical to the current decision when processing the current state vector (for example, during low-traffic periods at midnight, the energy consumption features of lighting and air conditioning may receive high weights; while during daytime surgical periods, the availability and energy consumption features of operating room equipment have higher weights). The resulting key variable weight distribution not only improves the model's decision-making accuracy under complex conditions, but more importantly, provides direct evidence for the subsequent generation of interpretable reports.

[0030] Step S104: Input the standardized state vector at the current moment into the trained reinforcement learning model, output the optimized decision action, generate a decision interpretability report based on the weight distribution of key variables, and call the pre-configured set of ethical constraint rules to verify the action. The system inputs the standardized state vector of the current moment into the trained model. Based on its learned strategy, the model outputs a specific optimization decision, such as "increasing the nighttime air conditioning temperature setting in Building A by 1 degree" or "temporarily relocating some non-emergency testing personnel to the emergency room window." Simultaneously, the system automatically generates a decision interpretability report using the key variable weight distribution generated by the attention mechanism during training. This report clearly identifies which key indicators of the current state (such as "extremely low ICU bed vacancy rate" and "instantaneous electricity price reaching peak") dominated the decision, enabling managers and medical staff to understand and trust the model's judgment. This is a prerequisite for the application of artificial intelligence in high-reliability fields such as healthcare.

[0031] Subsequently, the system will call the pre-configured set of ethical constraint rules to verify the action. The preset ethical constraint conditions for verification include: a pre-loaded list of prohibited actions, which includes energy consumption control instructions that would cause power outages in the intensive care unit and manpower scheduling plans that would prolong the waiting time for high-risk patients. If the output action matches the list of prohibited actions, a manual review process will be triggered.

[0032] Understandably, these rules are typically based on rigid logical conditions pre-set by medical safety regulations, industry standards, and hospital management red lines, such as "prohibiting any energy-saving instructions that may cause power outages in the intensive care unit" or "prohibiting any scheduling schemes that would prolong the waiting time for specific high-risk patient groups." If an action triggers any ethical constraints, it will be intercepted and a manual review process will be initiated. This step ensures that the optimization process is always conducted within a framework that complies with medical ethics and safety, achieving a balance between efficiency and safety.

[0033] Step S105: Send the optimized decision action that has passed the action verification to the hospital building management system through the application programming interface, collect the equipment status update data and patient flow change data after the action is executed, and calculate the actual status and actual reward value based on the reward function. The optimized decision actions that pass verification will be sent to execution terminals such as the hospital's building management system and human resources system via an application programming interface (API). The API, as a standardized software intermediary, ensures reliable and efficient exchange of instructions and data between this optimization system and the hospital's various existing heterogeneous operating systems. After the action is executed, the system will again collect equipment status update data (such as adjusted actual energy consumption) and patient flow change data (such as changes in queue length after scheduling). This data constitutes a new, actual state reflecting the result of the action.

[0034] Subsequently, the system calculates an actual reward value for this new real-world state based on a reward function designed during the training of the reinforcement learning model. This actual reward value represents the quantifiable, immediate benefit (or cost) of the decision in the real world. It contrasts with the reward value predicted by the model when making the decision and serves as the gold standard for evaluating the accuracy of model predictions and the effectiveness of decision-making.

[0035] Step S106: Compare the deviation between the predicted reward value output by the reinforcement learning model and the actual reward value. When the deviation exceeds the set threshold, trigger the reinforcement learning model parameter update instruction. Based on the actual state, evaluate the synergistic optimization results of cost saving rate, resource utilization rate and patient satisfaction index to obtain the degree of achievement of the optimization goal.

[0036] The system compares the predicted reward value output by the reinforcement learning model with the actual reward value. This deviation directly reflects the model's prediction error in responding to dynamic responses in the real world. The threshold is usually dynamically calculated based on the model's performance on historical training sets (e.g., a multiple of the standard deviation of the prediction error during training) to ensure the necessity and stability of updates.

[0037] In this embodiment, when the deviation consistently exceeds a threshold, it indicates that the model's internal strategy (i.e., its understanding of the state-action-reward relationship) has become significantly out of sync with the evolving reality. At this point, the system automatically triggers a model parameter update command. The update can employ an incremental training mode, adjusting the parameters of the fully connected layers of the policy network while retaining the ability to extract core features such as the attention mechanism. This allows the model to quickly adapt to new operating models without having to start training from scratch. Finally, based on the actual state after the action is executed, the system comprehensively evaluates the synergistic optimization results of the three objectives: cost savings, resource utilization, and patient satisfaction index.

[0038] Understandably, hospital operation optimization is a typical multi-objective game problem. The extreme pursuit of a single objective (such as reckless cost reduction) can harm other objectives (such as patient satisfaction). The system comprehensively measures the overall value of a decision by calculating the distance between the actual state point and the ideal optimal solution (Pareto front) in a multi-dimensional space centered on these three indicators. This ensures that the system's optimization orientation seeks the best balance among multiple key performance indicators, thereby achieving sustainable, human-centered, and lean operations.

[0039] The above implementation deeply integrates multi-source IoT data, extracts real-time event type identifiers, transforms external environmental shocks into decision-making context labels, and dynamically adjusts the weights of the reinforcement learning reward function based on these identifiers. This enables dynamic switching of multi-objective priorities according to the context, solving the problem of strategy adaptation under emergencies. By calling pre-configured ethical constraint rule sets to verify actions, it ensures that decisions are transparent and comply with medical safety bottom lines. By collecting actual data after execution, the system continuously adapts to changes in hospital operations, ultimately achieving multi-objective collaborative optimization of operating costs in a dynamic environment, improving resource utilization efficiency and decision credibility, and supporting long-term lean operations.

[0040] Reference Figure 2 As one implementation of step S102, the step of extracting real-time event type identifiers based on external public health event alert data includes: Step S201: Receive external public health event alert data streams, including WHO notifications, CDC early warning reports, and hospital internal epidemic monitoring records; Hospital operations are not isolated systems; their costs and resource pressures are heavily influenced by the external public health environment. Single data sources (such as internal hospital records alone) have blind spots and delays. Therefore, this method systematically receives three types of alert data streams: WHO announcements provide global or cross-regional warnings of major outbreaks, possessing the highest authority and foresight; CDC alert reports reflect national or regional epidemiological dynamics and are crucial information directly guiding local prevention and control strategies; and internal hospital epidemic surveillance records (such as nosocomial infection reports and outpatient / emergency symptom monitoring) provide the most direct and micro-level risk signals.

[0041] By aggregating these three data streams of different granularities and from different sources, the principle lies in achieving a combination of "long-term, medium-term, and short-term" risk perception. For example, the combined effect of WHO announcements regarding the emergence of a new virus in a neighboring country (long-term warning), local CDC alerts on rising influenza indices (medium-term warning), and records of a sudden increase in fever patients in the hospital's emergency department (short-term alert) can enable the system to identify the "local seasonal influenza active period" event type earlier and more accurately, thus gaining valuable time for resource pre-emptive adjustments.

[0042] Step S202: Perform word segmentation and entity recognition on the unstructured text alarm data in the external public health event alarm data stream, and extract the event type keyword set and geographical impact range parameter; The received data streams, especially WHO announcements and disease control reports, are primarily unstructured text (such as press releases and alerts). Computers cannot directly understand the meaning of this text and must use natural language processing techniques for parsing.

[0043] Specifically, word segmentation is based on the principle of dividing a continuous text sequence into a sequence of words (lexical units) with independent meaning. This is the foundation of language information processing, such as Chinese. For example, "a norovirus cluster outbreak occurred in this city" can be broken down into "this city / occurred / norovirus / cluster / outbreak".

[0044] Subsequently, a medical named entity recognition model was employed. This model, pre-trained on a large amount of medical literature and medical records, accurately identified medically relevant entities with specific meanings within the text. Focusing on three types of entities—disease name, transmission route, and hazard level—it demonstrates strong business relevance: identifying "norovirus" (disease name) and "contact transmission" (transmission route) clarifies the nature of the event; identifying "high risk" (hazard level) quantifies the urgency of the event. Simultaneously, geographical impact parameters (e.g., parsing "City A, District B") are extracted from the text. This is achieved through address resolution and geocoding techniques, transforming geographical location descriptions in the text into calculable spatial parameters.

[0045] Step S203: Input the set of event type keywords into the pre-trained word vector model to generate event semantic feature vectors; Although the event type keyword set is structured, the words are isolated and cannot reflect that the semantic correlation between "influenza" and "avian influenza" is higher than that between "influenza" and "power outage".

[0046] Therefore, a pre-trained word vector model (such as the BioWordVec biomedical word embedding model) can be used to output a 200-dimensional event semantic feature vector. This is based on the "distributed hypothesis," meaning that semantically similar words appear in similar ways within their contexts. The model is trained unsupervised on massive amounts of biomedical text (such as PubMed document abstracts), mapping each word to a fixed vector in a high-dimensional space (e.g., 200-dimensional), making semantically similar words (e.g., "influenza" and "viral cold") closer together in this vector space. All words from the event type keyword set are input into the model and aggregated through pooling and other methods to ultimately generate an event semantic feature vector that comprehensively represents the overall semantics of the alert.

[0047] Understandably, this vector not only contains information about the words themselves, but also implies complex medical semantic relationships between them. By using this type of domain-specific model rather than a general model, we can ensure that the vector representations of medical terms such as "incubation period" and "host" are more accurate, thereby improving the accuracy of subsequent event type identification.

[0048] Step S204: Calculate the spatial correlation matrix of the event based on the geographical influence range parameter, and construct a multi-dimensional event feature vector by combining the event semantic feature vector; The impact of a public health event on a specific hospital depends not only on the nature (semantics) of the event itself, but also heavily on its spatial geographical relationship with the hospital. Therefore, it is necessary to integrate semantic and spatial information to form a comprehensive judgment criterion.

[0049] In this embodiment, the geographical impact range parameter (such as the coordinates of the epidemic area center and the radius of influence) is the input. The core principle for calculating the spatial correlation matrix (or correlation value) is to quantify the "distance decay effect" and "population mobility risk". The calculation formula is: Correlation = 1 / (1 + straight-line distance between the hospital and the epidemic area) × regional population mobility coefficient, where the regional population mobility coefficient comes from publicly available datasets from the transportation department. Specifically, the straight-line distance between the hospital and the epidemic area is the basis; the closer the distance, the greater the risk of impact, usually increasing non-linearly. By introducing the regional population mobility coefficient, this reflects that even if the geographical distance is slightly far, if there is frequent movement of people between the two places (such as commuter arteries and transportation hubs), the risk is still very high. Through the above calculation formula that integrates distance and mobility, a quantified spatial correlation value is finally obtained.

[0050] Subsequently, this spatial correlation value is combined with the event semantic feature vector (for example, by adding correlation as a new dimension or by weighting it with the semantic vector) to construct a multidimensional event feature vector. This new vector simultaneously encodes both "what happened" and "how close this event is to me and how easily it can be transmitted," thus upgrading the representation of the event from a purely semantic level to a risk context level closely related to the hospital.

[0051] Step S205: Query the preset hospital event type mapping rule library, match the multi-dimensional event feature vector to the preset event type set, and output the real-time event type identifier.

[0052] In some embodiments, the preset hospital event type mapping rule base is built based on medical event classification standards (such as ICD-11) and predefines three basic event types such as "infectious disease outbreak", "medical device shortage" and "medical personnel shortage". Each event type has one or more baseline feature vectors in the base as its prototype.

[0053] Specifically, the matching process can employ a cosine similarity algorithm. This algorithm calculates the cosine of the angle between the multidimensional feature vector of the current event and the baseline vector of each event type in the database. The closer the cosine value is to 1, the more consistent the directions of the two vectors are, indicating greater similarity in semantic and spatial risk features. When the similarity exceeds a validated threshold (e.g., 0.85), the match is considered successful, and the event type identifier (e.g., "Type_A: Local infectious disease community transmission") is output.

[0054] In the above implementation, the similarity judgment based on comprehensive feature vectors that integrate deep semantics and spatial risk calculation greatly improves the accuracy and anti-interference ability of classification, and solves the problems of misjudgment, missed judgment, and delay caused by traditional methods that rely on simple keyword matching or rules. The final generated real-time event type identifier provides a high-quality and highly reliable environmental context signal for the reinforcement learning decision-making system, enabling subsequent cost dynamic optimization strategies (such as adjusting energy consumption, manpower, and inventory) to make truly context-aware intelligent responses. Targeted contingency plans can be activated in the early stages of an emergency or its spread, improving the accuracy of resource allocation and the scientific nature of cost control while ensuring medical safety and service quality.

[0055] Reference Figure 3 As one implementation of step S103, the steps of training a reinforcement learning model based on a historical state vector dataset, designing a reward function using a proximal policy optimization algorithm, dynamically adjusting the weight coefficients of the reward function according to the real-time event type identifier, and integrating an attention mechanism to generate the weight distribution of key variables include: Step S301: Obtain the standardized state vector dataset for historical time periods and the event type identifier for the corresponding time. Specifically, the standardized state vector dataset for historical time periods records a complete snapshot of the hospital's operations at various points in the past. Each vector encapsulates comprehensive quantitative information such as energy consumption, equipment status, patient flow, manpower, and inventory at that moment, forming the complete set of environmental states that the model needs to learn. Meanwhile, the event type labels that strictly correspond to the time (such as "infectious disease outbreak," "routine operation," "extreme weather," etc.) annotate the specific external context in which each historical experience occurred.

[0056] Understandably, this data organization method explicitly acknowledges that optimal hospital operating strategies are not static but highly dependent on the external environment. The goal of model training is not only to learn optimization patterns under normal conditions but also to identify how to dynamically adjust decision priorities and focuses under different types of external shocks. For example, the model needs to learn from the data those decision patterns that prioritized ventilation system operation and achieved good long-term returns even with increased energy consumption during historical periods labeled as "infectious disease outbreaks."

[0057] Step S302: Initialize the network parameters and basic weight coefficients of the reward function of the reinforcement learning model; Initializing network parameters involves assigning a set of random or policy-specific (such as Xavier initialization) initial values ​​to the subsequently built policy and value networks. This is equivalent to providing the model with an initial, untrained "brain" connectivity state, upon which all subsequent learning will be performed by adjusting these parameters.

[0058] Meanwhile, the initial reward function's base weight coefficients define the default importance ratios of each sub-objective (such as cost savings, resource utilization, and patient satisfaction) under "normal conditions" in a multi-objective optimization problem. These base weight coefficients serve as a benchmark for subsequent dynamic adjustments, providing an initial value orientation that aligns with managers' general understanding. This ensures that the model explores within a value framework consistent with basic business logic from the start of training, rather than through completely random exploration. This significantly improves efficiency and stability in the early stages of training.

[0059] Step S303: Construct a policy network and a value network based on network parameters and a near-end policy optimization algorithm. Input the standardized state vectors of historical periods into the policy network and the value network to generate action probability distributions and state value estimates, respectively. Among them, the Proximal Policy Optimization (PPO) algorithm is a state-of-the-art policy gradient-based reinforcement learning algorithm. By introducing a trust region constraint, it ensures that the policy does not deviate too far from the old policy in each update iteration. This effectively solves the problem of drastic fluctuations or even crashes in policy gradient methods due to excessively large single updates, making the training process smoother and more reliable.

[0060] Within this framework, two neural networks with distinct functions are constructed: a policy network and a value network. The policy network receives the current state vector and event identifier, and after internal computation, outputs an action probability distribution—the probability of selecting each action from all possible optimal actions (such as adjusting the air conditioner temperature or rescheduling shifts). This represents the model's tendency in the current situation. The value network evaluates the same input state, outputting a state value estimate—predicting the expected value of the future cumulative reward obtained from the current state by following the current policy. The policy network and value network work collaboratively; one explores and selects actions, while the other evaluates and guides learning, together forming the dual engines that enable the PPO algorithm to learn stably and efficiently.

[0061] Step S304: Query the preset weight adjustment rule base according to the event type identifier, and dynamically update the weight coefficient of the reward function; The pre-defined weight adjustment rule base is a knowledge base that maps specific event types to specific reward function weight adjustment strategies. Its principle is based on domain expert knowledge or historical data analysis, clearly defining how the hospital's operational priorities should change when a specific event occurs. For example, the rule base might define: when the event is identified as "infectious disease outbreak," automatically increase the weight coefficient of reward items related to "patient satisfaction" or "hospital infection control," while appropriately decreasing the weight of "energy cost savings"; when identified as "equipment shortage," increase the weight of items related to "resource utilization."

[0062] During model training, the system queries the rule base in real time based on the event identifier attached to each training sample and dynamically updates the weight coefficients of the reward function accordingly. That is, for the same state (such as a crowded waiting area), the optimization strategy learned by the model under the "normal operation" event (which may focus on energy-saving scheduling) will be very different from the strategy learned under the "emergency peak" event (which will focus on rapid diversion and reducing waiting time) due to the different weights of the reward function. This enables the model to learn complex strategies of "strategic application" rather than a rigid single strategy.

[0063] Step S305: Configure attention mechanism layers in the policy network and value network, calculate attention scores for each feature dimension of the standardized state vector for historical periods through the attention mechanism layers, and generate the weight distribution of key variables. In this embodiment, the attention mechanism layer adopts a multi-head self-attention structure: by dividing the state vector into multiple subspaces, calculating the attention score matrix of each subspace in parallel, and then concatenating them to generate the key variable weight distribution through a fully connected layer.

[0064] In a hospital, the state vector typically contains dozens or even hundreds of feature dimensions, but at a specific decision moment, not all features are equally important. The attention mechanism layer is integrated within the policy network or value network. When processing the current state vector, the model autonomously calculates the importance score of each feature dimension for making the current decision (or evaluating the current state), i.e., the attention score.

[0065] For example, when deciding whether to activate a backup generator, features such as "current grid electricity price," "ICU power supply status," and "estimated surgery duration" will have very high attention scores, while "outpatient pharmacy inventory" may have a low score. By calculating the attention scores of all features, the model generates a key variable weight distribution. This mechanism not only technically helps the model filter noise and focus on the most relevant information, improving learning efficiency and decision quality, but also clearly reveals the basis of the model's decisions through the key variable weight distribution.

[0066] It should be noted that the model training process can adopt a distributed architecture, where historical datasets are sharded and stored on multiple computing nodes, each node asynchronously calculates gradients, and the gradients are aggregated and global model parameters are updated through a parameter server.

[0067] Step S306: Calculate the time step reward value based on the updated reward function, calculate the advantage function by combining the state value estimate and the time step reward value, and update the network parameters through gradient backpropagation based on the advantage function; Specifically, the reward function, which is dynamically updated according to the event type, is first used to quantify and score the consequences of the actions taken (or simulated) by the model in historical data, and the time step reward value is obtained. This reward value reflects the immediate merits and demerits of the action in a specific context.

[0068] Next, the dominance function is calculated using state value estimation (a generalized dominance estimation algorithm can be employed). This dominance function is the difference between the actual (or estimated) reward of a specific action and the average value of the current state (estimated by the value network). A positive dominance value indicates that the action is better than average and should be encouraged; a negative dominance value indicates the opposite. Then, the network parameters are updated through gradient backpropagation. The calculated dominance function is used as a guiding signal, and the gradient of this signal with respect to each parameter in the policy network and value network is calculated using the backpropagation algorithm (i.e., in which direction and by how much should the parameters be adjusted to increase the probability of obtaining a high-dominance action).

[0069] Finally, an optimizer (such as Adam) is used to fine-tune all parameters along the gradient direction. This process is repeated, and the model continuously receives feedback and gradually adjusts its internal strategy, making its decisions increasingly inclined to obtain higher long-term cumulative rewards.

[0070] Step S307: When the prediction error calculated on the validation set based on the updated network parameters converges to the threshold range, output the completed reinforcement learning model and the weight distribution of key variables.

[0071] Specifically, the validation set is a portion of historical data that has never been used during training, used to independently evaluate the model's performance. Prediction error refers to the difference between the model's predicted state value or reward on the validation set and the actual historical value. As the number of training epochs increases, this error no longer decreases significantly but stabilizes around a pre-set acceptable level. This indicates that the model has fully extracted general patterns from the training data, and the final trained reinforcement learning model is a deployable agent that encapsulates the optimal decision-making strategy.

[0072] Meanwhile, the output key variable weight distribution is a summary of stable patterns of importance of different features learned by the attention mechanism in a large amount of training data, which is used to generate interpretable reports during real-time decision-making.

[0073] In the above implementation, the integrated proximal policy optimization algorithm ensures the stability of the training process. By introducing a dynamic reward function adjustment mechanism based on event identifiers, the model acquires the ability to understand the external environment and dynamically switch optimization priorities. The built-in attention mechanism not only improves the model's processing efficiency for high-dimensional state features but also generates crucial explanatory data for decision-making. The final trained model is no longer an automated program executing fixed rules, but an intelligent decision-making entity capable of balancing multiple objectives such as cost, efficiency, and patient satisfaction, and flexibly adjusting strategies according to public health emergencies. This provides a reliable and trustworthy AI-driven engine for hospitals to achieve refined, intelligent, and flexible dynamic optimization of operational costs.

[0074] Reference Figure 4 As a further implementation of the dynamic optimization method for hospital operating costs, before step S103, which involves training a reinforcement learning model based on a historical state vector dataset, the following method is also included: Step S401: Obtain historical external public health event alert data for a specified number of years and generate event type time series; The external risks faced by the hospital's operating environment are not static or isolated, but rather have profound temporal evolution characteristics and cross-scale transmission patterns. By retrieving multi-source alert data over long periods, including macro-level forward-looking WHO announcements, regionally guiding CDC early warning reports, and micro-level real-time in-hospital epidemic monitoring records, the system can construct a complete "far, medium, and near" risk perception perspective.

[0075] By transforming these heterogeneous data into event-type time series, the essence is to map unstructured, discrete public health emergency text information into structured mathematical objects arranged along a time axis. This transformation process enables the system not only to capture the instant of a single event, but also to uncover the sequential relationships between different events in the time dimension, the periodic outbreak patterns, and the spatiotemporal coupling characteristics of multiple concurrent events. This provides a historical experience foundation for subsequent reinforcement learning models to understand the complex dynamic evolution of the external environment.

[0076] Step S402: Perform cluster analysis on the event type time series, and identify the high-frequency event type set and the low-frequency event type set based on the cluster analysis results; In this embodiment, unsupervised machine learning algorithms (such as K-means) can be used to objectively define the natural distribution of massive historical events in a multidimensional feature space. In real public health environments, the frequency of different events and their impact on hospital operations exhibit extreme imbalances. For example, seasonal influenza is a recurring, high-frequency, routine event, while outbreaks of emerging infectious diseases or large-scale extreme weather events are rare, low-frequency, major events. Cluster analysis, by calculating the similarity distance between event type features, can automatically overcome the subjective limitations of manually pre-defined classifications, aggregating events with similar probabilities of occurrence and evolution patterns into a high-frequency event type set, and aggregating infrequent but highly destructive events into a low-frequency event type set.

[0077] Step S403: Based on the set of high-frequency event types and the set of low-frequency event types, select the corresponding high-frequency state vector subset and low-frequency state vector subset from the historical state vector dataset; Among them, the historical state vector dataset records digital snapshots of the hospital at various points in the past, including comprehensive information such as energy consumption, equipment status, and patient flow, while the event type set labels the specific external context in which these snapshots are located.

[0078] In this embodiment of the application, by aligning timestamps and matching event identifiers, the system can accurately extract operational status slices that are in the period of high-frequency event impact as high-frequency status vector subsets, and operational status slices that are under the impact of low-frequency events as low-frequency status vector subsets.

[0079] Understandably, the learning ability of a reinforcement learning model depends entirely on the distribution of the data samples it ingests. By performing this kind of refined stripping and classification based on event context, the model can distinguish which changes in state vectors are caused by regular fluctuations and which are caused by extreme events. This allows the model to establish a mapping relationship between specific external events and internal optimal coping strategies, further enhancing the accuracy of the model's output.

[0080] Step S404: Oversample the low-frequency state vector subset and merge the oversampled low-frequency state vector subset with the high-frequency state vector subset to generate a balanced state vector training set as the historical state vector dataset for subsequent training. Among them, since low-frequency events (such as major public health crises) are extremely rare in history, the sample size of the low-frequency state vector subset directly extracted is extremely scarce. If such an original imbalanced dataset is directly put into training, the model will be dominated by a large number of high-frequency regular samples during the gradient descent optimization process, resulting in an overfitting tendency. That is, the model will tend to ignore or even give up learning the coping strategies under low-frequency extreme events, leading to decision failure when facing real sudden crises.

[0081] Therefore, by introducing synthetic minority oversampling techniques (such as the SMOTE algorithm), the system does not simply replicate low-frequency samples. Instead, it artificially synthesizes a large number of new virtual edge scene state vectors that conform to the physical laws of low-frequency events in the feature space of the low-frequency state vectors based on the interpolation principle between samples. By reintegrating these oversampled low-frequency vectors with the original high-frequency vectors, the gap in the number of samples between different scenarios is completely eliminated from the bottom layer of data engineering. This generates a balanced state vector training set with coordinated sample ratios across various event scenarios, thereby forcing the model to treat and learn extremely rare events equally on the data supply side.

[0082] Step S405: Configure the event-aware loss function and configure low-frequency event penalty labels for samples belonging to the low-frequency state vector subset in the balanced state vector training set; The event-aware loss function is configured such that, during the training of the reinforcement learning model, when the input state vector is detected to carry a low-frequency event penalty label, the mean square error between the predicted reward value and the actual reward value calculated based on the state vector is added to the total loss as a penalty term.

[0083] Specifically, in the conventional training framework of proximal policy optimization algorithms, the loss function typically focuses only on maximizing the global expected reward. The event-aware loss function, however, introduces a targeted penalty and correction mechanism on this basis. This event-aware loss function has a two-term structure: the base term maintains the policy gradient updates of the model under normal states, while the penalty term specifically targets a subset of low-frequency state vectors.

[0084] In this embodiment, when the model processes low-frequency state vectors, if there is a significant deviation between the predicted reward value output by its policy network and the actual reward value that actually occurred in the past, the penalty term calculates the mean squared error between these two values ​​and adds it to the total loss with a significant non-negative weight coefficient. This design can increase the gradient penalty for the prediction error of low-frequency events during backpropagation, forcing the neural network to adjust the parameter weights of its internal fully connected layers and invest more network capacity to accurately fit those rare but fatal operational state transition patterns.

[0085] In the above implementation, the reinforcement learning model trained in the end can not only achieve a lean balance between cost and efficiency in daily operations, but also maintain the accuracy of strategies and the reliability of decisions when facing the impact of low-frequency public health emergencies, thus giving the hospital operation system strong robustness and long-term vitality.

[0086] Reference Figure 5 As one implementation of step S104, the steps of inputting the standardized state vector at the current moment into the trained reinforcement learning model, outputting optimized decision actions, generating a decision interpretability report based on the weight distribution of key variables, and calling a pre-configured set of ethical constraint rules to verify the actions include: Step S501: Input the standardized state vector at the current moment into the pre-trained reinforcement learning model, calculate the action probability distribution through the policy network, and sample to generate optimized decision actions; The standardized state vector at the current moment is a digital snapshot of the hospital's current environment (energy consumption, patient flow, resources, etc.). After this vector is input into the pre-trained reinforcement learning model, the model's policy network is activated. The policy network contains the optimal decision-making patterns learned from massive amounts of historical data. Based on the processing of the input state by the policy network, an action probability distribution covering all possible management actions (such as adjusting the room temperature setpoint, reallocating medical staff, and starting or stopping non-critical equipment) can be output.

[0087] It should be noted that this action probability distribution does not directly specify a unique action, but rather assigns a probability value to each possible action. The higher the probability, the more likely the model believes that the action is likely to bring long-term comprehensive benefits (taking into account cost, efficiency, and satisfaction) in the current position.

[0088] Subsequently, the system samples according to this probability distribution, selecting an action as the optimal decision action with a certain degree of randomness. It should be noted that "sampling" here does not mean taking the maximum probability, thus injecting a small amount of exploratory nature into the decision-making process, preventing the system from falling into absolutely deterministic outputs in edge scenarios, and maintaining the robustness of the decision.

[0089] Step S502: Call the key variable weight distribution output during the model training phase, analyze the weight values ​​of each feature dimension, and generate a decision interpretability report; During the model training phase, the attention mechanism has learned and solidified a set of key variable weight distribution data, which is essentially a mapping table that indicates the decision weight or attention of each feature dimension in the state vector (such as "current ICU bed occupancy rate", "instantaneous electricity price", "number of people waiting in the emergency room") when the model makes decision-making inferences.

[0090] Next, after the model makes a decision about the current state, the system immediately calls upon this distribution data to analyze the weight values ​​of each feature dimension. By identifying and extracting the top N features with the highest weights, the system can accurately pinpoint which key indicators dominated the decision. For example, if the generated decision is "reduce lighting power in non-medical areas," and the weight distribution shows that "peak electricity price" and "total power consumption of the entire hospital" have significantly higher weights than other features, then it can be inferred that the decision is primarily driven by cost pressures.

[0091] Subsequently, the system transforms the correlation between these high-weighted features and decision-making actions into a structured text report using a pre-defined natural language description template (e.g., "Given that [feature A] has a high value and its influence weight is significant, the system recommends taking [action X] to optimize [target Y]"). This decision interpretability report translates the machine's internal numerical calculation logic into business language that managers and healthcare workers can intuitively understand, greatly enhancing the trust foundation of human-machine collaboration and the transparency of decision-making.

[0092] Step S503: Query the set of ethical constraint rules pre-stored in the database; Step S504: Determine whether the optimization decision action violates the preset medical safety guidelines; if yes, proceed to step S505; otherwise, proceed to step S506. In this embodiment, the set of ethical constraint rules pre-stored in the database is a knowledge base defined in advance by domain experts, independent of the reinforcement learning model. It uses a structured approach (such as triples: triggering condition, prohibited action, associated consequence) to transform abstract medical safety principles (such as "uninterrupted power supply to the intensive care life support system must be guaranteed" and "delay in the treatment of critically ill patients due to resource allocation") into computable and matchable logical conditions.

[0093] Specifically, once an optimization decision action is generated, the system immediately queries the rule set and matches the action and its triggering context state with the rules. For example, if the generated decision action involves "reducing the load on a certain line during peak electricity consumption," and the rule set contains a rule that "when the ICU is powered by this line, any energy-saving instructions that may interrupt power supply are prohibited," and the current state meets the condition that the ICU is powered by this line, then the system will immediately determine that the action violates the preset medical safety guidelines.

[0094] Understandably, the principle behind this step is to establish a parallel, rule-based verification channel that does not question the effectiveness of the model's optimization logic but has the power to veto any decision that touches on safety red lines, ensuring that the exploration of artificial intelligence is always confined within an absolute framework that conforms to medical ethics and patient safety.

[0095] Step S505: Resample the action space based on the same state vector until a compliant optimized decision action is output and an ethical verification pass label is attached; Step S506: Directly output the optimized decision action that has passed the verification.

[0096] When an initial optimized solution is rejected for safety reasons, the system should not stagnate or simply revert to its previous state. Instead, it should proactively seek a "suboptimal but safe" alternative to ensure operational continuity. This search process does not involve retraining the model but rather performing constrained resampling within the existing decision-making framework.

[0097] Specifically, the system uses the same current state vector but applies an action mask to the model's action probability distribution. This mask, based on the ethical rule set, directly sets the probability of high-risk actions that violate the rules to zero. Then, the system re-normalizes this modified probability distribution (making the sum of the probabilities of all remaining compliant actions equal to 1) and performs secondary sampling until a compliant decision action is output. The principle behind this mechanism is to find the "optimal solution" under constraints by respecting and utilizing the model's original optimal decision-making tendencies learned from a large amount of data as much as possible, without violating safety constraints.

[0098] Ultimately, the system will attach an ethical verification pass label to this alternative compliance action. This label is not only a pass marker, but also a traceability record, indicating that the action is a revised version of the original decision, providing a basis for subsequent audits and model iterations.

[0099] The above implementation deeply integrates reinforcement learning decision-making capabilities with the safety prior knowledge and interpretability requirements of human experts, ensuring that the application of artificial intelligence in hospitals, a high-reliability scenario, is not only efficient but also safe and reliable. This technical solution transforms the generation of optimization decisions from a simple probability sampling process into a rigorous procedure with a "decision instruction manual" and a "safety compliance certificate," ultimately outputting a decision instruction that possesses both AI optimization potential and complies with medical ethical standards. This provides crucial technical assurance for the safe and reliable dynamic optimization of hospital operating costs.

[0100] Reference Figure 6 As a further implementation of the dynamic optimization method for hospital operating costs, after calculating the probability distribution of actions through a policy network and before sampling to generate optimization decision actions, it also includes: Step S601: Obtain a preset target priority plan library, query the target priority plan library based on the real-time event type identifier, and generate a target weight ranking table for multiple optimization targets; wherein, the multiple optimization targets include cost control, resource efficiency and patient satisfaction; Specifically, the pre-defined target priority contingency plan library is a rule knowledge base that encapsulates domain knowledge and historical experience. Its construction principle is based on in-depth analysis of the hospital's core value sequence under different types of emergencies.

[0101] In some embodiments, the contingency plan library may predefine: when identified as "infectious disease outbreak," controlling nosocomial infections and ensuring patient safety (i.e., the core of patient satisfaction) must be the highest priority and therefore given the highest weight; when identified as "regional power shortage," cost control to ensure the operation of critical life support systems (avoiding punitive high electricity prices or the risk of power outages due to excessive electricity consumption) is prioritized; and when identified as "mass casualties from a major traffic accident," resource efficiency (maximizing the throughput capacity of emergency access routes, operating rooms, and medical personnel) becomes crucial.

[0102] Understandably, by querying this contingency plan database and generating a target weight ranking table, the system is essentially automatically loading the corresponding execution principles based on the specific situation it faces. This ranking table is a weight vector that defines the relative importance of each target. It establishes an initial value coordinate system for all subsequent optimization calculations, ensuring that the system's decision focus can switch synchronously with the external challenges.

[0103] Step S602: Obtain the equipment load rate, number of emergency room patients, and peak energy consumption parameters from the standardized state vector at the current moment, and construct the target conflict resolution matrix; Among them, certain key features in the standardized state vector directly reflect the intense competition between different optimization objectives. For example, parameters such as equipment load rate (e.g., MRI equipment utilization rate), number of patients waiting in the emergency room (the number of patients waiting for treatment beyond the safe time limit), and peak energy consumption are the most direct and sensitive measures of the three core objectives of resource efficiency, patient satisfaction, and cost control, respectively.

[0104] However, these objectives often conflict with each other in the resource-constrained real world. The principle behind constructing an objective conflict resolution matrix is ​​to structurally and mathematically represent these conflict relationships. The rows and columns of this matrix typically represent different optimization objectives and state parameters (or directly represent the interactions between objectives). Each element value in the matrix represents the "conflict intensity" or "coupling relationship" between a certain objective and a certain state parameter, or between two objectives. For example, the matrix may reveal a strong positive correlation (conflict) between "increasing equipment load rate (objective: resource efficiency)" and "increasing equipment energy consumption and wear (objective: cost control)," while "reducing the number of people waiting in the emergency room (objective: patient satisfaction)" may require "temporarily increasing manpower, leading to increased costs (objective: cost control)."

[0105] By constructing this matrix, the system transforms qualitative questions such as "whether improving efficiency will increase costs" and "whether improving the user experience will sacrifice efficiency" into a mathematical model that can be quantitatively analyzed and calculated, thus facilitating the search for the optimal balance point in the next step.

[0106] Step S603: Input the target weight ranking table and the target conflict resolution matrix into the Nash equilibrium algorithm to calculate the Pareto compromise solution for multiple optimization objectives, and output the target weight adjustment coefficient vector based on the Pareto compromise solution. The target weight ranking table provides an initial value orientation based on event type, while the target conflict resolution matrix characterizes the inherent technical constraints and competitive relationships between targets. Directly applying the initial weights to decision-making may fail to achieve the desired outcome or even lead to system imbalance by ignoring actual conflicts.

[0107] In this embodiment, the Nash equilibrium algorithm can be used to solve this multi-objective collaborative optimization problem. The three optimization objectives—"cost control," "resource efficiency," and "patient satisfaction"—are simulated as multiple participants in a game. Each participant attempts to maximize its own payoff (i.e., achieve its own objective), but its payoff function is constrained by the conflict matrix, and their strategies (i.e., resource allocation schemes) influence each other. The algorithm iteratively calculates to find a Nash equilibrium point where no "objective participant" can obtain a higher payoff by unilaterally changing its strategy (i.e., demanding more resource allocation), as this would result in strong countermeasures from other objectives, leading to a decrease in overall payoff. This equilibrium point corresponds to the Pareto optimal solution or Pareto compromise in multi-objective optimization theory, representing an ideal balance where no other objective can be improved without worsening the existing one.

[0108] Ultimately, the output target weight adjustment coefficient vector is precisely the mapping of this equilibrium point in the weight space. It does not simply copy the initial weights from the contingency plan library, but rather, while respecting the initial value orientation, it scientifically "fine-tunes" and "rebalances" based on the current real-time conflict situation, thus deriving a set of the most feasible and coordinated ultimate target weight scheme in the current specific context.

[0109] Step S604: Update the target weight adjustment coefficient vector to the weight coefficients of the reward function to generate the updated reward function; In this process, the weight coefficients representing the optimal trade-off strategy are dynamically injected into the "value judgment standard" of the reinforcement learning model. By replacing the original weight coefficients in the reward function, the model is directly guided to produce decision-making behavior that conforms to the strategy.

[0110] In this embodiment, the technical essence of updating the model parameters is the real-time reconfiguration of the model's decision-making basis. For example, in the event of a power shortage, the initial plan might assign a high weight to cost control, but the conflict matrix shows that excessive power saving would severely impact the operating room's resource efficiency. After calculation by the Nash equilibrium algorithm, an adjustment coefficient might be output that moderately reduces the weight of cost control and slightly increases the weight of resource efficiency. After updating this coefficient to the reward function, the model's value judgment will precisely shift when evaluating the actions of "turning off non-critical lighting" and "ensuring backup power supply to the operating room," thus favoring the option that achieves a balance between power saving and safety. This allows the reinforcement learning model to evolve from an agent with fixed value preferences into an agent whose value preferences can dynamically and adaptively evolve with external events and internal conflicts, ensuring a high degree of consistency between its decisions and the optimization goals of the higher-level system.

[0111] Step S605: Obtain each candidate action in the action probability distribution, calculate the value estimate of each candidate action based on the updated reward function, and select the top K candidate actions with the highest value estimates as the target action set. Specifically, after the reward function weights are dynamically updated, the value estimates (i.e. Q-values ​​or expected cumulative returns) of all candidate actions (such as various equipment scheduling, energy consumption adjustment, and personnel allocation schemes) output by the previous policy network can be recalculated to ensure that the score of each action truly reflects the latest and most reasonable multi-objective collaborative optimization requirements.

[0112] Subsequently, the system does not simply select the action with the highest score, but instead filters the top K candidate actions with the highest value estimates to ensure that the subsequent sampling steps are provided with the optimal solution set currently available, representing the decision direction with the most comprehensive value under the latest weights. More importantly, considering that in complex medical scenarios, the optimal action may be rejected in ethical rule verification due to touching certain hard safety clauses (such as affecting specific critically ill patients), the slightly lower-ranked "second-best" actions become alternatives. They also have high comprehensive value and may perfectly comply with all ethical constraints.

[0113] Step S606: The target action set is used as a sampling candidate set for generating optimized decision actions, and sampling is performed from it to generate optimized decision actions.

[0114] Understandably, in traditional reinforcement learning frameworks, the action probability distribution output by the policy network typically covers all possible physical control commands. Randomly sampling within such a vast action space filled with invalid or suboptimal solutions can easily lead to local optima or generate marginal actions that deviate from actual business requirements. Setting the target action set as the sole sampling candidate set essentially redistributes and rigidly truncates the probability quality of the action space at a mathematical level. This forces the exploration probabilities, originally scattered across invalid actions, to collapse into an elite subspace composed of Pareto compromise solutions.

[0115] Furthermore, even within this limited candidate set, the system retains a probability-based random sampling mechanism rather than directly adopting a greedy strategy to select the highest-value action. The core logic of this design is to maintain a small exploratory entropy to prevent the decision model from degenerating into a rigid mapping table when facing a highly dynamic hospital environment. This ensures that the final optimized decision action is not only absolutely subordinate to the mathematically optimal compromise frontier, but also has robustness in dealing with small fluctuations in the real-time state.

[0116] For example, when the set of target actions includes three optimal options calculated by Nash equilibrium: "increase the air conditioning speed in the operating room by 10%", "increase by 15%", and "increase by 20%", the system will sample based on the relative posterior probabilities of these three actions in the current state. Regardless of which one is selected, it can ensure that it achieves the best balance between cost and infection control.

[0117] In the above implementation, the hospital's intelligent operation system can not only respond to external events, but also deeply understand the complex and dynamic trade-offs between multiple internal objectives. It can also use rigorous mathematical methods to automatically find the scientific balance point under specific constraints at specific times, thereby outputting optimized decision-making solutions that are in line with macro-strategic guidance, take into account micro-realistic constraints, and are technologically superior. Ultimately, it achieves sustainable and adaptive high-level synergistic optimization among cost, efficiency, and patient experience, further improving the scientific and refined level of smart hospital operation and management.

[0118] Reference Figure 7 As one implementation of step S105, the steps of collecting device status update data and patient flow change data after the action is executed, and calculating the actual status and actual reward value based on the reward function, include: Step S701: Collect equipment status update data in real time after the execution of optimization decision-making actions through the Internet of Things sensor network, and simultaneously obtain patient flow change data for the corresponding time period. In this process, once optimization decisions (such as adjusting HVAC setpoints or shutting down lighting in non-critical areas) are issued to specific devices through the building management system, their effectiveness must be objectively quantified. Internet of Things (IoT) sensor networks (such as smart meters, water meters, and sensors built into device controllers) form the sensory nerve endings of this quantification process, capable of collecting device status update data at high frequency and in real time, including device energy consumption change data (such as instantaneous power and flow values) and operational status logs (such as start / stop status, operating mode, and alarm information).

[0119] For example, after executing the instruction to "reduce the nighttime air supply temperature of the inpatient building by 1 degree Celsius", the sensor network immediately begins to monitor the power consumption changes, operating load, and other logs of the relevant air conditioning units and terminal equipment to assess whether the decision has indeed achieved the expected energy-saving effect and whether it has caused any unexpected equipment malfunctions.

[0120] Simultaneously, the assessment needs to cover the impact on patient flow and the utilization of medical resources, namely, obtaining patient flow change data from hospital information systems (such as HIS, LIS, PACS, and triage systems), including the number of registrations in each department (reflecting changes in demand distribution), the length of emergency waiting queues (directly related to the waiting time and safety of emergency patients), and the utilization rate of examination equipment (reflecting the efficiency of large medical assets). For example, a decision to "temporarily reinforce some outpatient nurses in the emergency department" directly aims to shorten the length of the emergency queue. This effect will not be immediately apparent after the instruction is issued, but will be reflected in real-time queue data in the information system several minutes to tens of minutes later.

[0121] Step S702: Perform time window alignment processing on the collected equipment status update data and patient flow change data to generate a status update dataset with consistent timestamps; Because the equipment response and the patient's behavioral response have different time constants (for example, the air conditioning power adjustment stabilizes within a few minutes, while the patient diversion effect may take half an hour to become clear), the original time-series data cannot be directly superimposed and analyzed.

[0122] Therefore, this difference can be reconciled by defining a dynamic observation window based on the decision execution time through time window alignment. For different types of decision actions, the time period during which their main impact manifests can be preset. For example, for equipment control actions, their effects are mainly reflected in a short "steady-state operation period" afterward, so the observation window may focus more on the short time after execution; while for manpower scheduling actions, their effects (such as shortening queue length) may have a brief period of chaos before entering the "effect manifestation period," thus requiring a longer backward observation window.

[0123] Through this dynamic sliding window mechanism, the system can extract data segments that logically belong to the same period of influence of the decision from data streams with different frequencies and delays, and integrate them into a state update dataset with consistent timestamps.

[0124] Step S703: Use the same feature dimensionality reduction algorithm and standardization parameters as in the model training phase to perform dimensionality reduction and standardization on the state update dataset, and generate a standardized actual state vector as the actual state. During the model training and decision-making phases, the system uses state vectors that have been processed by specific feature dimensionality reduction algorithms (such as Principal Component Analysis (PCA)) and standardized (such as Z-score standardization, which depends on the mean and standard deviation of the training set). To evaluate the new state after the decision is executed, it must be measured using the exact same "scale".

[0125] Therefore, it is necessary to call the pre-loaded model parameters saved during the training phase, including the projection matrix required for dimensionality reduction, the mean vector required for standardization, and the variance scaling factor. The time-aligned dataset is input into the same feature processing pipeline, ultimately generating a standardized actual state vector S_{t+1}. Here, "S{t+1}" represents the new state entered after the decision action is executed. Only when S{t+1} is completely isomorphic to the state space (i.e., the dimension of the vector, the physical meaning of each dimension, and the numerical range) during model training and decision-making can the model understand this new state and calculate an actual reward value comparable to the previously predicted value.

[0126] Step S704: Calculate the actual reward value based on the reward function used during reinforcement learning model training, inputting the standardized actual state vector; The reward function used during reinforcement learning model training is a mathematical expression that defines the system's optimization objective. It integrates multiple dimensions such as cost, efficiency, and satisfaction, and may dynamically adjust weights based on event types. When calculating the actual reward, the specific event type identifier and its corresponding weight coefficient used in the decision generation must be "locked in" or "reused." This is because evaluating the quality of an action must be based on the specific context and objective priority faced when that action is performed.

[0127] For example, in the context of an "infectious disease outbreak," a decision that increases energy consumption but reduces population density should be evaluated using a reward function that increases the weight of "infection control." Using the weights from the normal case to calculate the actual reward would lead to distorted evaluation. Therefore, when calculating the actual reward, the system looks back to the context of the decision-making process, invoking the same version of the weighting rules to ensure consistency in the evaluation criteria. By inputting the standardized actual state vector S{t+1} into this "context-frozen" reward function, the calculated actual reward value represents the real-time gain (or loss) that the decision actually generates in the real world, under its specific objective.

[0128] Step S705: Obtain the predicted reward value generated by the reinforcement learning model when outputting the optimization decision action, and store the actual reward value and the predicted reward value in the reward comparison database.

[0129] In the decision generation phase, the reinforcement learning model outputs an action along with its value network, which outputs a predicted reward value (Rpredicted) for the future return of that action. This represents the model's expectation that the action will bring. After execution and evaluation, the actual reward value (Ractual) is obtained, representing the actual return brought by the action. By storing this logarithm (Rpredicted, Ractual) along with its corresponding contextual information such as state, action, and event identifiers in a dedicated reward comparison database, a high-quality dataset is created to supervise the model's prediction accuracy.

[0130] Understandably, the accumulated (Rpredict, Ractual) data pairs can systematically reveal whether there are systematic biases in the model's predictions (such as persistent overestimation or underestimation), and in which contexts such biases are more significant (such as specific event types or specific departments). This database is the fuel driving the model's continuous self-optimization. When the deviation between the prediction and the actual reward exceeds a preset threshold, it indicates that the model is no longer adapted to the dynamic changes in the real world. At this point, based on these real feedback data, the model's incremental learning or parameter update process can be triggered, enabling the entire system to continuously learn and improve from actual operating results, thus becoming a truly adaptive intelligent system.

[0131] In the above implementation, synchronous acquisition of device response and patient flow data ensures the comprehensiveness of the evaluation. Innovative dynamic time window alignment processing solves the key challenge of asynchronous timing of multi-source responses in medical scenarios. Strict reuse of the feature processing pipeline and reward function calculation context from the training phase ensures the consistency between the evaluation benchmark and the model's cognitive framework, giving the comparison between predicted and actual values ​​rigorous mathematical and logical significance. Finally, by systematically recording "prediction-actual" reward pairs, not only is an objective quantitative evaluation of the effectiveness of a single decision provided, but also high-quality, scenario-based feedback data is provided for the continuous optimization and iteration of the reinforcement learning model.

[0132] Reference Figure 8 As one implementation of step S106, the step of evaluating the synergistic optimization results of cost savings rate, resource utilization rate, and patient satisfaction index based on actual conditions includes: Step S801: Based on the actual situation, extract the equipment energy consumption data and calculate the cost saving rate per unit time; The standardized actual state vector includes new equipment energy consumption data after the decision is executed, such as consumption readings for electricity, water, and medical gases.

[0133] In this embodiment, the key to calculating the cost savings rate per unit time lies in establishing a comparable benchmark. This involves parsing the actual energy consumption values ​​of each component from the standardized actual state vector S_{t+1} and converting them into monetary costs based on the hospital's real-time or contractual energy procurement prices. Subsequently, this actual cost needs to be compared with a cost benchmark, typically selected as the historical average cost over one or more similar time periods prior to the decision (e.g., the same time period on the same day last week), or the model-predicted cost under no-intervention conditions. The cost savings rate can then be calculated using the formula (benchmark cost - actual cost) / benchmark cost.

[0134] In practical applications, this cost-saving rate can directly answer questions such as "How much energy cost has this air conditioning system optimization strategy saved compared to the conventional approach?", providing a clear and direct input for financial evaluation.

[0135] Step S802: Based on the actual situation, extract the patient flow data to calculate the department-level resource utilization rate; Patient flow data contains information about the process of resource utilization and service delivery. The core of calculating departmental resource utilization is recognizing the heterogeneity of medical resources and integrating them within a unified evaluation framework. This typically involves calculating a composite indicator that comprehensively considers key resource dimensions: equipment utilization (such as the ratio of actual CT and MRI scan time to planned open time) reflects the intensity of use of high-value fixed assets; bed turnover rate (the number of patients served per bed per unit time) measures the efficiency of the flow of core inpatient resources; and human resource saturation (the proportion of time medical staff directly serve patients out of their total on-duty time) assesses the effectiveness of human resource input.

[0136] Because each department (such as emergency, internal medicine, and surgery) has different resource composition and operational priorities, the final departmental resource utilization rate is obtained by weighted summation after assigning different weight coefficients (α, β, γ) to the above-mentioned sub-indicators. For example, the emergency department may be given a higher weight for human resource saturation, while the inpatient department focuses more on bed turnover rate.

[0137] In some embodiments, the specific calculation formula is: Final resource utilization rate = α × equipment utilization rate + β × bed turnover rate + γ × human resource saturation, where α + β + γ = 1. The weighting coefficients (α, β, γ) can be dynamically configured according to the department type. For example, γ = 0.5 for emergency departments, β = 0.5 for inpatient departments, and α = 0.4 for outpatient departments. The configuration rules can be stored in the department metadata table of the hospital management system.

[0138] Step S803: Obtain patient satisfaction survey data within a preset time window, and calculate the patient satisfaction index based on the patient satisfaction survey data; The patient satisfaction survey database stores anonymous feedback collected through questionnaires, tablets, mobile devices, and other means. The matching time window refers to selecting feedback records submitted by patients who sought medical attention within the affected time period after the decision-making action was taken, to ensure the relevance of the assessment.

[0139] In this embodiment, the satisfaction index is typically calculated using a bimodal quantification strategy to overcome the limitations of a single scoring method. The principle is to combine structured data (such as a five-point Likert scale, ranging from "1 - very dissatisfied" to "5 - very satisfied") and unstructured text data (such as text descriptions in open rating boxes). For the text portion, natural language processing techniques, particularly sentiment analysis, can be used to identify the emotional tendencies (positive, neutral, negative) expressed in the text and words related to key service dimensions (such as waiting time, doctor-patient communication, and environment). Ultimately, the satisfaction index is a composite value obtained by weighting the average scale score with the positive sentiment rate of the text.

[0140] In some embodiments, sentiment analysis can employ a pre-trained model from the medical field, which, upon inputting feedback text, outputs sentiment tendency values ​​containing three key factors: "pain relief," "waiting time," and "service attitude." The calculation formula is: positive sentiment rate = number of positive factors / total number of factors × 100%.

[0141] Step S804: Construct a three-dimensional evaluation matrix based on unit time cost saving rate, departmental resource utilization rate, and patient satisfaction index; The core of this step is to establish a unified and visualized evaluation space for the typical multi-objective optimization problem of hospital operation, thereby incorporating the three inherently conflicting objectives (such as excessive cost saving may affect satisfaction, and blindly pursuing satisfaction may sacrifice efficiency) into the same analytical framework for overall consideration.

[0142] Specifically, constructing a three-dimensional evaluation matrix transforms the effects of each decision execution into a coordinate point in three-dimensional space, represented by three independent scalar indicators. Cost savings rate (X-axis), resource utilization rate (Y-axis), and patient satisfaction index (Z-axis) represent the three fundamental and crucial dimensions of operational optimization: economy, productivity, and human factors, respectively. By mapping the effect of a decision to a point P(x,y,z) in space, a complete description is made of "where one has reached in the value space jointly constituted by cost, efficiency, and experience." This transforms the comparison of the merits of different decisions from comparing a few separate numbers to comparing the "position" and "merit" of points in three-dimensional space, laying the geometric foundation for further scientific analysis using multi-objective optimization theory.

[0143] Step S805: The non-dominated solution set of the three-dimensional evaluation matrix is ​​identified by the Pareto front analysis algorithm, and a collaborative optimization result report is generated based on the identification results. Within the three-dimensional evaluation matrix, there are numerous historical or simulated decision effect points. The goal of the Pareto front analysis algorithm is to select the non-dominated solution set, i.e., the Pareto optimal solution set, from these.

[0144] In this embodiment, the Pareto front analysis algorithm execution process includes: establishing objective functions in three-dimensional space: Minimize(1-cost saving rate), Maximize(resource utilization rate), and Maximize(patient satisfaction index); iteratively selecting the optimal solution set using a non-dominated sorting genetic algorithm and marking it as the collaborative optimization frontier point. This Pareto front clearly reveals the objectively existing technical trade-off boundary between cost, resources, and satisfaction under current operating conditions.

[0145] Next, the effect point P of this decision is compared with this Pareto front, generating a co-optimization result report. The report can indicate how far point P is from the Pareto front, in which objective dimension there is potential for improvement, and in which region of the front it is located (whether it is a cost-sensitive region, an efficiency-sensitive region, or an experience-sensitive region). This quantitative analysis upgrades the evaluation of the decision's effect from a vague judgment of "good" or "bad" to a precise measurement of the degree of optimality under multiple constraints.

[0146] Step S806: Obtain the preset optimization benchmark value, compare the collaborative optimization result report with the optimization benchmark value, and output the optimization target achievement label based on the comparison result.

[0147] The preset optimization benchmark is a multi-dimensional combination of objectives. For example, it could be "achieving a cost saving rate of more than 5% and a resource utilization rate of more than 85% under the premise that the patient satisfaction index is not lower than 4.2", or it could be an ideal reference point selected on the historical Pareto frontier.

[0148] In this embodiment, the collaborative optimization result report is compared with the optimization benchmark value. This comparison considers not only whether individual indicators meet the target, but also whether the three-dimensional coordinates (cost, efficiency, satisfaction) achieved by this decision approach or exceed the "ideal point" representing the manager's value orientation. Based on this comparison, the system outputs an optimization goal achievement label. This label can be categorized, such as "fully achieved," "partially achieved (efficiency-driven)," or "not achieved (patient experience needs attention)"; it can also be quantitative, such as a percentage of achievement calculated based on the distance from the ideal point.

[0149] Understandably, optimizing the achievement labels provides hospital managers with an intuitive performance dashboard, while also offering the reinforcement learning model itself a sophisticated and comprehensive reward signal. The model can use these labels to reflect on its strategy: if the decision effect point is far from the Pareto front or fails to achieve the performance label, it indicates a flaw in its co-optimization and requires adjustment; if the effect point falls on the frontier and achieves a high performance label, it proves the effectiveness of its strategy.

[0150] The above implementation method overcomes the limitations of traditional evaluation methods that only focus on cost reduction. By constructing a three-dimensional evaluation matrix of "cost-resources-satisfaction," it holistically portrays the three core objectives of economy, operation, and human factors within a unified space. Furthermore, by introducing Pareto front analysis, it clearly identifies the optimal balance boundary under the constraints of multiple objectives, thereby accurately positioning the effect of any optimization decision. Finally, by comparing with preset management goals, it outputs a clear achievement label.

[0151] In practical applications, this technical solution not only provides hospital managers with a core strategy for evaluating the comprehensive value of operational decisions, enabling them to understand the trade-offs between different objectives and make scientific decisions, but also provides high-quality evaluation feedback signals to drive the upper-level reinforcement learning model toward achieving balanced and sustainable "collaborative optimization," ensuring that the development of the intelligent operation system does not deviate from the hospital's long-term value track of balancing efficiency, effectiveness, and human touch.

[0152] This application also discloses a dynamic optimization system for hospital operating costs based on reinforcement learning.

[0153] A dynamic optimization system for hospital operating costs based on reinforcement learning, specifically including: The multi-source sensing and state modeling module is used to collect real-time energy consumption data and equipment status data from IoT sensors, patient flow data, management database data, and external public health event alarm data. It preprocesses the collected data and generates standardized state vectors through feature dimensionality reduction. The event type parsing module is used to extract real-time event type identifiers based on external public health event alert data; The adaptive policy training module is used to train reinforcement learning models based on historical state vector datasets. It uses a proximal policy optimization algorithm to design the reward function, dynamically adjusts the weight coefficients of the reward function according to the real-time event type identifier, and integrates an attention mechanism to generate the weight distribution of key variables. The intelligent decision-making and verification module is used to input the standardized state vector of the current moment into the trained reinforcement learning model, output optimized decision actions, generate a decision interpretability report based on the weight distribution of key variables, and call the pre-configured set of ethical constraint rules to verify the actions. The execution feedback and evaluation module is used to send the optimized decision action that has passed the action verification to the hospital building management system through the application programming interface, collect the equipment status update data and patient flow change data after the action is executed, and calculate the actual status and actual reward value based on the reward function. The model iteration and collaborative optimization module is used to compare the deviation between the predicted reward value output by the reinforcement learning model and the actual reward value. When the deviation exceeds a set threshold, it triggers the reinforcement learning model parameter update instruction and evaluates the collaborative optimization results of cost saving rate, resource utilization rate and patient satisfaction index based on the actual situation.

[0154] The hospital operation cost dynamic optimization system based on reinforcement learning according to the embodiments of this application can implement any of the above methods, and the specific working process of each module in the system can refer to the corresponding process in the above method embodiments.

[0155] In the several embodiments provided in this application, it should be understood that the provided methods and systems can be implemented in other ways. For example, the system embodiments described above are merely illustrative; for example, the division of a certain module is merely a logical functional division, and in actual implementation there may be other division methods, such as multiple modules can be combined or integrated into another system, or some features can be ignored or not executed.

[0156] This application also discloses a computer-readable storage medium.

[0157] A computer-readable storage medium storing a computer program that can be loaded by a processor and executed as described above in any of the reinforcement learning-based methods for dynamic optimization of hospital operating costs.

[0158] The computer-readable storage medium can be any tangible medium that contains or stores a program that can be used by or in connection with an instruction execution system, apparatus, or device; the program code contained on the computer-readable medium can be transmitted using any suitable medium, including but not limited to wireless, wire, optical fiber, RF, etc., or any suitable combination thereof.

[0159] The above are all preferred embodiments of this application and are not intended to limit the scope of protection of this application. Any feature disclosed in this specification (including the abstract and drawings) may be replaced by other equivalent or similar features unless specifically stated otherwise. That is, unless specifically stated otherwise, each feature is only one example of a series of equivalent or similar features.

Claims

1. A method for dynamic optimization of hospital operating costs based on reinforcement learning, characterized in that, The method includes: Collect real-time energy consumption data and equipment status data from IoT sensors, patient flow data, management database data, and external public health event alarm data. Preprocess the collected data and generate standardized state vectors through feature dimensionality reduction. Extract real-time event type identifiers from the external public health event alert data; A reinforcement learning model is trained based on a historical state vector dataset. A reward function is designed using a proximal policy optimization algorithm. The weight coefficients of the reward function are dynamically adjusted according to the real-time event type identifier. An attention mechanism is integrated to generate the weight distribution of key variables. The standardized state vector at the current moment is input into the trained reinforcement learning model, which outputs an optimized decision action. At the same time, an interpretability report of the decision is generated based on the weight distribution of the key variables, and the pre-configured set of ethical constraint rules is called to verify the action. The optimized decision action, which has passed the action verification, is sent to the hospital building management system through the application programming interface. After the action is executed, the equipment status update data and patient flow change data are collected. The actual status and actual reward value are calculated based on the reward function. The deviation between the predicted reward value output by the reinforcement learning model and the actual reward value is compared. When the deviation exceeds a set threshold, a parameter update instruction for the reinforcement learning model is triggered. Based on the actual state, the synergistic optimization results of cost saving rate, resource utilization rate and patient satisfaction index are evaluated to obtain the degree of achievement of the optimization goal.

2. The method for dynamic optimization of hospital operating costs based on reinforcement learning according to claim 1, characterized in that, The steps for extracting real-time event type identifiers from the external public health event alert data include: Receives external public health event alert data streams, including notifications from the World Health Organization, early warning reports from disease control centers, and internal hospital epidemic monitoring records; The unstructured text alert data in the external public health event alert data stream is processed by word segmentation and entity recognition, and the event type keyword set and geographical impact range parameter are extracted. The event type keyword set is input into a pre-trained word vector model to generate event semantic feature vectors; Calculate the spatial correlation matrix of events based on the geographical influence range parameter, and construct a multidimensional event feature vector by combining the event semantic feature vector; The system queries a preset hospital event type mapping rule base, matches the multidimensional event feature vector to a preset event type set, and outputs a real-time event type identifier.

3. The method for dynamic optimization of hospital operating costs based on reinforcement learning according to claim 1, characterized in that, The steps include training a reinforcement learning model based on a historical state vector dataset, designing a reward function using a proximal policy optimization algorithm, dynamically adjusting the weight coefficients of the reward function according to the real-time event type identifier, and integrating an attention mechanism to generate the weight distribution of key variables. Obtain the standardized state vector dataset for historical time periods and the event type identifier for the corresponding time. Initialize the network parameters and base weights of the reward function of the reinforcement learning model; Based on the network parameters and the near-end policy optimization algorithm, a policy network and a value network are constructed. The standardized state vector of the historical period is input into the policy network and the value network to generate action probability distribution and state value estimation, respectively. The preset weight adjustment rule base is queried according to the event type identifier, and the weight coefficient of the reward function is dynamically updated. An attention mechanism layer is configured in the policy network and the value network. The attention mechanism layer is used to calculate the attention scores of each feature dimension of the standardized state vector of the historical period and generate the weight distribution of key variables. The time-step reward value is calculated based on the updated reward function. The advantage function is calculated by combining the state value estimate with the time-step reward value. The network parameters are then updated by gradient backpropagation based on the advantage function. When the prediction error calculated on the validation set based on the updated network parameters converges to the threshold range, the trained reinforcement learning model and the weight distribution of the key variables are output.

4. The method for dynamic optimization of hospital operating costs based on reinforcement learning according to claim 3, characterized in that, Before the step of training the reinforcement learning model based on the historical state vector dataset, the following is also included: Acquire historical external public health event alert data over a specified period of time and generate time series of event types; Cluster analysis is performed on the time series of the event types, and the high-frequency event type set and low-frequency event type set are identified based on the cluster analysis results; Based on the set of high-frequency event types and the set of low-frequency event types, the corresponding high-frequency state vector subset and low-frequency state vector subset are selected from the historical state vector dataset; The low-frequency state vector subset is oversampled, and the oversampled low-frequency state vector subset is merged with the high-frequency state vector subset to generate a balanced state vector training set as the historical state vector dataset for subsequent training. Configure an event-aware loss function and configure low-frequency event penalty labels for samples belonging to the low-frequency state vector subset in the balanced state vector training set; wherein, the event-aware loss function is configured such that: during the training of the reinforcement learning model, when the input state vector is detected to carry the low-frequency event penalty label, the mean square error between the predicted reward value and the actual reward value calculated based on the state vector is added to the total loss as a penalty term.

5. The method for dynamic optimization of hospital operating costs based on reinforcement learning according to claim 1, characterized in that, The steps of inputting the standardized state vector at the current moment into the trained reinforcement learning model, outputting an optimized decision action, generating a decision interpretability report based on the weight distribution of the key variables, and calling a pre-configured set of ethical constraint rules to verify the action include: The standardized state vector at the current moment is input into the pre-trained reinforcement learning model, and the action probability distribution is calculated through the policy network to sample and generate optimized decision actions. Call the key variable weight distribution output during the model training phase, analyze the weight values ​​of each feature dimension, and generate a decision interpretability report; The system queries a pre-stored set of ethical constraint rules in the database to determine whether the optimization decision action violates preset medical safety guidelines. If yes, then resample the action space based on the same state vector until a compliant optimization decision action is output and an ethical verification pass label is attached; otherwise, directly output the verified optimization decision action.

6. The method for dynamic optimization of hospital operating costs based on reinforcement learning according to claim 5, characterized in that, After calculating the action probability distribution through the policy network and before sampling to generate the optimized decision action, the method further includes: A preset target priority plan library is obtained, and the target priority plan library is queried based on the real-time event type identifier to generate a target weight ranking table of multiple optimization targets; wherein, the multiple optimization targets include cost control, resource efficiency and patient satisfaction; The equipment load rate, number of emergency room patients, and peak energy consumption parameters are obtained from the standardized state vector at the current moment, and the target conflict resolution matrix is ​​constructed. The target weight sorting table and the target conflict resolution matrix are input into the Nash equilibrium algorithm to calculate the Pareto compromise solution of the multiple optimization objectives, and the target weight adjustment coefficient vector is output based on the Pareto compromise solution. The target weight adjustment coefficient vector is updated to the weight coefficients of the reward function to generate the updated reward function; Obtain each candidate action in the action probability distribution, calculate the value estimate of each candidate action based on the updated reward function, and select the top K candidate actions with the highest value estimates as the target action set; The target action set is used as a sampling candidate set for generating the optimized decision action, and sampling is performed from it to generate the optimized decision action.

7. The method for dynamic optimization of hospital operating costs based on reinforcement learning according to claim 1, characterized in that, The steps of collecting device status update data and patient flow change data after the action is executed, and calculating the actual status and actual reward value based on the reward function, include: The device status update data after the execution of optimization decision-making actions is collected in real time through the Internet of Things sensor network, and the patient flow change data for the corresponding time period is obtained simultaneously. The collected device status update data and patient flow change data are aligned with time windows to generate a status update dataset with consistent timestamps. The same feature dimensionality reduction algorithm and standardization parameters as those used in the model training phase are applied to the state update dataset for dimensionality reduction and standardization, generating a standardized actual state vector as the actual state. The actual reward value is calculated based on the reward function used during the training of the reinforcement learning model, by inputting the standardized actual state vector. Obtain the predicted reward value generated by the reinforcement learning model when outputting the optimization decision action, and store the actual reward value and the predicted reward value in the reward comparison database.

8. The method for dynamic optimization of hospital operating costs based on reinforcement learning according to claim 7, characterized in that, The steps for evaluating the synergistic optimization results of cost savings, resource utilization, and patient satisfaction index based on the actual situation include: Based on the actual situation, the energy consumption data of the equipment is extracted to calculate the cost saving rate per unit time. Based on the actual situation, the patient flow data is extracted to calculate the departmental resource utilization rate. Obtain patient satisfaction survey data within a preset time window, and calculate the patient satisfaction index based on the patient satisfaction survey data; A three-dimensional evaluation matrix is ​​constructed based on the unit time cost saving rate, the department-level resource utilization rate, and the patient satisfaction index. The non-dominated solution set of the three-dimensional evaluation matrix is ​​identified using the Pareto front analysis algorithm, and a collaborative optimization result report is generated based on the identification results. Obtain a preset optimization benchmark value, compare the collaborative optimization result report with the optimization benchmark value, and output an optimization target achievement label based on the comparison result.

9. A dynamic optimization system for hospital operating costs based on reinforcement learning, characterized in that, The system includes: The multi-source sensing and state modeling module is used to collect real-time energy consumption data and equipment status data from IoT sensors, patient flow data, management database data, and external public health event alarm data. It preprocesses the collected data and generates standardized state vectors through feature dimensionality reduction. The event type parsing module is used to extract real-time event type identifiers based on the external public health event alert data; The adaptive policy training module is used to train a reinforcement learning model based on a historical state vector dataset. It designs a reward function using a proximal policy optimization algorithm, dynamically adjusts the weight coefficients of the reward function according to the real-time event type identifier, and integrates an attention mechanism to generate the weight distribution of key variables. The intelligent decision-making and verification module is used to input the standardized state vector at the current moment into the trained reinforcement learning model, output optimized decision actions, generate a decision interpretability report based on the weight distribution of the key variables, and call the pre-configured set of ethical constraint rules to verify the actions. The execution feedback and evaluation module is used to send the optimized decision action that has passed the action verification to the hospital building management system through the application programming interface, collect equipment status update data and patient flow change data after the action is executed, and calculate the actual status and actual reward value based on the reward function. The model iteration and collaborative optimization module is used to compare the deviation between the predicted reward value output by the reinforcement learning model and the actual reward value. When the deviation exceeds a set threshold, a reinforcement learning model parameter update instruction is triggered, and the collaborative optimization results of cost saving rate, resource utilization rate and patient satisfaction index are evaluated based on the actual state.

10. A computer-readable storage medium, characterized in that: The computer program is stored that can be loaded by a processor and executed as described in any one of claims 1 to 8.

Images