Artificial intelligence body for intelligent switching and scheduling of multiple heat sources in hospital

Abstract

The invention discloses an artificial intelligence body for intelligent switching and scheduling of multiple heat sources of a hospital, and relates to the technical field of hospital heating management. The artificial intelligence body comprises an influence parameter acquisition module, a state space construction module, an action space construction module, a reward function calculation module and a reinforcement learning execution module which cooperate with one another to comprehensively consider heat source static capability parameters and dynamic operation data, and a load prediction and reinforcement learning optimization mechanism is combined, so that the dynamic performance of the heat source is improved. On-demand starting and stopping, dynamic switching and load self-adaptive distribution of heat sources are achieved, the energy saving performance, responsiveness and operation safety of the whole heating system are improved, then the optimal switching and load dispatching of the multi-heat-source heating system can be achieved, the intelligent level and operation efficiency of the system are remarkably improved, and practical application and popularization are facilitated.

Description

Technical Field

[0001] The present invention belongs to the technical field of hospital heating management, and in particular relates to an artificial intelligence entity for intelligent switching and scheduling of multiple heat sources in a hospital. Background Art

[0002] Currently, heating systems in buildings such as large hospitals are typically equipped with multiple heat sources (such as gas boilers, electric boilers, and district heating networks) to meet the heat needs of different areas at different times. However, existing heat source switching and scheduling solutions have the following problems in actual operation, affecting energy efficiency and system responsiveness:

[0003] (1) Insufficient utilization of load perception and environmental data: Traditional solutions have difficulty in obtaining real-time changes in the actual heat load of each area, and do not fully integrate environmental information such as outdoor temperature and time characteristics. This results in static and lagging heat source switching and scheduling strategies, making it impossible to effectively respond to demand fluctuations.

[0004] (2) Delayed response of heat source switching: Existing heat source switching and scheduling schemes are mostly based on fixed schedules or manual intervention to switch heat sources. They lack the ability to dynamically make optimal decisions based on the operating status of the heat source (such as current load, efficiency, or number of starts and stops), resulting in low energy utilization.

[0005] (3) Lack of optimization mechanism for load distribution: when multiple heat sources operate in coordination, the heat load is often distributed in a static proportion or average manner, failing to make intelligent adjustments based on the actual operating efficiency, remaining capacity, or lifespan of the heat sources, resulting in overload of some heat sources and increased operating costs;

[0006] (4) Unclear configuration of heat source roles: that is, the positioning of each heat source in operation in the existing heat source switching and scheduling scheme (such as the main heat source role, auxiliary heat source role, and backup heat source role, etc.) is not clearly set, resulting in vague scheduling rules, lack of hierarchical and priority control of heat source switching, and lack of logical consistency in operation decisions.

[0007] Therefore, how to provide an artificial intelligence-based multi-heat source intelligent switching and scheduling solution to comprehensively consider the static capacity parameters and dynamic operation data of the heat source, and combine load forecasting and reinforcement learning optimization mechanism to realize on-demand start and stop of heat sources, dynamic switching and adaptive load distribution, thereby improving the energy saving, responsiveness and operation safety of the overall heating system, is a topic that technical personnel in this field urgently need to study. Summary of the Invention

[0008] The purpose of the present invention is to provide an artificial intelligence entity for intelligent switching and scheduling of multiple heat sources in hospitals, so as to solve the problems of low energy efficiency and limited system response capability in actual operation of existing heat source switching and scheduling schemes.

[0009] In order to achieve the above object, the present invention adopts the following technical solutions:

[0010] In the first aspect, an artificial intelligence agent for intelligent switching and scheduling of multiple heat sources in a hospital is provided, comprising an influencing parameter acquisition module, a state space construction module, an action space construction module, a reward function calculation module, and a reinforcement learning execution module;

[0011] The influencing parameter acquisition module is used to collect all parameters that can affect the hospital's multi-heat source intelligent switching and scheduling decisions;

[0012] The state space construction module is used to construct the state space based on all the parameters;

[0013] The action space construction module is used to construct, for each state in the state space, at least one optional action in the corresponding state and used for performing intelligent switching and scheduling of multiple heat sources in the hospital;

[0014] The reward function calculation module is used to calculate an objective function containing multiple optimization objectives after selecting any optional action in any state belonging to the state space, based on various monitoring indicators of the hospital heating system in the next time step, and use the calculation result as the reward value corresponding to any state and any optional action, wherein the multiple optimization objectives include minimizing the total energy consumption of the hospital heating system and shortening the adjustment response time when the environment / load changes, ensuring the comfort of various areas in the hospital and / or equipment health management, and the equipment health management includes balancing the operating time of various heat source equipment in the hospital heating system, reducing the failure rate of each heat source equipment, extending the service life of each heat source equipment and / or preventing any heat source equipment in the hospital heating system from overloading;

[0015] The reinforcement learning execution module is used to determine the current state according to the real-time values of all the parameters in each time step, and then, based on the reward value of the current action, use the Q-learning algorithm to select and execute the next action for intelligent switching and scheduling of multiple heat sources in the hospital from at least one optional action in the current state.

[0016] Based on the above invention content, a new solution for intelligent switching and scheduling of multiple heat sources in hospitals with "environmental state perception-strategy intelligent decision-making-real-time control execution" as the core closed loop is provided, which includes an influencing parameter acquisition module, a state space construction module, an action space construction module, a reward function calculation module and a reinforcement learning execution module. Through their mutual collaboration, the static capacity parameters and dynamic operation data of the heat source can be comprehensively considered, and combined with load prediction and reinforcement learning optimization mechanism, the on-demand start and stop, dynamic switching and adaptive load distribution of the heat source can be realized, thereby improving the energy saving, responsiveness and operation safety of the overall heating system, and then realizing the optimal switching and load scheduling of the multi-heat source heating system, and significantly improving the intelligence level and operation efficiency of the system, which is convenient for practical application and promotion.

[0017] In one possible design, all the parameters include indoor environmental parameters of each area in the hospital, outdoor environmental parameters of the area outside the hospital, static parameters of each heat source equipment in the hospital heating system, dynamic parameters of each heat source equipment in the hospital heating system, heating load demand parameters and / or time factor parameters of each area in the hospital.

[0018] In one possible design, the influencing parameter acquisition module includes an environmental parameter acquisition unit, a static parameter acquisition unit, a dynamic parameter acquisition unit and / or a load demand acquisition unit;

[0019] The environmental parameter acquisition unit is used to periodically acquire indoor environmental parameters of various areas within the hospital and outdoor environmental parameters of areas outside the hospital, wherein the indoor environmental parameters include indoor temperature, humidity and air flow rate, and the outdoor environmental parameters include outdoor temperature, humidity and wind speed;

[0020] The static parameter acquisition unit is used to acquire the static parameters of each heat source device in the hospital heating system, wherein the static parameters include the rated heating power and energy efficiency coefficient of the corresponding device;

[0021] The dynamic parameter acquisition unit is used to periodically acquire the dynamic parameters of each heat source device in the hospital heating system, wherein the dynamic parameters include the real-time heating power, outlet / return water temperature, outlet / return water flow rate, switch status, operation mode, fault alarm status and operation time of the corresponding device;

[0022] The load demand acquisition unit is used to obtain heating load demand parameters of each area in the hospital based on historical load data and real-time load monitoring data of each area in the hospital, wherein the heating load demand parameters include current heating load and future heating load forecast values.

[0023] In one possible design, each state in the state space contains the outdoor temperature T out, the indoor temperature of the i-th area in the hospital and indoor humidity The current total heating load of the hospital is L current And the future total heating load forecast value L pred , the current energy efficiency P of the jth heat source equipment in the hospital heating system (j) and system control parameters S equip , where i and j represent positive integers respectively, and the system control parameters include equipment start / stop status values and / or regional temperature setting values.

[0024] In one possible design, the optional actions include the start-stop combination of all heat source equipment in the hospital heating system, the heating load distribution ratio, the water supply temperature setting target value and the operating frequency adjustment value.

[0025] In one possible design, the static parameters of each heat source device in the hospital heating system include the defined roles of the corresponding device, wherein the defined roles are divided into a main heat source role, an auxiliary heat source role, and a backup heat source role. The main heat source role refers to the device role that is started when the hospital heating system is started and assumes the basic load task. The auxiliary heat source role refers to the device role that is started in parallel to ensure the heating capacity of the system when the main heat source device cannot independently meet the load. The backup heat source role refers to the device role that automatically starts when the main / auxiliary heat source device fails to switch the failed device and ensure uninterrupted operation of the system.

[0026] The start-stop combination meets the following constraints:

[0027] If the first heat source device having the role of the main heat source is running, the second heat source device having the role of the auxiliary heat source is in a standby state and is started only when the first heat source device cannot meet the load independently;

[0028] If the third heat source device having the main heat source role / the auxiliary heat source role fails, the fourth heat source device having the backup heat source role starts to operate, and the third heat source device stops operating.

[0029] In one possible design, the objective function F is expressed as follows:

[0030]

[0031] Where w1, w2, w3, w4, w5, w6 and w7 represent the preset weight coefficients, E t represents the total energy consumption of the hospital heating system in the next time step, C t represents the operating cost of the hospital heating system in the next time step, N represents the total number of areas in the hospital, i represents a positive integer, represents the indoor temperature of the i-th area in the hospital, represents the target value of the indoor temperature of the i-th area, N switch Indicates the number of heat source switching times per unit time, represents the indoor humidity of the i-th area, represents the target value of indoor humidity in the i-th area, D response represents the time delay for the hospital heating system to respond to the indoor temperature and humidity target setting value or heating load change, M represents the total number of heat source devices in the hospital heating system, j represents a positive integer, Represents the current health status indicator of the j-th heat source equipment in the hospital heating system.

[0032] In one possible design, the current health status indicator includes a current load rate and / or a current remaining life, wherein the current load rate η and the current remaining life RUL are calculated according to the following formulas:

[0033]

[0034] Where, P Ture Indicates the current operating power of the corresponding device, P Rated Indicates the rated power of the corresponding equipment, L design Indicates the design life of the corresponding equipment, represents the moment, T represents the last moment in the next time step, Δt represents the sampling time interval, Indicates that the corresponding device is at time The load weighting factor.

[0035] In one possible design, based on the reward value of the current action, a Q-learning algorithm is used to select the next action for intelligent switching and scheduling of multiple heat sources in the hospital from at least one optional action in the current state, including:

[0036] Determining a currently faulty heat source device in the hospital heating system according to the current state;

[0037] Based on the reward value of the current action, the Q-learning algorithm is used to select the next action for intelligent switching and scheduling of multiple heat sources in the hospital from at least one optional action in the current state and prohibiting the startup of the currently faulty heat source equipment.

[0038] In a possible design, the action selection strategy in the Q-learning algorithm adopts an ε-greedy strategy.

[0039] Beneficial effects of the above scheme:

[0040] (1) The present invention creatively provides a new solution for intelligent switching and scheduling of multiple heat sources in hospitals with "environmental state perception-strategy intelligent decision-making-real-time control execution" as the core closed loop, which includes an influencing parameter acquisition module, a state space construction module, an action space construction module, a reward function calculation module and a reinforcement learning execution module. Through their mutual cooperation, the static capacity parameters and dynamic operation data of the heat source can be comprehensively considered, and combined with load prediction and reinforcement learning optimization mechanism, the heat source can be started and stopped on demand, dynamically switched and load adaptively distributed, and the energy saving, responsiveness and operation safety of the overall heating system can be improved. In addition, the optimal switching and load scheduling of the multi-heat source heating system can be achieved, and the intelligence level and operation efficiency of the system can be significantly improved, which is convenient for practical application and promotion.

[0041] (2) Reinforcement learning can be used to drive intelligent switching and scheduling of multiple heat sources: that is, to address the problems of untimely switching of multiple heat sources (such as air source heat pumps, municipal heating networks, and gas boilers) and the dependence of strategies on human experience, by adopting a reinforcement learning algorithm based on Q-learning, the optimal start and stop of heat sources and load distribution can be achieved while ensuring stable heating supply. After simulation and field testing, the system operating energy consumption can be reduced by 12% to 20%;

[0042] (3) Load forecasting and scheduling priority determination can be performed through multi-dimensional data fusion: that is, a load forecasting model can be constructed using multi-source data such as regional energy consumption history, weather forecast, building structure, and crowd density to achieve dynamic heating strategy adjustment for key areas (such as inpatient and outpatient areas), improve the system's response to load fluctuations, and ensure user comfort;

[0043] (4) The system adaptability can be improved through the ability of self-evolution of strategies: that is, unlike traditional rule control, the reinforcement learning agent has the ability of self-learning and strategy evolution. It can automatically adjust the scheduling strategy according to different time periods, equipment status and operation feedback, adapt to seasonal changes and the diversity of the operating environment, and improve the long-term stable operation capability of the system;

[0044] (5) It can optimize the timing of heat source use and reduce equipment loss and maintenance costs: that is, it can dynamically adjust the start and stop sequence of the heat source through the algorithm to avoid long-term high-load operation of the equipment, extend the equipment life, and reduce the failure rate; at the same time, it can trigger the switching mechanism in time when the equipment status is abnormal, reduce human intervention, and improve the safety and reliability of the system operation. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0046] Figure 1 A schematic diagram of the structure of an artificial intelligence entity for intelligent switching and scheduling of multiple heat sources in a hospital provided in an embodiment of the present application.

[0047] Figure 2 A schematic diagram of the process structure of the Q-learning algorithm provided in the embodiment of the present application. DETAILED DESCRIPTION

[0048] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the present invention will be briefly introduced below in conjunction with the drawings and the description of the embodiments or the prior art. Obviously, the following description of the structures of the drawings is only some embodiments of the present invention. For those skilled in the art, other embodiments can be obtained based on these embodiments without creative work. It should be noted that the description of these embodiments is used to help understand the present invention, but does not constitute a limitation of the present invention.

[0049] It should be understood that although the terms first, second, etc. may be used herein to describe various objects, these objects should not be limited by these terms. These terms are merely used to distinguish one object from another. For example, a first object can be referred to as a second object, and similarly, a second object can be referred to as a first object without departing from the scope of the exemplary embodiments of the present invention.

[0050] It should be understood that the term "and / or" that may appear in this document is merely a description of the association relationship between associated objects, indicating that there may be three relationships. For example, A and / or B can indicate three situations: A exists alone, B exists alone, or A and B exist at the same time. For another example, A, B and / or C can indicate the existence of any one of A, B and C or any combination of them. The term " / and" that may appear in this document describes another type of association object relationship, indicating that there may be two relationships. For example, A / and B can indicate two situations: A exists alone or A and B exist at the same time. In addition, the character " / " that may appear in this document generally indicates that the previous and next associated objects are in an "or" relationship.

[0051] Example

[0052] like Figures 1-2As shown, the artificial intelligence entity provided in this embodiment and used for intelligent switching and scheduling of multiple heat sources in hospitals includes but is not limited to an influencing parameter acquisition module, a state space construction module, an action space construction module, a reward function calculation module and a reinforcement learning execution module.

[0053] The influencing parameter acquisition module is used to collect all parameters that can affect the intelligent switching and scheduling decisions of multiple heat sources in the hospital. Specifically, all the parameters include but are not limited to indoor environmental parameters of various areas in the hospital (which have different requirements for different areas), outdoor environmental parameters of areas outside the hospital (which can be used to help adjust the operating mode of heat source equipment), static parameters of various heat source equipment in the hospital heating system (such as gas boilers, electric boilers and / or regional heating networks, etc.), dynamic parameters of various heat source equipment in the hospital heating system, heating load demand parameters and / or time factor parameters of various areas in the hospital (which have an impact on heating load demand based on time series laws), etc. In detail, the influencing parameter acquisition module includes but is not limited to an environmental parameter acquisition unit, a static parameter acquisition unit, a dynamic parameter acquisition unit and / or a load demand acquisition unit, etc.

[0054] The environmental parameter acquisition unit is used to periodically acquire the indoor environmental parameters of each area in the hospital and the outdoor environmental parameters of the area outside the hospital, wherein the indoor environmental parameters include but are not limited to indoor temperature and humidity and air flow rate, and the outdoor environmental parameters include but are not limited to outdoor temperature and humidity and wind speed. The aforementioned indoor environmental parameters and outdoor environmental parameters can be specifically acquired regularly through conventional environmental sensors such as temperature sensors, humidity sensors and anemometers. In addition, considering that the operating state of the hospital heating system is relatively stable at the minute level, and in order to provide sufficient time to complete calculations and make decisions for the hospital's multi-heat source intelligent switching and scheduling decision-making process, the aforementioned cycle can be specifically designed to be at the minute level, for example, the indoor environmental parameters and the outdoor environmental parameters are acquired every 5 minutes (i.e., the cycle length is equal to 5 minutes), 10 minutes or 15 minutes.

[0055] The static parameter acquisition unit is used to obtain the static parameters of each heat source device in the hospital heating system, wherein the static parameters include but are not limited to the rated heating power and energy efficiency coefficient of the corresponding equipment. The aforementioned static parameters can be obtained by routine input by operation and maintenance personnel. In addition, in order to achieve the purpose of orderly scheduling and safe operation of the heat source, the static parameters may also include the defined roles of the corresponding equipment, wherein the defined roles are divided into main heat source role, auxiliary heat source role and backup heat source role, etc. The main heat source role refers to the equipment role that starts when the hospital heating system starts and undertakes the basic load task, the auxiliary heat source role refers to the equipment role that starts in parallel to ensure the system's heating capacity when the main heat source equipment cannot independently meet the load, and the backup heat source role refers to the equipment role that automatically starts when the main / auxiliary heat source equipment fails to switch the faulty equipment and ensure uninterrupted operation of the system. The defined roles of each heat source device are determined by operations and maintenance personnel based on the device capabilities and system structure. These roles can be manually entered and stored in the device archive. Furthermore, they can be managed through system configuration files (i.e., the roles and responsibilities of each heat source device are clearly defined in the system configuration files, allowing the defined roles of different heat source devices to be determined by reading these configuration files during system operation management). Furthermore, the defined roles of each device can be dynamically adjusted during system operation based on information such as load demand, device status, and start / stop frequency, to achieve intelligent heat source reconfiguration.

[0056] The dynamic parameter acquisition unit is used to periodically acquire the dynamic parameters of each heat source device in the hospital heating system, wherein the dynamic parameters include but are not limited to the real-time heating power, outlet / return water temperature, outlet / return water flow, switch status, operation mode, fault alarm status and operation time of the corresponding equipment. The aforementioned cycle can also be specifically designed to be at the minute level, for example, the dynamic parameters of each heat source device are acquired every 5 minutes (i.e., the cycle duration is equal to 5 minutes), 10 minutes or 15 minutes. The dynamic parameters can be specifically acquired by real-time collection and uploading through existing Internet of Things sensors, for example, the real-time heating power of each heat source device can be acquired in real time through existing power meter products. In addition, the operation time can specifically adopt actual operation time or equivalent operation time, wherein the former can be obtained by real-time recording of the switch status of the corresponding device and accumulating the duration of the switch status being on, while the latter can be obtained by real-time recording of the real-time operating power of the corresponding device and calculating according to the following formula:

[0057] t Eqrun,m =(P real,m,T ×ΔT)÷P rated,m

[0058] In the formula, m represents a positive integer, t Eqrun,mP represents the equivalent operating time of the mth heat source equipment in the hospital heating system within a cycle, real,m,T represents the real-time operating power of the mth heat source device in the cycle, ΔT represents the cycle duration, P rated,m represents the rated power of the mth heat source device.

[0059] The load demand acquisition unit is configured to acquire heating load demand parameters for each area within the hospital based on historical load data and real-time load monitoring data for each area within the hospital. The heating load demand parameters include, but are not limited to, current heating load and future heating load forecast values. The current heating load can also be acquired through real-time collection and upload by existing IoT sensors; the future heating load forecast value can be conventionally estimated using existing time series prediction models in combination with historical load data.

[0060] The state space construction module is used to construct a state space based on all the parameters. Each state s in the state space is used to guide the agent to optimize the scheduling strategy, specifically considering factors such as energy consumption, operating costs and regional comfort. Specifically, each state in the state space includes but is not limited to the outdoor temperature T out , the indoor temperature of the i-th area in the hospital and indoor humidity The current total heating load of the hospital is L current And the future total heating load forecast value L pred , the current energy efficiency P of the jth heat source equipment in the hospital heating system (j) and system control parameters S equip etc. (i.e. the state at time t ), wherein i and j represent positive integers respectively, and the system control parameters include but are not limited to equipment start / stop status values and / or regional temperature setting values. The aforementioned current energy efficiency is specifically but not limited to COP (which is the heating performance coefficient of the air conditioner, indicating the heating capacity per unit power of the air conditioner) or gas consumption, etc. In addition, the state space construction module is communicatively connected to the influencing parameter acquisition module, and N represents the total number of regions in the hospital, and M represents the total number of heat source devices in the hospital heating system.

[0061] The action space construction module is used to construct, for each state in the state space, at least one optional action in the corresponding state for performing intelligent switching and scheduling of multiple heat sources in the hospital. The optional actions are used to represent the intelligent switching and scheduling strategies that the system can adopt for multiple heat sources in the hospital; specifically, the optional actions include, but are not limited to, the start-stop combinations of all heat source equipment in the hospital heating system, the heating load distribution ratio, the target value for the water supply temperature, and the operating frequency adjustment value. Examples of the aforementioned start-stop combinations include [heat pump on, boiler off, heat exchange station on], etc. In order to achieve the purpose of orderly scheduling and safe operation of heat sources, preferably, when the static parameters of each heat source device in the hospital heating system include the defined role of the corresponding device, the start-stop combination satisfies the following constraints: if the first heat source device with the main heat source role is running, the second heat source device with the auxiliary heat source role is in standby state, and is started only when the first heat source device cannot meet the load independently; if the third heat source device with the main heat source role / the auxiliary heat source role fails, the fourth heat source device with the backup heat source role starts and stops running the third heat source device. The aforementioned heating load distribution ratio can be used for a vector χ=[χ1,χ2,…,χ j ,…,χ M ] to express, where χ j represents the load distribution ratio value of the j-th heat source equipment, and has The aforementioned operating frequency may specifically be, but is not limited to, the frequency of a fan or a water pump. In addition, the action space construction module is communicatively connected to the state space construction module.

[0062] The reward function calculation module is configured to, after selecting any optional action in any state belonging to the state space, calculate an objective function containing multiple optimization objectives based on various monitoring indicators of the hospital heating system in the next time step, and use the calculation result as the reward value corresponding to the arbitrary state and the arbitrary optional action, wherein the multiple optimization objectives include, but are not limited to, minimizing the total energy consumption of the hospital heating system and minimizing the adjustment response time when the environment / load changes, ensuring the comfort level of various areas in the hospital and / or equipment health management, etc. The equipment health management includes, but is not limited to, balancing the operating time of various heat source devices in the hospital heating system, reducing the failure rate of each heat source device, extending the service life of each heat source device, and / or preventing any heat source device in the hospital heating system from overloading, etc. The aforementioned time step is exemplified by, but not limited to, 5 minutes, 10 minutes, or 15 minutes. The reward value is used to guide the intelligent agent to optimize the scheduling strategy, mainly considering the following factors: (A) Energy efficiency optimization: rewards are given when energy consumption is effectively reduced, and penalties are given otherwise. Specifically, the overall energy consumption of the system can be reduced through intelligent selection of heat sources and dynamic load distribution; (B) Comfort assurance: maintaining the accuracy of temperature and humidity control (usually humidity between 40% and 60%, temperature between 18 and 28 degrees Celsius), and rewards are obtained when the comfort level remains within the set range, that is, ensuring that the temperature and humidity in each area meet the requirements, especially for precise control of high-priority areas such as wards and operating rooms; (C) Response time: rewards are given for rapid response to temperature and humidity adjustments, and penalties are given for delays, that is, rapid adjustments are made when the environment or load changes, and priority is given to ensuring the adjustment speed of key areas; (D) Fault prevention and equipment maintenance: rewards are given for reducing equipment failures, and penalties are given otherwise, that is, balancing the operating time of each device, preventing overload operation, reducing the failure rate, and extending the service life of the equipment. Based on the above four factors, the above-mentioned multiple optimization objectives are constructed. In addition, the reward function calculation module is communicatively connected to the influencing parameter acquisition module, the state space construction module and the action space construction module respectively.

[0063] In order to accurately reflect the multiple optimization objectives, preferably, the objective function F is represented by but not limited to the following:

[0064]

[0065] Where w1, w2, w3, w4, w5, w6 and w7 represent the preset weight coefficients, E t represents the total energy consumption of the hospital heating system in the next time step, C t represents the operating cost of the hospital heating system in the next time step, N represents the total number of areas in the hospital, i represents a positive integer, represents the indoor temperature of the i-th area in the hospital, represents the target value of the indoor temperature of the i-th area, N switch Indicates the number of heat source switching times per unit time, represents the indoor humidity of the i-th area, represents the target value of indoor humidity in the i-th area, D response represents the time delay for the hospital heating system to respond to the target indoor temperature and humidity settings or changes in heating load (i.e., changes in the system's required heating / cooling capacity due to changes in the external environment or internal demand, where the instantaneous heat load can be estimated by the indoor and outdoor temperature difference, heat transfer coefficient, and air volume). M represents the total number of heat source devices in the hospital heating system, and j represents a positive integer. The weight coefficients w1, w2, w3, w4, w5, w6 and w7 can be manually set according to specific operation requirements, or can be uniformly normalized to ∑w k =1, k=1,2,3,…,7, so as to ensure the stability of weight adjustment in the subsequent learning process. In addition, the current health status indicator includes but is not limited to the current load rate and / or the current remaining life, wherein the current load rate η and the current remaining life RUL are calculated according to the following formulas:

[0066]

[0067] Where, P Ture Indicates the current operating power of the corresponding device, P Rated Indicates the rated power of the corresponding equipment, L design Indicates the design life of the corresponding equipment, represents the moment, T represents the last moment in the next time step, Δt represents the sampling time interval, Indicates that the corresponding device is at time The load weighting factor.

[0068] The reinforcement learning execution module is used to determine the current state according to the real-time values of all the parameters in each time step, and then, based on the reward value of the current action, use the Q-learning algorithm to select the next action for intelligent switching and scheduling of multiple heat sources in the hospital from at least one optional action in the current state and execute it. The aforementioned Q-learning algorithm is a model-free reinforcement learning algorithm based on value iteration. It searches for the optimal strategy by continuously updating the action value function (Q value). It is widely used in fields such as robot control, game AI, and autonomous driving. Therefore, it can be applied to this embodiment to use the Q table Q(s,a) to store the estimated rewards of state-action pairs, and to use the following method: Figure 2The process shown continuously selects and executes the next action for intelligent switching and scheduling of multiple heat sources in the hospital from at least one optional action in the current state. In detail, the Q-value update formula in the Q-learning algorithm is as follows:

[0069]

[0070] In the formula, α represents the learning rate, γ represents the discount factor, and r t Represents the current reward, and α′ represents the optimal action in the next state. In addition, the reinforcement learning execution module is respectively connected to the influencing parameter acquisition module, the state space construction module, the action space construction module and the reward function calculation module.

[0071] Considering that actions that conflict with the fault state need to be excluded before action selection (that is, before action selection, all actions that conflict with the current fault or maintenance state of the equipment should be filtered out first) in order to ensure the system safety when the subsequent selected actions are executed, preferably, based on the reward value of the current action, the Q-learning algorithm is used to select the next action for intelligent switching and scheduling of multiple heat sources in the hospital from at least one optional action in the current state, including but not limited to the following steps: first, according to the current state, determine the current faulty heat source equipment in the hospital heating system; then, based on the reward value of the current action, the Q-learning algorithm is used to select the next action for intelligent switching and scheduling of multiple heat sources in the hospital from at least one optional action in the current state that prohibits the startup of the current faulty heat source equipment. In addition, the action selection strategy in the Q-learning algorithm preferably adopts the ε-greedy strategy, that is, specifically as follows: exploration with probability ε: randomly select an action; exploitation with probability 1-ε: select the action with the largest current Q value. For example: Assume that in the current state, there are three optional actions A1, A2 and A3, with corresponding Q values of 10, 6 and 8 respectively, and ε = 0.1, then there is a 90% probability of selecting action A1 with the largest Q value, and a 10% probability of randomly selecting one of the three actions.

[0072] Therefore, based on the aforementioned artificial intelligence body for intelligent switching and scheduling of multiple heat sources in hospitals, a new solution for intelligent switching and scheduling of multiple heat sources in hospitals is provided with "environmental state perception-strategy intelligent decision-making-real-time control execution" as the core closed loop, which includes an influencing parameter acquisition module, a state space construction module, an action space construction module, a reward function calculation module and a reinforcement learning execution module. Through their mutual collaboration, the static capacity parameters and dynamic operation data of the heat source can be comprehensively considered, and combined with load prediction and reinforcement learning optimization mechanism, the heat source can be started and stopped on demand, dynamically switched and adaptively distributed in the load, thereby improving the energy saving, responsiveness and operation safety of the overall heating system, and then realizing the optimal switching and load scheduling of the multi-heat source heating system, and significantly improving the intelligence level and operation efficiency of the system, which is convenient for practical application and promotion.

[0073] Finally, it should be noted that the above description is only a preferred embodiment of the present invention and is not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention shall be included in the scope of protection of the present invention.

Claims

1. An artificial intelligence agent for intelligent switching and scheduling of multiple heat sources in a hospital, characterized by: It includes an impact parameter acquisition module, a state space construction module, an action space construction module, a reward function calculation module and a reinforcement learning execution module; The influencing parameter acquisition module is used to collect all parameters that can affect the hospital's multi-heat source intelligent switching and scheduling decisions; The state space construction module is used to construct the state space based on all the parameters; The action space construction module is used to construct, for each state in the state space, at least one optional action in the corresponding state and used for performing intelligent switching and scheduling of multiple heat sources in the hospital; The reward function calculation module is used to calculate an objective function containing multiple optimization objectives after selecting any optional action in any state belonging to the state space, based on various monitoring indicators of the hospital heating system in the next time step, and use the calculation result as the reward value corresponding to any state and any optional action, wherein the multiple optimization objectives include minimizing the total energy consumption of the hospital heating system and shortening the adjustment response time when the environment / load changes, ensuring the comfort of various areas in the hospital and / or equipment health management, and the equipment health management includes balancing the operating time of various heat source equipment in the hospital heating system, reducing the failure rate of each heat source equipment, extending the service life of each heat source equipment and / or preventing any heat source equipment in the hospital heating system from overloading; The reinforcement learning execution module is used to determine the current state according to the real-time values of all the parameters in each time step, and then, based on the reward value of the current action, use the Q-learning algorithm to select and execute the next action for intelligent switching and scheduling of multiple heat sources in the hospital from at least one optional action in the current state.

2. The artificial intelligence agent according to claim 1, characterized in that All the parameters include indoor environmental parameters of each area in the hospital, outdoor environmental parameters of the area outside the hospital, static parameters of each heat source equipment in the hospital heating system, dynamic parameters of each heat source equipment in the hospital heating system, heating load demand parameters and / or time factor parameters of each area in the hospital.

3. The artificial intelligence agent according to claim 1, characterized in that The influencing parameter acquisition module includes an environmental parameter acquisition unit, a static parameter acquisition unit, a dynamic parameter acquisition unit and / or a load demand acquisition unit; The environmental parameter acquisition unit is used to periodically acquire indoor environmental parameters of various areas within the hospital and outdoor environmental parameters of areas outside the hospital, wherein the indoor environmental parameters include indoor temperature, humidity and air flow rate, and the outdoor environmental parameters include outdoor temperature, humidity and wind speed; The static parameter acquisition unit is used to acquire the static parameters of each heat source device in the hospital heating system, wherein the static parameters include the rated heating power and energy efficiency coefficient of the corresponding device; The dynamic parameter acquisition unit is used to periodically acquire the dynamic parameters of each heat source device in the hospital heating system, wherein the dynamic parameters include the real-time heating power, outlet / return water temperature, outlet / return water flow rate, switch status, operation mode, fault alarm status and operation time of the corresponding device; The load demand acquisition unit is used to obtain heating load demand parameters of each area in the hospital based on historical load data and real-time load monitoring data of each area in the hospital, wherein the heating load demand parameters include current heating load and future heating load forecast values.

4. The artificial intelligence agent according to claim 1, wherein: Each state in the state space contains the outdoor temperature T out , the indoor temperature of the i-th area in the hospital and indoor humidity The current total heating load of the hospital is L current And the future total heating load forecast value L pred , the current energy efficiency P of the jth heat source equipment in the hospital heating system (j) and system control parameters S equip , where i and j represent positive integers respectively, and the system control parameters include equipment start / stop status values and / or regional temperature setting values.

5. The artificial intelligence agent according to claim 1, characterized in that The optional actions include the start and stop combination of all heat source equipment in the hospital heating system, the heating load distribution ratio, the water supply temperature setting target value and the operating frequency adjustment value.

6. The artificial intelligence agent according to claim 5, characterized in that The static parameters of each heat source device in the hospital heating system include the defined roles of the corresponding devices, wherein the defined roles are divided into a main heat source role, an auxiliary heat source role, and a backup heat source role. The main heat source role refers to the device role that is started when the hospital heating system is started and assumes the basic load task. The auxiliary heat source role refers to the device role that is started in parallel to ensure the heating capacity of the system when the main heat source device cannot independently meet the load. The backup heat source role refers to the device role that automatically starts when the main / auxiliary heat source device fails to switch the failed device and ensure uninterrupted operation of the system. The start-stop combination meets the following constraints: If the first heat source device having the role of the main heat source is running, the second heat source device having the role of the auxiliary heat source is in a standby state and is started only when the first heat source device cannot meet the load independently; If the third heat source device having the main heat source role / the auxiliary heat source role fails, the fourth heat source device having the backup heat source role starts to operate, and the third heat source device stops operating.

7. The artificial intelligence agent according to claim 1, characterized in that The objective function F is expressed as follows: Where w1, w2, w3, w4, w5, w6 and w7 represent the preset weight coefficients, E t represents the total energy consumption of the hospital heating system in the next time step, C t represents the operating cost of the hospital heating system in the next time step, N represents the total number of areas in the hospital, i represents a positive integer, represents the indoor temperature of the i-th area in the hospital, represents the target value of the indoor temperature of the i-th area, N switch Indicates the number of heat source switching times per unit time, represents the indoor humidity of the i-th area, represents the target value of indoor humidity in the i-th area, D response represents the time delay for the hospital heating system to respond to the indoor temperature and humidity target setting value or heating load change, M represents the total number of heat source devices in the hospital heating system, j represents a positive integer, Represents the current health status indicator of the j-th heat source equipment in the hospital heating system.

8. The artificial intelligence agent according to claim 7, characterized in that: The current health status indicator includes the current load rate and / or the current remaining life, wherein the current load rate η and the current remaining life RUL are calculated according to the following formulas: Where, P Ture Indicates the current operating power of the corresponding device, P Rated Indicates the rated power of the corresponding equipment, L design Indicates the design life of the corresponding equipment, represents the moment, T represents the last moment in the next time step, Δt represents the sampling time interval, Indicates that the corresponding device is at time The load weighting factor.

9. The artificial intelligence agent according to claim 1, characterized in that Based on the reward value of the current action, a Q-learning algorithm is used to select the next action for intelligent switching and scheduling of multiple heat sources in the hospital from at least one optional action in the current state, including: Determining a currently faulty heat source device in the hospital heating system according to the current state; Based on the reward value of the current action, the Q-learning algorithm is used to select the next action for intelligent switching and scheduling of multiple heat sources in the hospital from at least one optional action in the current state and prohibiting the startup of the currently faulty heat source equipment.

10. The artificial intelligence agent according to claim 1, characterized in that The action selection strategy in the Q-learning algorithm adopts the ε-greedy strategy.

Images