A method for generating training strategies for intelligent agents in a time-series-driven dynamic simulation heat pump system

Abstract

This invention belongs to the field of building energy management technology, and relates to a method for generating a training strategy for a heat pump system intelligent agent driven by dynamic simulation time sequence. By acquiring the hourly heat load supply of fan coil units throughout the year, the water supply temperature of fan coil units, the number of heat pumps in operation, the number of hot water pumps in operation, the water flow rate of hot water pumps, and the operating frequency of hot water pumps, a complete hourly training sample of the heat pump system intelligent agent is established. The intelligent agent of the heat pump system is then trained using the training sample to achieve efficient and stable operation of the building heat pump system.

Description

Technical Field

[0001] This invention belongs to the field of building energy management technology, specifically relating to a method for generating intelligent agent training strategies for heat pump systems driven by dynamic simulation time sequence. Background Technology

[0002] Currently, agent training mainly relies on offline datasets constructed from historical operational data, using reward functions in reinforcement learning to guide policy optimization. However, this approach has significant drawbacks: real-world data struggles to cover complex operating conditions such as extreme weather and equipment malfunctions, and the setting of reward functions is highly dependent on human experience, making the trained policies prone to getting trapped in local optima and unable to meet the global control requirements in complex dynamic environments.

[0003] To improve learning efficiency, existing technologies often incorporate expert knowledge pre-training models, such as setting device start / stop thresholds and temperature fluctuation ranges. However, existing technologies also suffer from two major problems: First, rule bases are mostly based on independent device control logic, failing to consider the balance between energy consumption and comfort when multiple devices are coupled; second, the dynamic correlation between environmental parameters (such as outdoor temperature and humidity) and the time dimension (such as hourly load changes) is not effectively decoupled, making it difficult for preset rules to adapt to the real-time changing thermal inertia characteristics of buildings and user behavior patterns. Taking intelligent agent training under winter heating conditions as an example, traditional heat pump system operation control methods are crude, resulting in energy waste and low equipment operating efficiency. Specifically, they cannot accurately adjust the operating parameters of heat pumps and hot water pumps according to the actual heat load, leading to high energy consumption operation of equipment at low loads; at the same time, frequent switching of the number of devices not only affects the lifespan of the units but also increases maintenance costs. These problems essentially reflect the inadequacy of existing strategy generation methods in dealing with dynamic and complex scenarios. Summary of the Invention

[0004] The purpose of this invention is to provide a method for generating training strategies for intelligent agents in a heat pump system driven by dynamic simulation and time sequence. This method enables refined control of building heating systems and provides accurate decision-making strategies for agent training. This invention constructs a fine-grained knowledge model through time-series simulation, deeply explores the dynamic correlation between environmental parameters and the time dimension, and fully considers the coupling characteristics of multiple devices. It provides a decision-making benchmark for agent training that combines physical realism and temporal continuity, thereby optimizing the agent training strategy and improving system operating efficiency and stability.

[0005] This invention is achieved through the following technical solution:

[0006] This paper proposes a dynamic simulation-driven intelligent agent training strategy generation method for heat pump systems, comprising the following steps: acquiring hourly heat load data for the entire building throughout the year; acquiring hourly water flow rate for each fan coil unit throughout the year based on the supply and return water temperature difference and the hourly heat load data for the entire building throughout the year; acquiring the hourly actual heat supply for each fan coil unit at each supply water temperature based on the supply and return water temperature difference, the hourly water flow rate for each fan coil unit throughout the year, and multiple preset supply water temperatures; summarizing the actual heat supply of multiple fan coil units at the same supply water temperature and during the same time period; and determining the hourly actual heat supply for the entire building throughout the year based on the hourly heat load data for the entire building throughout the year and the summed actual heat supply. The system calculates the following: water supply temperature; the number of heat pumps in operation hourly throughout the year based on the building's heat load data and the actual heating capacity of each heat pump hourly throughout the year; the number of hot water pumps in operation hourly throughout the year based on the number of heat pumps in operation hourly throughout the year and the engineering ratio of heat pumps to hot water pumps hourly throughout the year; the water flow rate of each hot water pump hourly throughout the year based on the supply and return water temperature difference, the building's heat load data hourly throughout the year, and the number of hot water pumps in operation hourly throughout the year; the operating frequency of each hot water pump hourly throughout the year based on the water flow rate of each hot water pump hourly throughout the year; and a training strategy for the intelligent agent is generated by combining the actual water supply temperature, the number of heat pumps in operation, the number of hot water pumps, and the operating frequency of each hot water pump hourly throughout the year.

[0007] Compared with existing technologies, this invention has the following advantages and beneficial effects: By refining the agent control strategy, a complete set of hourly refined control strategies for heat pump feedwater temperature, number of heat pumps, number of hot water pumps, and hot water pump frequency is formed. The hourly data sequences generated by these strategies provide a large number of real and effective data samples and clear target strategies for agent training, effectively optimizing the agent training process, improving system operating efficiency, reducing energy consumption, and ensuring stable equipment operation. Attached Figure Description

[0008] The accompanying drawings, which are included to provide a further understanding of embodiments of the invention and form part of this application, do not constitute a limitation thereof. In the drawings:

[0009] Figure 1 The flowchart shows the method for generating a training strategy for a heat pump system intelligent agent driven by dynamic simulation time sequence, as provided in Embodiment 1 of the present invention. Detailed Implementation

[0010] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to embodiments. The illustrative embodiments and descriptions of this invention are for illustrative purposes only and are not intended to limit the invention. The embodiments described below are some, but not all, of the embodiments of this invention. All other embodiments obtained by those skilled in the art based on the embodiments of this invention without creative effort are within the scope of protection of this invention.

[0011] In the following description, numerous specific details are set forth to provide a thorough understanding of the invention. However, it will be apparent to those skilled in the art that these specific details are not necessary to practice the invention. In other embodiments, well-known structures, materials, or methods are not specifically described to avoid obscuring the invention. Unless otherwise specified, the materials, instruments, and reagents used in the following embodiments are commercially available. Unless otherwise specified, the techniques used in the embodiments are conventional methods well known to those skilled in the art.

[0012] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of that feature. In the description of this application, "multiple" means two or more, unless otherwise explicitly specified.

[0013] Example 1: A method for generating intelligent agent training strategies for a dynamically simulated, time-driven heat pump system, including... Figure 1 The following steps are shown:

[0014] Step 1: Establish a dynamic mapping between actual heat load demand and coil water flow rate.

[0015] The purpose of this step is to provide the agent with training data samples that reflect the actual heat load demand of the entire building, enabling the agent to learn the changing patterns of the fan coil unit's water flow. This step is implemented by: acquiring hourly heat load data for the entire building throughout the year; and based on the supply and return water temperature difference and the hourly heat load data for the entire building throughout the year, acquiring the hourly water flow of each fan coil unit throughout the year.

[0016] The hourly heat load data for the entire building throughout the year can be obtained using the following methods: Collect the building's structural parameters, internal air conditioning system parameters, and local meteorological data for the entire year. Based on the structural parameters, create a 3D model of the entire building, assign parameter values ​​to the building envelope, and set the internal heat source dissipation mode. Establish an air conditioning system model for the entire building based on the air conditioning system parameters. Set the boundary conditions for the building energy consumption simulation based on local meteorological data. Set the time step, iteration accuracy, and output format for the building energy consumption simulation. Input the 3D structural model of the entire building, envelope parameters, internal heat source dissipation mode, and air conditioning system model, the local climate data for the entire year, and the boundary conditions, time step, iteration accuracy, and output format for the building energy consumption simulation into the building energy consumption simulation software. Run the building energy consumption simulation software to obtain and output the hourly heat load data for the entire building throughout the year.

[0017] Furthermore, obtaining the hourly water flow rate for each fan coil unit throughout the year includes the following steps:

[0018] 1. Set the supply and return water temperature difference for the fan coil unit.

[0019] In this embodiment, the supply and return water temperature difference of the fan coil unit is fixed at 5°C.

[0020] 2. For the hourly heat load data of the entire building throughout the year corresponding to each time period, execute steps A1 to A2.

[0021] Step A1: Distribute the heat load data of the entire building to each fan coil unit to obtain the heat load data of each fan coil unit.

[0022] Step A2: Calculate the water flow rate of each fan coil unit based on the supply and return water temperature difference and heat load data.

[0023] According to the heat transfer formula Q=CRΔT Calculate the water flow rate of the fan coil unit, where Q represents the heat load. C This indicates the specific heat capacity of water. R Indicates water flow rate. ΔT This indicates the temperature difference between the supply and return water. For example, according to step A1, if the calculated heat load is 3560... W The specific heat capacity of water is 4.1868 (kJ / kg·°C). Based on the determined supply and return water temperature difference of 5°C, the water flow rate of the fan coil unit is R = 3560. W ÷4.1868(kJ / kg·°C)÷5℃≈170m 3 / h.

[0024] Step 2: Calculate the actual heat output of the fan coil unit at different water supply temperatures and at different times.

[0025] The correspondence between these actual heating data and different water supply temperatures constitutes a knowledge system about heating performance in the training of the agent, helping the agent understand the impact of water supply temperature on the heating capacity of the fan coil unit, thereby optimizing decision-making strategies during training. This step is implemented as follows: based on the supply and return water temperature difference, the hourly water flow rate of each fan coil unit throughout the year, and multiple preset water supply temperatures, obtain the actual hourly heating capacity of each fan coil unit at each water supply temperature throughout the year.

[0026] Specifically, the software provided by the fan coil unit manufacturer is used to calculate the actual heat output of the fan coil units in the heating system. The specific method is as follows:

[0027] First, set the input parameters in the software:

[0028] (1) Supply and return water temperature difference. The supply and return water temperature difference is the same as that set in step 1, and is fixed at 5℃.

[0029] (2) Annual hourly water flow rate of the fan coil unit. That is, the annual hourly water flow rate of the fan coil unit obtained in step 1 above.

[0030] (3) Water supply temperature value sequence. This water supply temperature value sequence includes multiple water supply temperature values ​​(such as 40℃, 41℃ and 42℃) that are lower than the factory design water supply temperature (45℃) of the fan coil unit.

[0031] Then, the set input parameters are imported into the software provided by the fan coil unit manufacturer, which automatically calculates the actual hourly heat load supply for each fan coil unit at each water supply temperature throughout the year. After organizing this heat supply data with the corresponding water supply temperature and time information, it is input into the intelligent agent training model, allowing the agent to learn the relationship between water supply temperature and heat supply, so as to make better decisions in actual operation.

[0032] Step 3: Check the water supply temperature.

[0033] First, the actual heat output of multiple fan coil units is summarized under the same water supply temperature and at the same time period.

[0034] For example, with a water supply temperature of 41℃, the actual heat load supply of each fan coil unit corresponds to the period from 0:00 to 1:00 on January 1st. This step is to add up the actual heat load supply of all fan coil units corresponding to the period from 0:00 to 1:00 on January 1st when the water supply temperature is 41℃.

[0035] Then, based on the hourly heat load data of the entire building throughout the year and the summaries of multiple actual heat supplies, the actual water supply temperature for each hour throughout the year is determined.

[0036] The purpose of this step is to enable the agent to learn how to select the appropriate water supply temperature based on heat load requirements in order to achieve rational energy utilization.

[0037] The method for determining the actual hourly water supply temperature throughout the year is as follows:

[0038] 1. Subtract the sum of the actual heat supply for each building from the heat load data for the corresponding time period.

[0039] For example, the difference between the actual heat supply at 41℃ and the heat load data of the entire building at the first hour is calculated to obtain the difference for the first hour at 41℃. The calculation method for each hour at the other temperatures is the same, ultimately yielding the difference for each hour at each water supply temperature.

[0040] 2. Filter out the differences that are greater than zero from the differences corresponding to each time period.

[0041] If, at a given water supply temperature, the actual heat output of the fan coil units can cover the building's heat load demand at all times, then that water supply temperature is determined to be the true water supply temperature the system should provide at that time. Therefore, from the multiple heat load differences corresponding to each time period, the differences greater than zero are selected. For example, if the first hour corresponds to differences at 40℃, 41℃, and 42℃, then the differences greater than zero are selected from these three heat load differences at 40℃, 41℃, and 42℃.

[0042] 3. For each time period, the water supply temperature corresponding to the selected difference is determined as the actual water supply temperature.

[0043] Step 4: Determine the number of heat pumps to be turned on.

[0044] The purpose of this step is to provide the agent with learning samples for heat pump control strategies, enabling the agent to learn how to rationally control the number of heat pumps under different heat loads and operating conditions. This step is implemented by obtaining the number of heat pumps activated hourly throughout the year based on the building's hourly heat load data and the actual heating capacity of each individual heat pump.

[0045] Before proceeding, it is necessary to first obtain the actual heating capacity of a single heat pump. The specific method is to use the defrost correction model, the regional humidity correction model, and the ambient temperature correction model corresponding to the water supply temperature at each time period to correct the rated heating capacity of a single heat pump.

[0046] 1. Correct using the defrost correction model

[0047] First, collect defrosting data of the heat pump under different operating conditions, including defrosting frequency, defrosting time, and heat loss during defrosting. This data can be obtained through experimental testing or monitoring during actual operation.

[0048] Next, the main factors affecting the defrosting process are analyzed, such as outdoor temperature, humidity, wind speed, and heat pump operating time. Generally, the lower the outdoor temperature and the higher the humidity, the higher the defrosting frequency and the greater the heat loss.

[0049] Next, based on the collected data and analyzed influencing factors, a defrosting correction model is established. A functional relationship between the defrosting-related correction coefficients and each influencing factor can be obtained through numerical fitting. For example, the correction coefficients... K d It can be represented as K d =f ( T 0 ,H 0 ,t ),in, T 0 Outdoor temperature H 0 Outdoor humidity, t This refers to the continuous operating time of the heat pump.

[0050] Finally, parameters such as outdoor temperature, humidity, and heat pump operating time are monitored in real time, and these parameters are substituted into the defrosting correction model to calculate the defrosting correction coefficient. K d Multiply this defrosting correction factor by the rated heating capacity of the heat pump. Q e The heat output after defrosting correction is obtained. Q d = K d Q e .

[0051] 2. Correct using a regional humidity correction model

[0052] First, collect humidity data from different regions. This can be done by obtaining long-term average humidity data from meteorological departments or by utilizing specialized meteorological databases. Simultaneously, determine the specific geographical location of the heat pump.

[0053] Then, the variation pattern of the heat pump's heating performance under different humidity conditions was obtained. Generally, when the humidity is high, the evaporator surface of the heat pump is more prone to frost formation, which affects the heat transfer effect and reduces the heating capacity.

[0054] Next, based on the analysis of the impact of humidity on heat pump performance, a regional humidity correction model is established. For example, a region can be divided into several humidity zones based on the average humidity of different regions, with each zone corresponding to a correction coefficient. K h Alternatively, a continuous correction function can be established, such as...K h = a + bKH a ,in, H a The local average annual humidity. a and b These are constants determined through data analysis.

[0055] Finally, the humidity data of the region where the heat pump is located is substituted into the regional humidity correction model to calculate the regional humidity correction coefficient. K h Then, multiply this coefficient by the defrosting correction for heating capacity. Q d The heating capacity is obtained after regional humidity correction. Q h = K h Q d .

[0056] 3. Using an ambient temperature correction model

[0057] For each time period determined in step 3, perform the following steps:

[0058] First, during the operation of the heat pump, the outdoor ambient temperature is monitored in real time. T 0 A high-precision temperature sensor can be installed near the outdoor unit of the heat pump to obtain accurate ambient temperature data.

[0059] Then, based on the performance characteristics of the heat pump and experimental data, a correction function for the heating capacity based on ambient temperature is determined. Generally, the heating capacity of a heat pump decreases as the ambient temperature decreases. Common correction functions include linear, quadratic, or exponential functions. For example, the correction function can be expressed as... K t = c + dT o + e ( T o ) 2 ,in c , d and e These are the polynomial coefficients determined experimentally.

[0060] Finally, the real-time monitored ambient temperature T o Substitute into the ambient temperature correction function to calculate the ambient temperature correction coefficient. K tUse this ambient temperature correction factor. K t Heating capacity multiplied by regional humidity correction Q h The final corrected heating capacity was obtained. Q f = K t Q h .

[0061] Furthermore, to avoid frequent switching of the number of heat pumps in operation and to ensure the service life of the heat pump units, this embodiment implements a protection mechanism for the hourly calculated number of heat pumps in operation after obtaining the number of heat pumps in operation. This mechanism limits the frequency of changes in the number of heat pumps in operation. For example, the number of heat pumps in operation is limited to changing only once every 3 hours. The hourly data on the changes in the number of heat pumps in operation is input into the agent's training model, enabling the agent to learn strategies for rationally controlling the number of heat pumps under different operating conditions, thus avoiding damage to the equipment due to frequent switching.

[0062] Step 5: Determine the number of variable frequency hot water pumps and their frequency.

[0063] The purpose of this step is to provide the agent with learning content on hot water pump control strategies, enabling it to learn how to adjust the hot water pump's operating status according to system requirements and achieve efficient collaborative work with the heat pump. This step is implemented as follows: Based on the number of heat pumps operating hourly throughout the year and the engineering ratio of heat pumps to hot water pumps, obtain the number of hot water pumps operating hourly throughout the year; based on the supply and return water temperature difference, the building's hourly heat load data, and the number of hot water pumps operating hourly throughout the year, obtain the hourly water flow rate of each hot water pump; based on the hourly water flow rate of each hot water pump, obtain the hourly operating frequency of each hot water pump throughout the year; and limit the hourly operating frequency of each hot water pump throughout the year.

[0064] Specifically, based on the actual ratio of heat pumps and hot water pumps in the building, and according to the number of heat pumps turned on hourly throughout the year determined in step 4, the corresponding number of hot water pumps is calculated hourly throughout the year to ensure the coordinated operation of the hot water pumps and heat pumps.

[0065] The specific method for obtaining the hourly water flow rate of each hot water pump throughout the year is as follows:

[0066] 1. Set the supply and return water temperature difference for the hot water pump.

[0067] The supply and return water temperature difference in this step is the same as the supply and return water temperature difference set in step 1, both being a fixed value of 5℃.

[0068] 2. For the heat load data of the entire building corresponding to each time period, execute steps B1 to B2.

[0069] Step B1: Distribute the heat load data of the entire building to each hot water pump to obtain the heat load data of each hot water pump.

[0070] For a detailed implementation of this step, please refer to step A1.

[0071] Step B2: Calculate the water flow rate of each hot water pump based on the supply and return water temperature difference and the heat load data of each hot water pump.

[0072] For a detailed implementation of this step, please refer to step A2.

[0073] Furthermore, the operating frequency of the hot water pump is calculated based on the water flow rate of the hot water pump calculated above, and the corresponding relationship between the water flow rate and the operating frequency recorded on the rated nameplate of the hot water pump, combined with the calculation formula "operating frequency = actual water flow rate of hot water pump ÷ rated water flow rate of hot water pump × rated operating frequency of hot water pump".

[0074] Furthermore, the efficiency of a hot water pump decreases significantly when its operating frequency is below 30Hz. To improve efficiency, the hourly operating frequency of the heat pump is limited to a minimum of 30Hz, and the final hot water pump operating frequency is determined within the range of 30Hz to the rated maximum operating frequency. The purpose of this step is to guide the agent to prioritize decisions that keep the hot water pump in its high-efficiency operating range when learning the hot water pump's control strategy, ensuring that the hot water pump operates within this efficient range.

[0075] Step 6: Generate the agent training strategy.

[0076] By combining the actual water supply temperature, the number of heat pumps in operation, the number of hot water pumps, and the operating frequency of each hot water pump throughout the year, an intelligent agent training strategy is generated.

[0077] It should be understood that the terms "system," "device," "unit," and / or "module" as used in this specification are a method of distinguishing different components, elements, parts, sections, or assemblies at different levels. However, if other terms can achieve the same purpose, they may be replaced by other expressions.

[0078] As indicated in this specification and claims, unless the context clearly indicates otherwise, the words "a," "an," "an," and / or "the" do not specifically refer to the singular and may also include the plural. Generally speaking, the terms "comprising" and "including" only indicate the inclusion of expressly identified steps and elements, which do not constitute an exclusive list, and the method or apparatus may also include other steps or elements.

[0079] The specific embodiments described above further illustrate the purpose, technical solution, and beneficial effects of the present invention. It should be understood that the above description is only a specific embodiment of the present invention and is not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

[0080] It should be noted that the structures, proportions, sizes, etc., illustrated in the accompanying drawings are merely for illustrative purposes to aid those skilled in the art and are not intended to limit the scope of the invention. Therefore, they have no substantial technical significance. Any modifications to the structure, changes in proportions, or adjustments to size, without affecting the effectiveness and purpose of the invention, should still fall within the scope of the disclosed technical content. Furthermore, terms such as "upper," "lower," "left," "right," and "middle" used in this specification are merely for clarity and not intended to limit the scope of the invention. Changes or adjustments to their relative relationships, without substantially altering the technical content, should also be considered within the scope of the invention.

Claims

1. A method for generating training strategies for intelligent agents in a heat pump system driven by dynamic simulation time sequence, characterized in that, Includes the following steps: Obtain hourly heat load data for the entire building throughout the year; based on the supply and return water temperature difference and the hourly heat load data for the entire building throughout the year, obtain hourly water flow rate for each fan coil unit throughout the year; Based on the supply and return water temperature difference, the annual hourly water flow of each fan coil unit, and multiple preset water supply temperatures, the actual annual hourly heat supply of each fan coil unit at each water supply temperature is obtained. Under the same water supply temperature and at the same time, the actual heat supply of multiple fan coil units is summarized; based on the hourly heat load data of the entire building throughout the year and the summarized actual heat supply data, the actual water supply temperature for each hour throughout the year is determined. The rated heating capacity of a single heat pump is corrected by using a defrosting correction model, a regional humidity correction model, and an ambient temperature correction model corresponding to the actual water supply temperature for each time period, so as to obtain the actual heating capacity of a single heat pump. The number of heat pumps turned on throughout the year is obtained based on the hourly heat load data of the entire building and the actual heating capacity of a single heat pump. The number of hot water pumps is obtained hourly throughout the year based on the number of heat pumps in operation and the engineering ratio of heat pumps to hot water pumps. The water flow rate of each hot water pump is obtained hourly throughout the year based on the supply and return water temperature difference, the heat load data of the entire building hourly throughout the year, and the number of hot water pumps hourly throughout the year. The operating frequency of each hot water pump is obtained hourly throughout the year based on the water flow rate of each hot water pump. By combining the actual water supply temperature, the number of heat pumps in operation, the number of hot water pumps, and the operating frequency of each hot water pump throughout the year, an intelligent agent training strategy is generated.

2. The method for generating intelligent agent training strategies for a heat pump system driven by dynamic simulation time sequence according to claim 1, characterized in that, To obtain the hourly water flow rate for each fan coil unit throughout the year, the following steps are included: For the hourly heat load data of the entire building for each time period, execute steps A1 to A2; wherein, step A1: distribute the heat load data of the entire building to each fan coil unit to obtain the heat load data of each fan coil unit; step A2: calculate the water flow rate of each fan coil unit based on the supply and return water temperature difference and heat load data.

3. The method for generating intelligent agent training strategies for a heat pump system driven by dynamic simulation time sequence according to claim 1, characterized in that, Determining the actual hourly water supply temperature throughout the year includes the following steps: The difference between each actual heat supply and the heat load data of the entire building for the corresponding time period is calculated. Filter out the differences that are greater than zero from the differences corresponding to each time period; For each time period: the water supply temperature corresponding to the selected difference is determined as the actual water supply temperature.

4. The method for generating intelligent agent training strategies for a heat pump system driven by dynamic simulation time sequence according to claim 1, characterized in that, After obtaining the number of heat pumps in operation hourly throughout the year, the following steps are also included: limiting the frequency of change in the number of heat pumps in operation.

5. The method for generating intelligent agent training strategies for a heat pump system driven by dynamic simulation time sequence according to claim 1, characterized in that, To obtain the hourly water flow rate for each hot water pump throughout the year, the following steps are required: For the heat load data of the entire building corresponding to each time period, execute steps B1 to B2; wherein, step B1: distribute the heat load data of the entire building to each hot water pump to obtain the heat load data of each hot water pump; step B2: calculate the water flow rate of each hot water pump based on the supply and return water temperature difference and the heat load data of each hot water pump.

6. The method for generating intelligent agent training strategies for a heat pump system driven by dynamic simulation time sequence according to claim 1, characterized in that, After obtaining the hourly operating frequency of each hot water pump throughout the year, the following steps are also included: The operating frequency of each hot water pump is limited throughout the year and hourly. Add the limited running frequency to the agent training strategy.

7. The method for generating intelligent agent training strategies for a heat pump system driven by dynamic simulation time sequence according to claim 6, characterized in that, The operating frequency range is [30Hz, rated maximum frequency].

Images