Control method and device for a solar ground source heat pump system

Abstract

The application provides a control method and device of a solar ground source heat pump system, which can be applied to the field of building energy saving and intelligent control technology. The method comprises the following steps: performing nonlinear conversion on state information of the solar ground source heat pump system to obtain a plurality of control action information; controlling an execution device in the solar ground source heat pump system based on target action information, and updating or maintaining the running state of the solar ground source heat pump system; wherein the control agent is obtained by training in the following manner: an initial agent performs a current action in a virtual environment, and obtains reward information and post-state information of the current action based on a reward function; the intermediate agent is updated based on interaction information obtained by interaction between the intermediate agent and the real environment, so that the control agent meeting the preset condition is obtained.

Description

Technical Field

[0001] This invention relates to the field of building energy conservation and intelligent control technology, and more specifically to a control method and device for a solar ground source heat pump system. Background Technology

[0002] A solar ground source heat pump system is a highly efficient and renewable energy utilization system. It combines solar collectors and a ground source heat pump, utilizing the heat storage characteristics of underground soil to provide heating in winter, cooling in summer, and a year-round supply of domestic hot water. The energy sources (solar energy, soil energy, electricity) and power supply of solar heat pump systems are complex. Related technologies primarily rely on preset rules and model predictive control to achieve control of the solar ground source heat pump system.

[0003] However, control strategies based on preset rules are difficult to cope with complex and ever-changing external environments and have poor adaptability to different environments; model predictive control relies on accurate mathematical models and focuses too much on energy efficiency control, making it difficult to achieve a balance between equipment safety and energy efficiency. The accuracy of mathematical models is limited by the external environment and the accuracy of load prediction, and the construction cost is high; traditional intelligent agents are trained in static or fixed environments, which leads to performance degradation and difficulty in coping with sudden changes in dynamic and non-stationary real-world working conditions. Summary of the Invention

[0004] In view of the above problems, the present invention provides a control method and apparatus for a solar ground source heat pump system.

[0005] According to a first aspect of the present invention, a control method for a solar ground source heat pump system is provided, applied to a control agent of the solar ground source heat pump system, comprising: performing a nonlinear transformation on the state information of the solar ground source heat pump system to obtain multiple control action information, wherein the state information is determined by environmental information, system state information, and equipment state information of the solar ground source heat pump system; controlling the execution device in the solar ground source heat pump system based on the target action information to update or maintain the operating state of the solar ground source heat pump system, wherein the target action information is action information among the multiple control action information that satisfies preset constraints, and the preset constraints include at least one of constraint verification information and constraint functions; wherein the aforementioned... The control agent is trained as follows: the initial agent performs the current action in a virtual environment and obtains the reward information and subsequent state information of the current action based on the reward function. The virtual environment is constructed from the initial information, the heat collection model, the heat transfer model, and the prediction model. The reward function is obtained by integrating power consumption, temperature difference information, soil thermal balance information, the smoothness of the action of the execution device, and peak and off-peak electricity price information. Based on the interaction information obtained by the intermediate agent and the real environment, the intermediate agent is updated to obtain a control agent that meets the preset conditions. The intermediate agent is obtained by updating the initial agent using the current state information, the current action, the reward information, and the subsequent state information.

[0006] A second aspect of the present invention provides a control device for a solar ground source heat pump system, comprising: a conversion module for performing nonlinear conversion on the state information of the solar ground source heat pump system to obtain multiple control action information, wherein the state information is determined by environmental information, system state information, and equipment state information of the solar ground source heat pump system; and an update module for controlling the execution equipment in the solar ground source heat pump system based on target action information to update or maintain the operating state of the solar ground source heat pump system, wherein the target action information is action information among the multiple control action information that satisfies preset constraints, and the preset constraints include at least one of constraint verification information and constraint functions; wherein the control agent... The initial agent is trained as follows: it performs a current action in a virtual environment and obtains the reward information and subsequent state information of the current action based on the reward function. The virtual environment is constructed from initial information, a heat collection model, a heat transfer model, and a prediction model. The reward function is obtained by integrating power consumption, temperature difference information, soil thermal balance information, the smoothness of the action of the execution device, and peak-valley electricity price information. Based on the interaction information obtained by the intermediate agent interacting with the real environment, the intermediate agent is updated to obtain a control agent that meets preset conditions. The intermediate agent is obtained by updating the initial agent using the current state information, the current action, the reward information, and the subsequent state information.

[0007] A third aspect of the present invention provides an electronic device comprising: one or more processors; and a memory for storing one or more computer programs, wherein the one or more processors execute the one or more computer programs to implement the steps of the method described above.

[0008] A fourth aspect of the present invention also provides a computer-readable storage medium having a computer program or instructions stored thereon, wherein the computer program or instructions, when executed by a processor, implement the steps of the above-described method.

[0009] A fifth aspect of the present invention also provides a computer program product, including a computer program or instructions that, when executed by a processor, implement the steps of the above-described method.

[0010] According to embodiments of the present invention, control action information is verified and dynamically corrected in real time through constraint verification information and constraint functions, achieving optimal energy efficiency while strictly adhering to the physical safety boundaries of the equipment. By integrating power consumption, temperature difference information, soil thermal balance information, action smoothness, and peak-valley electricity price information into a multi-objective reward function, equipment safety constraints are simultaneously optimized with operating energy efficiency and economy in a virtual training environment. This effectively avoids the control risks of traditional methods when the model is inaccurate, achieving multi-objective synergy of safety, energy efficiency, and economy. Furthermore, through a two-stage training strategy of virtual environment pre-training and real environment online fine-tuning, the control agent masters the basic control strategy in a virtual environment based on a physical model, and then continuously optimizes it through real system interaction data. This significantly improves the robustness of the control agent under non-stationary operating conditions, effectively responding to the challenges of sudden changes in real scenarios and meeting the control needs of different operating conditions in practical applications. Attached Figure Description

[0011] The above-described features, other objects, and advantages of the present invention will become clearer from the following description of embodiments of the invention with reference to the accompanying drawings, in which:

[0012] Figure 1 The diagram illustrates an application scenario of the control method and apparatus for a solar-powered ground-source water pump system according to an embodiment of the present invention.

[0013] Figure 2 A flowchart of a control method for a solar ground source heat pump system according to an embodiment of the present invention is shown;

[0014] Figure 3 A training flowchart for the control agent in a solar ground source heat pump system according to an embodiment of the present invention is shown;

[0015] Figure 4 A structural block diagram of a control device for a solar ground source heat pump system according to an embodiment of the present invention is shown. Detailed Implementation

[0016] Hereinafter, embodiments of the present invention will be described with reference to the accompanying drawings. However, it should be understood that these descriptions are exemplary only and are not intended to limit the scope of the invention. In the following detailed description, numerous specific details are set forth to provide a thorough understanding of the embodiments of the invention for ease of explanation. However, it will be apparent that one or more embodiments may be practiced without these specific details. Furthermore, descriptions of well-known structures and techniques are omitted in the following description to avoid unnecessarily obscuring the concept of the invention.

[0017] The terminology used herein is for the purpose of describing particular embodiments only and is not intended to limit the invention. The terms “comprising,” “including,” etc., as used herein indicate the presence of the stated features, steps, operations, and / or components, but do not exclude the presence or addition of one or more other features, steps, operations, or components.

[0018] All terms used herein (including technical and scientific terms) have the meanings commonly understood by those skilled in the art, unless otherwise defined. It should be noted that the terms used herein are to be interpreted in a manner consistent with the context of this specification, and not in an idealized or overly rigid way.

[0019] When using expressions such as "at least one of A, B and C", they should generally be interpreted in accordance with the meaning that is commonly understood by those skilled in the art (e.g., "a system having at least one of A, B and C" should include, but is not limited to, a system having A alone, a system having B alone, a system having C alone, a system having A and B, a system having A and C, a system having B and C, and / or a system having A, B and C, etc.).

[0020] A solar ground source heat pump system comprises a solar collector loop, a buried pipe heat exchanger loop, a heat pump unit loop, and a user-side loop, with complex energy sources (solar energy, soil energy, and electricity) and destinations. Furthermore, solar irradiance, ambient temperature, and user load demand exhibit strong randomness and intermittency, representing time-varying disturbances that are difficult to predict accurately. In addition, control objectives often simultaneously include energy saving and consumption reduction (minimizing total system power consumption), comfort assurance (maintaining indoor temperature within the set range), and soil thermal balance (preventing long-term uneven heat intake / discharge from the soil from causing system performance degradation). Trade-offs and conflicts exist among these objectives.

[0021] In related technologies, control strategies based on preset rules are difficult to cope with complex and ever-changing external environments and have poor adaptability to different environments; model predictive control relies on accurate mathematical models and focuses too much on energy efficiency control, making it difficult to achieve a balance between equipment safety and energy efficiency. The accuracy of mathematical models is limited by the external environment and the accuracy of load prediction, and the construction cost is high; traditional intelligent agents are trained in static or fixed environments, resulting in performance degradation and difficulty in coping with sudden changes in dynamic and non-stationary real-world working conditions.

[0022] In view of this, embodiments of the present invention provide a control method for a solar ground source heat pump system, applied to a control agent of the solar ground source heat pump system, comprising: performing a nonlinear transformation on the state information of the solar ground source heat pump system to obtain multiple control action information, wherein the state information is determined by the environmental information, system state information, and equipment state information of the solar ground source heat pump system; controlling the execution device in the solar ground source heat pump system based on the target action information to update or maintain the operating state of the solar ground source heat pump system, wherein the target action information is the action information among the multiple control action information that satisfies preset constraints, and the preset constraints include at least one of constraint verification information and constraint functions; The control agent is trained as follows: the initial agent performs the current action in a virtual environment and obtains the reward information and subsequent state information of the current action based on the reward function. The virtual environment is constructed from the initial information, the heat collection model, the heat transfer model, and the prediction model. The reward function is obtained by integrating power consumption, temperature difference information, soil thermal balance information, the smoothness of the action of the execution equipment, and peak and valley electricity price information. Based on the interaction information obtained by the intermediate agent and the real environment, the intermediate agent is updated to obtain the control agent that meets the preset conditions. The intermediate agent is obtained by updating the initial agent using the current state information, the current action, the reward information, and the subsequent state information.

[0023] According to embodiments of the present invention, control action information is verified and dynamically corrected in real time through constraint verification information and constraint functions, achieving optimal energy efficiency while strictly adhering to the physical safety boundaries of the equipment. By integrating power consumption, temperature difference information, soil thermal balance information, action smoothness, and peak-valley electricity price information into a multi-objective reward function, equipment safety constraints are simultaneously optimized with operating energy efficiency and economy in a virtual training environment. This effectively avoids the control risks of traditional methods when the model is inaccurate, achieving multi-objective synergy of safety, energy efficiency, and economy. Furthermore, through a two-stage training strategy of virtual environment pre-training and real environment online fine-tuning, the control agent masters the basic control strategy in a virtual environment based on a physical model, and then continuously optimizes it through real system interaction data. This significantly improves the robustness of the control agent under non-stationary operating conditions, effectively responding to the challenges of sudden changes in real scenarios and meeting the control needs of different operating conditions in practical applications.

[0024] Figure 1 The diagram illustrates an application scenario of the control method and apparatus for a solar-powered ground source water pump system according to an embodiment of the present invention.

[0025] like Figure 1As shown, the application scenario 100 according to this embodiment may include a heat collection device 101, a user-side loop 102, a water pump 103, a heat exchange device 104, a network 105, and a server 106. The heat collection device 101 may be a solar collector, which can convert solar energy into heat energy to heat the room or transfer the heat energy to the heat exchange device for storage. The user-side loop 102 may be an underfloor heating system installed indoors. The water pump 103 is used to circulate the heat medium in the heat exchange device 104 to the user-side loop 102. The heat exchange device 104 may be a buried pipe heat exchanger, used to absorb heat from the soil into the heat exchange device or release excess heat from the room into the soil.

[0026] The heat collection device 101, water pump 103, heat exchange device 104 and server 106 can communicate through network 105, which can include various connection types, such as wired, wireless communication links or fiber optic cables, etc.

[0027] Server 106 can be a server that provides various services, such as a backend management server that supports websites browsed by users using terminal devices (for example only). The backend management server can analyze and process data such as received user requests, and feed back the processing results (such as web pages, information, or data obtained or generated according to user requests) to the terminal device.

[0028] It should be noted that the control method for the solar ground source water pump system provided in this embodiment of the invention can generally be executed by server 106. Correspondingly, the control device for the solar ground source water pump system provided in this embodiment of the invention can generally be located in server 106. The control method for the solar ground source water pump system provided in this embodiment of the invention can also be executed by a server or server cluster that is different from server 106 and can communicate with the heat collection device 101, water pump 103, heat exchange device 104, and / or server 106. Correspondingly, the control device for the solar ground source water pump system provided in this embodiment of the invention can also be located in a server or server cluster that is different from server 106 and can communicate with the heat collection device 101, water pump 103, heat exchange device 104, and / or server 106.

[0029] It should be understood that Figure 1 The number of solar collectors, water pumps, heat exchangers, networks, and servers shown is merely illustrative. Any number of solar collectors, water pumps, heat exchangers, networks, and servers can be included depending on the implementation requirements.

[0030] The following will be based on Figure 1 The described scene, through Figures 2-3 The control method of the solar ground source heat pump system according to an embodiment of the present invention will be described in detail.

[0031] Figure 2 A flowchart of a control method for a solar ground source heat pump system according to an embodiment of the present invention is shown.

[0032] like Figure 2 As shown, the control method of the solar ground source heat pump system in this embodiment includes operation S210 to operation S220.

[0033] In operation S210, the state information of the solar ground source heat pump system is nonlinearly transformed to obtain multiple control action information.

[0034] The status information is determined by the environmental information, system status information, and equipment status information of the solar ground source heat pump system.

[0035] In embodiments of the present invention, the aforementioned environmental information may include, for example, outdoor air temperature, total solar irradiance, indoor temperature at the previous moment, and indoor temperature change rate; system status information may characterize the operating load of the solar ground source heat pump system, and the aforementioned system status information may include system electrical power; equipment status information may characterize the heat transfer status inside or between equipment in the solar ground source heat pump system, and the aforementioned equipment status information may include, for example, the inlet fluid temperature of the heat collector, the outlet fluid temperature of the heat collector, the water temperature of the buried pipe entering the heat pump unit, the user-side return water temperature, and the user-side supply water temperature.

[0036] The status information can be represented by the following formula (1).

[0037] (1)

[0038] in, This indicates the inlet fluid temperature of the solar collector (in °C). This indicates the outlet fluid temperature of the solar collector (in °C). This indicates the water temperature (in °C) at which the buried pipe enters the heat pump unit. This indicates the return water temperature on the user side (in °C). This indicates the water supply temperature on the user side (in °C). Indicates outdoor air temperature. This represents total solar irradiance (in W / m²). This indicates the mass flow rate of the fluid in the solar collector circuit (unit: kg / s). This indicates the water mass flow rate on the load side (unit: kg / s). This indicates the system's electrical power (in kW). Indicates time information, Indicates whether it is a day or a holiday. This indicates the indoor temperature at the previous moment. This indicates the rate of change of indoor temperature.

[0039] In operation S220, the actuators in the solar ground source heat pump system are controlled based on the target action information to update or maintain the operating status of the solar ground source heat pump system.

[0040] Among them, the target action information is the action information that satisfies the preset constraints among multiple control action information. The preset constraints include at least one of constraint verification information and constraint function.

[0041] In an embodiment of the present invention, the target action information can be an incremental adjustment action. The incremental adjustment action does not directly calculate the control amount at the current moment, but calculates the change of the current adjustment action compared to the previous adjustment action, and then adds the aforementioned change amount to the control amount of the previous adjustment action to obtain the final execution signal, which can avoid the problem of excessively coarse adjustment gears.

[0042] In an embodiment of the present invention, the aforementioned execution device may be a frequency converter or a valve controller. The frequency converter is used to control the execution frequency of the water pump 103, and the valve controller is used to switch the mode of the water pump 103.

[0043] The target action information can be represented by the following formula (2).

[0044]

[0045] (2)

[0046] in, It can control the target action information at time t. It can represent the control increment at time t. This can represent the frequency increment of the solar-powered water pump. This can represent the frequency increment of the ground source side water pump. This can represent the frequency increment of the user-side water pump. It can indicate the mode switching status of the heat pump (a value of 0 indicates that the heat pump is off, a value of 1 indicates that the heat pump is in heating mode, and a value of 2 indicates that the heat pump is in cooling mode).

[0047] In related technologies, control agents often select the control action information that maximizes the reward value as the target action information. However, this approach only focuses on maximizing short-term gains and does not impose constraints on the control amplitude, rate of change, and equipment safety boundaries. This can lead to output exceeding limits or long-term fatigue operation of the system, which in turn can cause system instability, reduced energy efficiency, or even damage to the execution equipment.

[0048] To this end, the present invention introduces preset constraints, wherein the constraint verification information is the set of all criteria used to verify the security of the control action information output by the control agent, and the constraint function is a continuous function that mathematically describes the security boundary of the system, which can be a control barrier function (CBF).

[0049] A control barrier function is a mathematical tool in control theory used to constrain the state of a system in real time and ensure its safe operation. It constructs a continuous "barrier" function to keep the system state within a safe set, thereby avoiding dangerous or infeasible regions.

[0050] The constraint verification information can set upper limits for the amplitude, rate of change, and duration of control action information using conditional statements; it can also be verified by looking up a preset safety table, which records the maximum allowed frequency, opening degree, or power under typical operating conditions. In the process of determining the target action information, each control action information is compared with the value in the preset safety table, and the control action information that exceeds the limit is reduced to the maximum value in the preset safety table.

[0051] In an embodiment of the present invention, the aforementioned control agent can be trained in the following manner: the initial agent performs the current action in a virtual environment and obtains the reward information and subsequent state information of the current action based on the reward function. The virtual environment is constructed from the initial information, the heat collection model, the heat transfer model, and the prediction model. The reward function is obtained by fusing power consumption, temperature difference information, soil thermal balance information, the smoothness of the action of the executing device, and peak-valley electricity price information. Based on the interaction information obtained by the intermediate agent interacting with the real environment, the intermediate agent is updated to obtain a control agent that meets the preset conditions. The intermediate agent is obtained by updating the initial agent using the current state information, the current action, the reward information, and the subsequent state information.

[0052] In practical applications, training the initial agent directly in the real environment is time-consuming and poses security risks. Therefore, the initial agent can be trained in a virtual environment. After the initial agent meets the preset convergence conditions, it can be further transferred to the real environment to interact with the real environment, obtain interaction information, and then update the intermediate agent through the interaction information to obtain a control agent that meets the preset conditions.

[0053] The heat collection model is a virtual model that describes the heat collection efficiency and fluid thermal process of the heat collection device. The heat exchange model is a virtual model that covers the heat and mass transfer of the evaporator, condenser and buried pipe side, and can characterize the dynamics of soil temperature and the relationship between the inlet and outlet water temperatures of the heat exchange equipment.

[0054] The current state information is the state information of the real environment detected by the sensor. The current action is the action corresponding to the control command determined by the intermediate agent based on the current state information. The reward information is the single-step evaluation value calculated by the reward function after the intermediate agent executes the current action. The subsequent state information is the state information of the real environment returned by the sensor after the action is completed.

[0055] According to embodiments of the present invention, control action information is verified and dynamically corrected in real time through constraint verification information and constraint functions, achieving optimal energy efficiency while strictly adhering to the physical safety boundaries of the equipment. By integrating power consumption, temperature difference information, soil thermal balance information, action smoothness, and peak-valley electricity price information into a multi-objective reward function, equipment safety constraints are simultaneously optimized with operating energy efficiency and economy in a virtual training environment. This effectively avoids the control risks of traditional methods when the model is inaccurate, achieving multi-objective synergy of safety, energy efficiency, and economy. Furthermore, through a two-stage training strategy of virtual environment pre-training and real environment online fine-tuning, the control agent masters the basic control strategy in a virtual environment based on a physical model, and then continuously optimizes it through real system interaction data. This significantly improves the robustness of the control agent under non-stationary operating conditions, effectively responding to the challenges of sudden changes in real scenarios and meeting the control needs of different operating conditions in practical applications.

[0056] According to an embodiment of the present invention, the above-mentioned nonlinear transformation of the state information of the solar ground source heat pump system to obtain multiple control action information includes: combining the weighted result obtained by weighting the environmental information, system state information and equipment state information with their respective weights with the transformation coefficient to obtain control action information for the water pump speed and switching action information for the heat pump mode among the multiple control action information.

[0057] According to an embodiment of the present invention, environmental information, system state information and device state information are first weighted and fused by the hidden layer of the control agent, combined with conversion coefficients, and then nonlinearly activated to generate control action information that can be directly executed.

[0058] According to embodiments of the present invention, control action information may include adjusting the water pump speed to the target speed, and switching action information may include turning off the heat pump, setting the heat pump to heating mode or cooling mode. The aforementioned water pumps (including solar-side pumps, ground-source pumps, and user-side pumps) are all variable frequency pumps, each including 20 control speeds (corresponding to value ranges of [-10, 0, 10]), where speed 0 indicates no change in frequency, -1 indicates a decrease in frequency, +1 indicates an increase in frequency, and so on. The heat pump has three control modes: 0 indicates the heat pump is off, 1 indicates the heat pump is in heating mode, and 2 indicates the heat pump is in cooling mode.

[0059] According to an embodiment of the present invention, by weighted fusion of dispersed environmental information, system information and equipment status information through a control agent, executable control action information is directly generated. Then, the water pump speed and heat pump mode are controlled through the control action information, so that the solar ground source heat pump system can maintain high energy efficiency operation under different working conditions, while ensuring system safety and indoor comfort.

[0060] According to an embodiment of the present invention, the method further includes: when the comparison result between multiple control action information and constraint verification information indicates that the control action information is safe action information, determining the control action information as target action information, wherein the constraint verification information includes at least one of conditional statements and lookup table logic; or using the state information and control action information of the solar ground source heat pump system as independent variables of the constraint function, determining the safety level corresponding to the control action information, and determining the control action information as target action information when the safety level is greater than or equal to the safety threshold.

[0061] For example, if the control agent determines the control action information as increasing the frequency of the ground source pump to level 10 based on the reward value, it first needs to determine whether the control action information meets the conditions corresponding to the conditional statement (assuming that the inlet water temperature of the buried pipe is greater than 42 degrees Celsius and further heating is prohibited). At this time, the inlet water temperature of the buried pipe has already reached 45 degrees Celsius, so the control action information is determined to be illegal information and cannot be determined as the target action information.

[0062] For example, when the outdoor air temperature is 36 degrees Celsius and the solar irradiance is 900 W / m², the lookup table logic shows "maximum allowable for solar side pump +6 levels", and the control action information generated by the control agent is "solar side +8 levels". Then the target action information can be determined as "solar side pump +6 levels", thereby ensuring that all target action information is safe action information.

[0063] For example, the controlling agent requests that the compressor frequency jump directly from 30Hz to 50Hz. The constraint function can calculate the safety level of the compressor frequency change in real time. At this time, the constraint function outputs a safety level of 0.2, which is less than the safety threshold (set to 0.3 here). Therefore, this control action information is illegal and cannot be identified as the target action information.

[0064] According to embodiments of the present invention, by performing security constraint verification through conditional statements and lookup table logic, control action information that obviously exceeds the limits can be eliminated. Furthermore, by using state information and control action information as independent variables of the constraint function, the security level corresponding to the control action information can be determined. This preserves the action direction of the control agent in pursuing the maximum reward, while ensuring the security of the execution device and system through constraint verification information.

[0065] According to an embodiment of the present invention, controlling the actuator in a solar ground source heat pump system based on target action information to update or maintain the operating state of the solar ground source heat pump system includes: converting the target action information into a physical control signal and sending the physical control signal to the driver of the actuator, so that the driver adjusts the operating parameters of the actuator based on the physical control signal to update or maintain the operating state of the solar ground source heat pump system; wherein the actuator includes at least one of a water pump, a valve, a compressor, and a frequency converter, and the operating parameters include at least one of speed, opening degree, frequency, or power.

[0066] The target action information is a discrete or continuous digital instruction output by the control agent based on the current state and policy network, used to describe the expected adjustment amount of the solar ground source heat pump system at the next moment. It may include: percentage change in opening degree, frequency increment, power percentage or normalized action code; the physical control signal is the electrical data that can drive the execution device after the target action information is converted from digital to analog or pulse width modulation; the operating parameters are the real-time operating parameters of the execution device.

[0067] For example, if the target action information requires the ground source pump to increase the flow rate by 5%, a corresponding physical control signal (action code) can be generated and sent to the driver via the bus. The driver adjusts the output voltage of the actuator (the frequency converter of the ground source pump) according to the physical control signal (from 40Hz to 42Hz), thereby increasing the speed of the ground source pump by 5%, which in turn increases the flow rate accordingly.

[0068] According to an embodiment of the present invention, by converting the target action information into a physical control signal in the form of current, voltage or pulse through digital-to-analog conversion, and sending it to the driver of the actuator such as water pump, valve, compressor and frequency converter, the driver adjusts the speed, opening degree, frequency or power according to the physical control signal, so that the flow rate and heat exchange are dynamically matched with the building load, maintaining the stable and efficient operation of the system, and realizing the real-time adjustment of the operating status of the solar ground source heat pump system.

[0069] According to an embodiment of the present invention, the above method further includes: combining multiple weighted results obtained by weighting power consumption, temperature difference information, thermal balance information, motion smoothness and peak-valley electricity price information with their respective weights to obtain a reward function, wherein the peak-valley electricity price information includes starting the water pump when the electricity price is at the valley price and limiting the frequency of water pump starting when the electricity price is at the peak price.

[0070] According to an embodiment of the present invention, the aforementioned power consumption may be, for example, the total power of the system (in kW, including the power of the compressor and the water pump); the aforementioned temperature difference information may include indoor temperature difference information and soil temperature difference information, wherein the indoor temperature difference information represents the deviation between the actual indoor temperature and the indoor set temperature, and the soil temperature difference information represents the difference between the actual soil temperature and the soil set temperature; the aforementioned heat balance information may represent the deviation between the heat extracted from the soil and the reinjection heat, and the smoothness of the action may represent the amount of change between the current control action and the previous control action.

[0071] The reward function can be expressed by the following formula (3).

[0072] (3)

[0073] in, It can represent the reward function value. It can indicate power consumption. Information on indoor temperature difference. For thermal balance information, It can represent the heat extracted from the soil. It can represent the heat generated by soil recharge. It can indicate the smoothness of motion. It can display peak and off-peak electricity price information. It can represent soil temperature difference information. It can indicate soil temperature. It can indicate the set temperature of the soil. , , , , , These are the weight parameters.

[0074] In an embodiment of the present invention, weight Weights used to adjust the importance of energy consumption in the overall reward function Weighting is used to balance the relationship between comfort and power consumption. Optimization for controlling soil thermal balance, weights Priorities and weights used to control motion smoothness Used to balance the relationship between peak and off-peak electricity prices and comfort, weighting Priority for controlling soil temperature balance.

[0075] In an embodiment of the present invention, weight It can be set to 1.0 (power consumption as the primary factor), weight It can be set to 5.0 (comfort priority increased), weight It can be set to 0.2 (cumulative deviation penalty within the season, increasing the weight of this item when heat extraction in winter is greater than heat reinjection in summer), weight It can be set to 0.05 (for smoother action and reduced frequent changes).

[0076] According to an embodiment of the present invention, by using a five-dimensional weighted reward system based on power consumption, temperature difference information, thermal balance information, smoothness of action, and peak-valley electricity price, the water pump is activated during off-peak electricity prices and the activation frequency of the water pump is limited during peak electricity prices. This makes the intelligent agent more inclined to use low-cost electricity to complete heat transfer and energy storage, reducing operating costs while maintaining a comfortable indoor temperature and improving the user experience.

[0077] According to an embodiment of the present invention, the initial information includes meteorological information and building load information for historical periods, heat collection performance parameters of the heat collection device, and heat exchange performance parameters of the heat exchange device; the method further includes: constructing a heat collection model based on meteorological information and heat collection performance parameters to predict the heat collection efficiency of the heat collection device and the water temperature at the outlet of the device using the heat collection model; and constructing a heat exchange model based on building load information and heat exchange performance parameters to predict soil temperature information and the inlet and outlet water temperatures of the heat exchange device using the heat exchange model.

[0078] Historical meteorological information may include: outdoor air temperature and total solar irradiance. Building load information may include: power supply under heating conditions and power supply under cooling conditions. The performance parameters of the solar collector may include: inlet fluid temperature, outlet fluid temperature, effective area, loop fluid mass flow rate, and isobaric specific heat of the working fluid. The performance parameters of the heat exchanger may include: equivalent heat source temperature on the evaporator side, equivalent heat sink temperature on the condenser side, water temperature entering the heat exchanger unit on the source side, return water temperature on the load side, supply water temperature on the load side, evaporator side temperature difference, condenser side temperature difference, power consumption of the heat exchanger unit, water mass flow rate on the load side, and isobaric specific heat constant of water.

[0079] Furthermore, the heat collection model and heat transfer model can be constructed using the initial information mentioned above. The specific construction process is shown below.

[0080] The heat collection model can characterize the heat transfer performance of the heat collection equipment in engineering applications. The heat collection model can be constructed using the Hottel–Whillier equivalent form, which is consistent with the engineering design and easy to fit on site. The main parameters of the heat collection model are shown in the following formula (4):

[0081] (4)

[0082] in, It can represent instantaneous thermal efficiency. It can represent the effective optical efficiency (a parameter that needs to be fitted). It can represent the equivalent total heat dissipation coefficient (the parameter that needs to be fitted). This indicates the inlet fluid temperature of the solar collector (in °C). This indicates the outlet fluid temperature of the solar collector (in °C). This indicates the outdoor air temperature (in °C). This represents total solar irradiance (in W / m²). It can represent the net output heat of the solar collector (in W). It can represent the effective area of ​​the solar collector (in m²). This indicates the mass flow rate of the fluid in the circuit of the solar collector (unit: kg / s). It can represent the specific heat of the working fluid at constant pressure in the circuit of the solar collector (unit: J / (kg·K)).

[0083] A heat exchange model can characterize the heat exchange performance of heat exchange equipment in engineering applications. In engineering applications, it is generally based on the "mass flow rate - inlet and outlet temperature difference - electric power / measured efficiency" data that can be obtained in real time on site to construct a Carnot efficiency reduction model (i.e., a heat exchange model). Celsius temperature (°C) is used as the input layer variable, and the Carnot ratio is converted to absolute temperature according to T(K)=T (unit is °C)+273.15 when calculating the Carnot ratio.

[0084] The main parameters of the heat exchange model are shown in the following formulas (5) to (9).

[0085] (5)

[0086] in, It can represent the coefficient of performance (COP) of heat exchange equipment. This can represent the Carnot efficiency reduction factor. It can represent the equivalent heat sink temperature on the condenser side (in °C). It can represent the equivalent heat source temperature on the evaporator side (in °C).

[0087] (6)

[0088] EER can represent the cooling energy efficiency ratio of a heat exchanger. This can represent the Carnot efficiency reduction factor. It can represent the equivalent heat sink temperature on the condenser side (in °C). It can represent the equivalent heat source temperature on the evaporator side (in °C).

[0089] The source / sink equivalent temperature is shown in the following formula (7).

[0090] (7)

[0091] in, It can represent the equivalent heat source temperature on the evaporator side. This can indicate the water temperature entering the heat exchanger unit from the source side. It can represent the difference on the evaporation side (fitting parameters). It can represent the equivalent heat sink temperature on the condensing side. It can represent the difference on the condensation side (fitting parameters). It can represent the water supply temperature on the load side (in °C).

[0092] (8)

[0093] in, It can represent the power supplied in heating mode (in W). It can represent the water mass flow rate on the load side (unit: kg / s). It can represent the isobaric specific heat constant of water (unit: J / (kg·K)). It can represent the return water temperature on the load side (in °C). It can represent the water supply temperature on the load side (in °C). It can represent the electrical power of the heat exchange equipment unit (the metering caliber is consistent with the on-site caliber, and the unit is W, depending on whether it includes an external pump / fan). It can represent the coefficient of performance (COP) of heat exchange equipment. EER can represent the power supplied in cooling mode (in W), and can represent the cooling energy efficiency ratio of heat exchange equipment.

[0094] According to an embodiment of the present invention, The Carnot efficiency reduction factor can be set within the range of [0.35, 0.55]. (Evaporation side end difference) and (Condensing side temperature difference) can be set in the range of [2,6] ℃ in heating mode and in the range of [3,8] ℃ in cooling mode.

[0095] In an embodiment of the present invention, the heating performance coefficient can be calculated by actual measurement of the heat exchange equipment, as shown in the following formula (9) for the actual measured heating performance coefficient of the heat exchange equipment.

[0096] (9)

[0097] in, It can represent the coefficient of performance (COP) of heat exchange equipment. It can represent the water mass flow rate on the load side (unit: kg / s). It can represent the isobaric specific heat constant of water (unit: J / (kg·K)). It can represent the return water temperature on the load side. It can represent the water supply temperature on the load side. It can represent the electrical power of a heat exchange equipment unit.

[0098] Furthermore, the Carnot efficiency reduction factor, evaporation-side difference, and condensation-side difference can be fitted using the nonlinear least squares method. The Carnot efficiency reduction factor should be constrained to the range of 0.3 to 0.7, the evaporation-side difference should be greater than 1, and the condensation-side difference should be less than or equal to 8.

[0099] According to embodiments of the present invention, by using historical meteorological information and building load information, the heat collection performance parameters of the heat collection equipment and the heat exchange performance parameters of the heat exchange equipment as input data, a heat collection model is further modeled based on the heat collection performance parameters of the heat collection equipment and meteorological information, and a heat exchange model is modeled based on the building load information and heat exchange performance parameters. This allows the virtual environment to deduce the heat collection efficiency, water temperature at the equipment outlet, soil temperature information, and inlet and outlet water temperatures of the heat exchange equipment under different weather conditions and building loads. This provides a high-fidelity training scenario for the initial agent, reduces trial and error costs, and ensures that the initial agent has the ability to judge the heat collection efficiency of the heat collection equipment and the soil thermal effect before being deployed to the real environment.

[0100] According to an embodiment of the present invention, the above method further includes: using meteorological information, building load information, heat collector performance parameters, and heat exchange performance parameters as sample data to train an initial prediction model; obtaining a prediction model under the condition that the trained initial prediction model meets the prediction accuracy; and using the prediction model to analyze the time-series dependencies between meteorological information, heat exchange equipment response information, heat collector efficiency change information, and building load information, respectively, to obtain the load demand information of the target building in the solar ground source heat pump system during a specific period.

[0101] The prediction model can be an autocorrelation transformation (ACR) model, which can effectively decompose long-term trends (such as seasonal temperature changes) and short-term fluctuations (such as daily load peaks and troughs) in time series. It captures key time steps and periodic patterns through an autocorrelation attention mechanism, improving the ability to model load demand. Furthermore, the ACR model can handle complex time-series dependencies between meteorological factors (such as solar radiation and outdoor temperature) and system dynamics (such as ground source temperature field response and collector efficiency changes), thus adapting to solar ground source heat pump systems and providing accurate load forecasts. Compared with traditional models, the ACR model performs exceptionally well in multi-step prediction tasks, effectively reducing accumulated errors and ensuring long-term prediction accuracy. The high-precision load forecasts of the ACR model provide accurate data support for control strategies (such as intelligent start-stop scheduling and energy optimization management), thereby improving system energy efficiency and stability.

[0102] The prediction model can be constructed using the following formulas (10) to (19).

[0103] (10)

[0104] in, This can represent the input data (such as building load information (unit: kW)) at time t. As a trend term, it can characterize low-frequency changes (such as seasonal changes, climate background, etc.). The residual term can characterize high-frequency fluctuations (such as daily variations, random disturbances, etc.).

[0105] According to an embodiment of the present invention, the above-mentioned input data can be the target building's heating and cooling load information, solar irradiance, outdoor temperature, and holiday information (i.e., the order of holidays and weekdays in the past 30 days) over the past 30 days. The input sequence length of the autocorrelation transformation model can be set to 168 hours, the prediction step size can be set to 24 hours, the model capacity can be set to 128, the number of attention heads can be set to 4, the number of training rounds can be set to 50-100 rounds, the batch size is 64, and the initial value of the Adam (Adaptive Moment Estimation) optimizer learning rate is set to 0.001, which decreases monotonically to 0.0001 according to the cosine curve during training.

[0106] The trend term can be calculated using the following formula (11).

[0107] (11)

[0108] in, It can represent the trend term, and k can represent the size of the moving average window (the value can be 12~24 hours or 7~14 days). It can represent input data. It can represent a trend. It can represent a residual term, where i is the index term.

[0109] According to an embodiment of the present invention, in the prediction of building heating and cooling loads, k can be set to 24 hours or 7 days to smooth out short-term fluctuations.

[0110] The following formula (12) shows the autocorrelation function of the prediction model.

[0111] (12)

[0112] in, The autocorrelation strength at a delay of τ, It can represent a query vector (with dimension dk). It can represent a key vector; It can represent the delay length (the value range is 1≤τ≤T).

[0113] In embodiments of the present invention, τ can be limited to a possible period range of the system (e.g., 24 hours, 7 days, or 365 days) to reduce the amount of calculation. In a solar ground source heat pump system, τ can be set to 24 hours, 7 days, or 30 days, etc.

[0114] Furthermore, the optimal delay (i.e. the delay length that is most similar to the current moment in the historical sequence) can be calculated using the following formula (13).

[0115] (13)

[0116] in, It can represent the optimal delay, which is the delay length (usually 24 hours, 48 ​​hours, or 7 days) that is most similar to the current moment in the historical sequence. It can represent the delay length. It can represent the autocorrelation strength under a delay τ.

[0117] In embodiments of the present invention, the optimal delay is generally 24 hours for summer cooling and 7 days, 30 days or 31 days for winter heating, in order to reflect persistent low temperatures.

[0118] Furthermore, in order to enable the prediction model to fully exploit the periodic features corresponding to the above-mentioned optimal delay, the present invention introduces autocorrelation attention calculation in the prediction model, which can be calculated by the following formula (14) to obtain the autocorrelation attention output matrix.

[0119] (14)

[0120] in, The output matrix can be represented by the autocorrelation attention matrix, where Q represents the query vector, K represents the key vector, and V represents the value vector. This can represent the optimal delay. It can represent normalized weights. It can be represented at time step The value vector.

[0121] The normalized weights can be calculated using the following formula (15).

[0122] (15)

[0123] in, It can represent normalized weights. This can be represented as the optimal delay. The autocorrelation strength under the following conditions Indicates delay The autocorrelation strength under [condition].

[0124] Furthermore, based on the above prediction model, the output data of the prediction model can be further described by the following formula (16).

[0125] (16)

[0126] in, It can represent the predicted value (in kW) for the next L steps. It can represent the trend portion predicted by the predictive model. It can represent the residual / part of the prediction by the prediction model.

[0127] In embodiments of the present invention, the prediction step size L is generally selected as 24 hours or 72 hours. In a solar ground source heat pump system, a reasonable value of L should cover the optimization time domain (e.g., 3-24 hours) to schedule the operation of the heat pump and the energy storage of the water tank.

[0128] In an embodiment of the present invention, during the modeling process of the prediction model, meteorological variables can be weighted as partial features of the value vector, thereby improving the prediction accuracy.

[0129] Furthermore, the query vector can be updated using the following formula (17).

[0130] (17)

[0131] in, It can represent the value of the target action. It can represent an instant reward. It can represent a discount factor. It can represent the prediction model's prediction of the next state. and actions Action value estimation, E can represent the loss function, and E can represent the expected value. It can represent the parameters of the prediction model. It can represent the value of actions predicted by the predictive model.

[0132] According to an embodiment of the present invention, the prediction model can be determined to be convergent if the constraints shown in the following formulas (18) and (19) are satisfied.

[0133] (18)

[0134] (19)

[0135] Among them, Q X Q can represent the value of an action at the current moment. X-1 It can represent the value of an action at the previous moment; R t R can represent the reward value at the current moment. t-1 It can represent the reward value from the previous moment.

[0136] According to an embodiment of the present invention, the initial intelligent agent performs a current action in a virtual environment and obtains reward information and subsequent state information of the current action based on a reward function, including: after performing the current action in the virtual environment, obtaining the current power consumption, the temperature difference between the current indoor temperature and the target temperature, the current thermal balance information, the smoothness of the current action, and the current electricity price information; determining the reward information based on the reward function, the current power consumption, the temperature difference, the current thermal balance information, the smoothness of the current action, and the current electricity price information; and obtaining the updated state information of the virtual environment after responding to the current action as subsequent state information.

[0137] After the agent performs its current action in the virtual environment, the virtual environment returns a set of actual operating metrics, which serve as the basis for calculating rewards and subsequent states. For example, suppose the agent decides to increase the setpoint of the ground-source water pump's supply temperature by 2 degrees Celsius and simultaneously increase the frequency. The virtual environment will provide the current power consumption, the temperature difference between the current indoor temperature and the target temperature, the current thermal balance information, the smoothness of the current action, and the current electricity price. The reward function weights and combines these five metrics: the lower the power consumption, the smaller the temperature difference, the closer the thermal balance is to zero, the smoother the instruction change, and the lower the electricity price, the higher the reward; conversely, the reward decreases. The resulting reward value and its corresponding five metrics together constitute the subsequent state information, which the agent uses in the next training or decision-making step.

[0138] According to an embodiment of the present invention, based on the interaction information obtained by the interaction between the intermediate agent and the real environment, the intermediate agent is updated to obtain a control agent that meets preset conditions. This includes: inputting operational information into the intermediate agent to obtain control actions, wherein the operational information is obtained by collecting operational data from multiple subsystems in a solar ground source heat pump system in a real environment using real sensing devices; collecting real-time interactive information and energy consumption data after the execution of control actions by the executing device to form real-time interactive information; storing the real-time interactive information in an experience database, and periodically sampling data batches from the experience database to update the parameters of the intermediate agent with the goal of minimizing time-series difference errors; after the intermediate agent has continuously run for a preset period in the real environment, obtaining the average cumulative reward information of the intermediate agent; and determining the intermediate agent as the control agent when the average cumulative reward information meets preset reward conditions.

[0139] Real-time state information represents the measured results returned by sensors after the executing device completes the control action; real-time interaction information is a complete record packaged from real-time state information and energy consumption data; time-series difference error can characterize the difference between the expected reward and the actual reward, and the intermediate agent continuously improves by constantly reducing the time-series difference error. Preset reward conditions may include the error of the average cumulative reward value being less than a preset error threshold or the average cumulative reward value tending to stabilize.

[0140] For example, the operating information is set as follows: outdoor temperature 32 degrees Celsius, solar irradiance 850 W / m², load-side return water temperature 12 degrees Celsius, temperature difference between the current indoor temperature and the target temperature 0.8 degrees Celsius, and current power consumption 45 kW. The intermediate agent outputs control action to maintain the ground source side water pump frequency at 35 W. After the ground source side water pump executes the control action, the actual status information is: load-side return water temperature 12.1 degrees Celsius, temperature difference 0.7 degrees Celsius, power consumption 0.075 kWh (1 minute). The reward score is calculated as 0.12 according to the reward function. At the same time, this real-time interaction information is stored in the experience database, and the average hourly reward (i.e., average cumulative reward information) is calculated based on the records of 7 days. When the average hourly reward meets the preset reward condition (set to be less than the preset error threshold), the intermediate agent is considered to have completed training and is identified as the control agent.

[0141] In one embodiment of the present invention, a multi-objective optimization scenario is provided for controlling a comprehensive energy station in a park / factory (which also provides cooling in summer, heating in winter, and domestic hot water for some periods throughout the year). The control objectives include: minimizing total annual power consumption, minimizing indoor temperature deviation (for comfort), maintaining soil thermal balance (avoiding cold and heat accumulation and preventing long-term imbalance), ensuring smooth operation of the equipment (reducing the number of start-ups and shutdowns and minimizing frequent frequency conversion operations), and optimizing the peak and off-peak electricity prices (optimizing the start-up and shutdown of the heat pump during off-peak hours to reduce power consumption during peak hours).

[0142] The system is configured with a data acquisition duration of 1 hour (to match building heating and cooling load changes and water system inertia, reducing operation frequency and optimizing noise), a prediction step size of 24 hours (covering daily load and irradiance fluctuations, facilitating water tank filling and emptying and unit peak shaving), and a water pump step frequency of 5Hz (to achieve adjustability within the inverter's linear range and avoid excessively fine movements that could cause vibration). The ground source heat pump (heat exchanger) has a cooling capacity of 220kW (46kW) and a heating capacity of 226kW (64kW); the air source heat pump (collector) provides 68kW (21.3kW) for cooling and 73kW (21.6kW) for heating. It features a three-loop variable frequency pump (0–50 / 60Hz) for the ground source, collector, and user sides. The water tank (with temperature / level / heat loss monitoring) has a capacity of 500L. Sensors collect data on solar irradiance, outdoor air temperature, indoor temperature, supply and return water temperature and flow rate for the collector / source / load sides, power consumption of the collector, and time. The instantaneous thermal efficiency in the heat collection device can be [0.6, 0.8], and the equivalent total heat dissipation coefficient can be [3, 9].

[0143] Figure 3 A flowchart illustrating the training process of a control agent in a solar ground source heat pump system according to an embodiment of the present invention is shown.

[0144] like Figure 3 As shown, the training process for the control agent in this solar ground source heat pump system includes operations S310 to S350.

[0145] In operation S310, a virtual environment is constructed based on initial information, heat collection model, heat exchange model and prediction model.

[0146] In operating S320, the initial agent is trained in a virtual environment until the initial agent converges.

[0147] The initial agent's structure and training hyperparameter configuration are as follows: Network structure: A 2-3 layer multilayer perceptron is used, with 256, 256, and 128 neurons per layer, respectively. The activation function is ReLU (Rectified Linear Unit) to introduce non-linear characteristics. Dual network architecture: It includes an online network and a target network. Both have the same structure but different parameter update strategies to stabilize the training process.

[0148] Discount factor: set to 0.98 to balance immediate rewards with long-term returns, ensuring that the agent considers seasonal soil thermal balance while pursuing daily energy savings.

[0149] Optimizer: The Adam optimizer was selected, the learning rate was set to 0.0003, and the Huber loss function was used to improve the model's robustness to outliers.

[0150] Experience replay: The experience replay pool has a capacity of 1,000,000, and 256 samples are randomly selected from the pool for training each time. In the priority replay strategy, the importance sampling parameter α is set to 0.6, and the importance sampling parameter β increases linearly from 0 to 1.0 to improve learning efficiency.

[0151] Target network update: The target network parameters are updated using a soft update strategy at a frequency of 5 × 10⁻³, or a hard update every 5,000 steps, to reduce oscillations during training. With this configuration, the agent can effectively learn the optimal action selection under different environmental conditions during training, achieving efficient and stable control of the solar ground source heat pump system.

[0152] In operation S330, the initial agent is transferred to the real environment to obtain an intermediate agent.

[0153] During the operation of S340, real-time interactive information of the execution device is collected and stored in the experience database.

[0154] In operating the S350, the parameters of the intermediate agent are updated based on the experience database until the intermediate agent is stable.

[0155] During training and deployment, measured data from the past 12 months were selected and interpolated to fill in missing values, while holiday information was also labeled. Simultaneously, the parameters of the heat collection and exchange equipment were fitted using the methods described in the above embodiments. After ensuring the error was within an acceptable range, the model parameters were frozen in the virtual environment for transfer to the real environment. Furthermore, the initial agent was trained for 200,000 to 400,000 cycles in the virtual environment, while an early stopping strategy was employed to verify rewards or performance indicators. When transferring the model to the real environment, online updates with low weights (learning rate annealing 10x) can be performed, and action smoothing and switching cooldown mechanisms can be enabled. If the comfort level exceeds the threshold for 2 hours or the soil temperature deviation exceeds the set threshold, the system can be promptly switched to rule-based control or predictive control strategies to control the solar ground source heat pump system.

[0156] During the evaluation process, the performance of the control method of the solar ground source heat pump system provided by this invention and the rule-based control method in related technologies can be judged by the following indicators, including: energy consumption (total power consumption, peak power consumption, pump consumption ratio), comfort (absolute error of indoor temperature deviation, duration of out-of-bounds), smoothness of operation (number of start-ups and shutdowns, number / amplitude of frequency changes), soil heat balance (seasonal / annual net heat take-off from the source side is close to 0; ground temperature drift ≤0.5–1.0K / season), and robustness (performance does not deteriorate by more than 10% under extreme weather (high temperature / cold wave / continuous rain).

[0157] Compared with traditional rule-based control methods, the control method for solar ground source heat pump systems proposed in this invention reduces total power consumption by 8-15%, and by 3-7% compared with predictive control strategies. Furthermore, the control method for solar ground source heat pump systems proposed in this invention maintains an indoor temperature difference of 0.5-0.7 degrees Celsius, while rule-based control methods maintain it at 0.8-1.1 degrees Celsius, and predictive control strategies maintain it at 0.6-0.8 degrees Celsius. Simultaneously, compared with the other two control methods, the control method for solar ground source heat pump systems proposed in this invention reduces the average daily start-up and shutdown frequency of water pumps and heat pumps by 25%-40%, and reduces soil heat extraction from imbalance accumulation by 10%-20% compared to rule-based control methods (with more timely active reinjection in summer).

[0158] Based on the above-described control method for a solar ground source heat pump system, this invention also provides a control device for a solar ground source heat pump system. The following will be combined with... Figure 4 The device is described in detail.

[0159] Figure 4 A structural block diagram of a control device for a solar ground source heat pump system according to an embodiment of the present invention is shown.

[0160] like Figure 4 As shown, the control device 400 of the solar ground source heat pump system in this embodiment includes a conversion module 410 and an update module 420.

[0161] The conversion module 410 is used to perform nonlinear conversion on the state information of the solar ground source heat pump system to obtain multiple control action information, wherein the state information is determined by the environmental information, system state information, and equipment state information of the solar ground source heat pump system. In one embodiment, the conversion module 410 can be used to execute the operation S210 described above, which will not be repeated here.

[0162] The update module 420 is used to control the execution equipment in the solar ground source heat pump system based on target action information, and to update or maintain the operating status of the solar ground source heat pump system. The target action information is action information that satisfies preset constraints from multiple control action information. The preset constraints include at least one of constraint verification information and constraint functions. In one embodiment, the update module 420 can be used to perform the operation S220 described above, which will not be repeated here.

[0163] The control agent is trained as follows: the initial agent performs the current action in a virtual environment and obtains the reward information and subsequent state information of the current action based on the reward function. The virtual environment is constructed from the initial information, the heat collection model, the heat transfer model, and the prediction model. The reward function is obtained by integrating power consumption, temperature difference information, soil thermal balance information, the smoothness of the action of the execution equipment, and peak and valley electricity price information. Based on the interaction information obtained by the intermediate agent and the real environment, the intermediate agent is updated to obtain the control agent that meets the preset conditions. The intermediate agent is obtained by updating the initial agent using the current state information, the current action, the reward information, and the subsequent state information.

[0164] According to an embodiment of the present invention, the conversion module 410 includes a combination submodule, which is used to combine the weighted result obtained by weighting environmental information, system status information and equipment status information with their respective weights with the conversion coefficient to obtain control action information for water pump gear and switching action information for heat pump mode among multiple control action information.

[0165] According to an embodiment of the present invention, the above-mentioned device further includes: a first determining module, configured to determine the control action information as target action information when the comparison result between multiple control action information and constraint verification information indicates that the control action information is safe action information, wherein the constraint verification information includes at least one of conditional statements and lookup table logic; or a second determining module, configured to use the state information of the solar ground source heat pump system and the control action information as independent variables of the constraint function to determine the safety level corresponding to the control action information, and determine the control action information as target action information when the safety level is greater than or equal to a safety threshold.

[0166] According to an embodiment of the present invention, the update module 420 includes: a conversion submodule, used to convert target action information into physical control signals and send the physical control signals to the driver of the execution device, so that the driver adjusts the operating parameters of the execution device based on the physical control signals to update or maintain the operating status of the solar ground source heat pump system; wherein the execution device includes at least one of a water pump, a valve, a compressor and a frequency converter, and the operating parameters include at least one of speed, opening degree, frequency or power.

[0167] According to an embodiment of the present invention, the above-mentioned device further includes: a weighted combination module, used to combine multiple weighted results obtained by weighting power consumption, temperature difference information, thermal balance information, action smoothness and peak-valley electricity price information with their respective weights to obtain a reward function, wherein the peak-valley electricity price information includes starting the water pump when the electricity price is the valley price and limiting the starting frequency of the water pump when the electricity price is the peak price.

[0168] According to an embodiment of the present invention, the initial information includes meteorological information and building load information for historical periods, heat collection performance parameters of the heat collection device, and heat exchange performance parameters of the heat exchange device; the device further includes: a first construction module for constructing a heat collection model based on meteorological information and heat collection performance parameters, so as to use the heat collection model to predict the heat collection efficiency of the heat collection device and the water temperature at the outlet of the device; and a second construction module for constructing a heat exchange model based on building load information and heat exchange performance parameters, so as to use the heat exchange model to predict soil temperature information and the inlet and outlet water temperatures of the heat exchange device.

[0169] According to an embodiment of the present invention, the above-mentioned device further includes: a training module, used to train an initial prediction model using meteorological information, building load information, heat collection performance parameters, and heat exchange performance parameters as sample data, and to obtain a prediction model under the condition that the trained initial prediction model meets the prediction accuracy; and an analysis module, used to analyze the time-series dependencies between meteorological information, heat exchange equipment response information, heat collection equipment efficiency change information, and building load information, respectively, using the prediction model, to obtain the load demand information of the target building in the solar ground source heat pump system during a specific period.

[0170] According to an embodiment of the present invention, the above-mentioned device further includes: an execution module, configured to obtain current power consumption, temperature difference between current indoor temperature and target temperature, current thermal balance information, current action smoothness, and current electricity price information after performing the current action in the virtual environment; a third determination module, configured to determine reward information based on the reward function, current power consumption, temperature difference, current thermal balance information, current action smoothness, and current electricity price information; and an acquisition module, configured to acquire the updated state information of the virtual environment after responding to the current action, as subsequent state information.

[0171] According to an embodiment of the present invention, the above-mentioned device further includes: an input module for inputting operating information into an intermediate intelligent agent to obtain control actions, wherein the operating information is obtained by collecting operating data of multiple subsystems in a solar ground source heat pump system in a real environment using real sensing devices; an acquisition module for collecting real state information and energy consumption data after the execution device performs control actions to form real-time interactive information; a storage module for storing the real-time interactive information in an experience database and periodically sampling data batches from the experience database to update the parameters of the intermediate intelligent agent with the goal of minimizing time-series difference errors; and an operation module for obtaining the average cumulative reward information of the intermediate intelligent agent after the intermediate intelligent agent has continuously run for a preset period in a real environment, and determining the intermediate intelligent agent as a control intelligent agent when the average cumulative reward information meets preset reward conditions.

[0172] According to embodiments of the present invention, any plurality of modules in the conversion module 410 and the update module 420 may be combined into one module, or any one of the modules may be split into multiple modules. Alternatively, at least a portion of the functionality of one or more of these modules may be combined with at least a portion of the functionality of other modules and implemented in one module. According to embodiments of the present invention, at least one of the conversion module 410 and the update module 420 may be at least partially implemented as hardware circuitry, such as a field-programmable gate array (FPGA), a programmable logic array (PLA), a system-on-a-chip, a system-on-a-substrate, a system-on-package, an application-specific integrated circuit (ASIC), or any other reasonable means of integrating or packaging circuitry, or implemented in software, hardware, or firmware, or in any appropriate combination of any of these three implementation methods. Alternatively, at least one of the conversion module 410 and the update module 420 may be at least partially implemented as a computer program module, which, when run, can perform corresponding functions.

[0173] According to embodiments of the present invention, program code for executing the computer programs provided in the embodiments of the present invention can be written in any combination of one or more programming languages. Specifically, these computational programs can be implemented using high-level procedural and / or object-oriented programming languages, and / or assembly / machine languages. Programming languages ​​include, but are not limited to, languages ​​such as Java, C++, Python, "C", or similar programming languages. The program code can be executed entirely on the user's computing device, partially on the user's device, partially on a remote computing device, or entirely on a remote computing device or server. In cases involving remote computing devices, the remote computing device can be connected to the user's computing device via any type of network, including a local area network (LAN) or a wide area network (WAN), or it can be connected to an external computing device (e.g., via the Internet using an Internet service provider).

[0174] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of the present invention. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in a block diagram or flowchart, and combinations of blocks in a block diagram or flowchart, may be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0175] Those skilled in the art will understand that the features described in the various embodiments of the present invention can be combined and / or combined in various ways, even if such combinations or combinations are not explicitly described in the present invention. In particular, the features described in the various embodiments of the present invention can be combined and / or combined in various ways without departing from the spirit and teachings of the present invention. All such combinations and / or combinations fall within the scope of the present invention.

[0176] The embodiments of the present invention have been described above. However, these embodiments are merely illustrative and not intended to limit the scope of the invention. Although various embodiments have been described above, this does not mean that the measures in the various embodiments cannot be used advantageously in combination. Various substitutions and modifications can be made by those skilled in the art without departing from the scope of the invention, and all such substitutions and modifications should fall within the scope of the invention.

Claims

1. A control method for a solar ground source heat pump system, characterized in that, A control agent applied to a solar ground source heat pump system, the method comprising: The state information of the solar ground source heat pump system is nonlinearly transformed to obtain multiple control action information, wherein the state information is determined by the environmental information, system state information and equipment state information of the solar ground source heat pump system; Based on target action information, the actuators in the solar ground source heat pump system are controlled to update or maintain the operating status of the solar ground source heat pump system. The target action information is action information that satisfies preset constraints among a plurality of control action information. The preset constraints include at least one of constraint verification information and constraint function. The control agent is trained in the following manner: The initial intelligent agent performs the current action in the virtual environment and obtains the reward information and subsequent state information of the current action based on the reward function. The virtual environment is constructed from initial information, heat collection model, heat exchange model and prediction model. The reward function is obtained by integrating power consumption, temperature difference information, soil thermal balance information, the action smoothness of the execution device and peak and valley electricity price information. Based on the interaction information obtained by the intermediate agent interacting with the real environment, the intermediate agent is updated to obtain a control agent that meets preset conditions. The intermediate agent is obtained by updating the initial agent using the current state information, the current action, the reward information, and the subsequent state information.

2. The method according to claim 1, characterized in that, The state information of the solar ground source heat pump system is nonlinearly transformed to obtain multiple control action information, including: The weighted result obtained by weighting the environmental information, the system status information, and the equipment status information with their respective weights is combined with the conversion coefficient to obtain the control action information for the water pump speed and the switching action information for the heat pump mode among the multiple control action information.

3. The method according to claim 1, characterized in that, The method further includes: If the comparison results between multiple sets of control action information and constraint verification information indicate that the control action information is safe action information, then the control action information is determined as the target action information, wherein the constraint verification information includes at least one of conditional statements and lookup table logic; or The state information of the solar ground source heat pump system and the control action information are used as independent variables of the constraint function to determine the safety level corresponding to the control action information. If the safety level is greater than or equal to the safety threshold, the control action information is determined as the target action information.

4. The method according to claim 1, characterized in that, Controlling the actuators in the solar ground source heat pump system based on target action information, and updating or maintaining the operating status of the solar ground source heat pump system, including: The target action information is converted into a physical control signal, and the physical control signal is sent to the driver of the execution device, so that the driver adjusts the operating parameters of the execution device based on the physical control signal to update or maintain the operating status of the solar ground source heat pump system; wherein, the execution device includes at least one of a water pump, a valve, a compressor, and a frequency converter, and the operating parameters include at least one of speed, opening degree, frequency, or power.

5. The method according to claim 1, characterized in that, The method further includes: The reward function is obtained by combining multiple weighted results obtained by weighting the power consumption, temperature difference information, thermal balance information, motion smoothness, and peak-valley electricity price information with their respective weights. The peak-valley electricity price information includes starting the water pump when the electricity price is at the valley price and limiting the frequency of starting the water pump when the electricity price is at the peak price.

6. The method according to claim 1, characterized in that, The initial information includes historical meteorological information and building load information, heat collection performance parameters of the heat collection equipment, and heat exchange performance parameters of the heat exchange equipment; the method further includes: The heat collection model is constructed based on the meteorological information and the heat collection performance parameters, so as to predict the heat collection efficiency of the heat collection device and the water temperature at the outlet of the device. The heat exchange model is constructed based on the building load information and the heat exchange performance parameters, so as to use the heat exchange model to predict soil temperature information and the inlet and outlet water temperatures of the heat exchange equipment.

7. The method according to claim 6, characterized in that, The method further includes: The meteorological information, the building load information, the heat collector performance parameters, and the heat exchange performance parameters are used as sample data to train the initial prediction model. Under the condition that the trained initial prediction model meets the prediction accuracy, the prediction model is obtained. By using the prediction model to analyze the time-series dependencies between the meteorological information, the response information of the heat exchange equipment, the efficiency change information of the heat collection equipment, and the building load information, the load demand information of the target building in the solar ground source heat pump system during a specific period is obtained.

8. The method according to claim 1, characterized in that, The initial agent performs the current action in the virtual environment and obtains the reward information and subsequent state information of the current action based on the reward function, including: After performing the current action in the virtual environment, the current power consumption, the temperature difference between the current indoor temperature and the target temperature, the current thermal balance information, the smoothness of the current action, and the current electricity price information are obtained. The reward information is determined based on the reward function, the current power consumption, the temperature difference, the current thermal balance information, the current action smoothness, and the current electricity price information; Obtain the updated state information of the virtual environment after responding to the current action, and use it as the subsequent state information.

9. The method according to claim 1, characterized in that, Based on the interaction information obtained from the interaction between the intermediate agent and the real environment, the intermediate agent is updated to obtain a control agent that meets preset conditions, including: The operation information is input into the intermediate intelligent agent to obtain control actions. The operation information is obtained by collecting operation data of multiple subsystems in a solar ground source heat pump system in a real environment using real sensing devices. Collect the actual status information and energy consumption data of the execution device after it performs the control action to form real-time interactive information; The real-time interaction information is stored in an experience database, and data batches are periodically sampled from the experience database to update the parameters of the intermediate agent with the goal of minimizing the time-series difference error. After the intermediate agent runs continuously for a preset period in the real environment, the average cumulative reward information of the intermediate agent is obtained. If the average cumulative reward information meets the preset reward conditions, the intermediate agent is determined as the control agent.

10. A control device for a solar ground source heat pump system, characterized in that, A control agent installed in a solar ground source heat pump system, the device comprising: The conversion module is used to perform nonlinear conversion on the state information of the solar ground source heat pump system to obtain multiple control action information, wherein the state information is determined by the environmental information, system state information and equipment state information of the solar ground source heat pump system. An update module is used to control the execution device in the solar ground source heat pump system based on target action information, and to update or maintain the operating status of the solar ground source heat pump system. The target action information is action information that satisfies preset constraints among a plurality of control action information. The preset constraints include at least one of constraint verification information and constraint function. The control agent is trained in the following manner: The initial intelligent agent performs the current action in the virtual environment and obtains the reward information and subsequent state information of the current action based on the reward function. The virtual environment is constructed from initial information, heat collection model, heat exchange model and prediction model. The reward function is obtained by integrating power consumption, temperature difference information, soil thermal balance information, the action smoothness of the execution device and peak and valley electricity price information. Based on the interaction information obtained by the intermediate agent interacting with the real environment, the intermediate agent is updated to obtain a control agent that meets preset conditions. The intermediate agent is obtained by updating the initial agent using the current state information, the current action, the reward information, and the subsequent state information.

Images