A primary network control method and device for a multi-heat source heating system including a high-temperature heat pump

Abstract

The present invention discloses a primary network control method for a multi-heat source heating system containing a high-temperature heat pump, comprising: determining a low-temperature, high-flow heat source and surrounding heat source objects according to the heating index of each heat source jurisdiction area, setting a high-temperature heat pump for the low-temperature, high-flow heat source to perform flow diversion and primary network temperature improvement, and determining the regional heating area that needs to be reduced in the jurisdiction area of ​​the low-temperature, high-flow heat source, and using the surrounding heat sources for collaborative heating to form a multi-heat source heating system containing a high-temperature heat pump; setting temperature control logic; setting a primary network temperature target value based on the heat demand of the area governed by the low-temperature, high-flow heat source, using the high-temperature heat pump operation data, the primary network temperature, and the high-temperature heat pump design performance as model inputs, using the compressor operation frequency, the diverter valve opening, and the bypass valve group switch state as model outputs, establishing a primary network temperature control model after model training through a hybrid algorithm of reinforcement learning and model predictive control, and outputting an optimal control strategy for the primary network temperature.

Description

Technical Field

[0001] The present invention belongs to the technical field of heating system control, and in particular relates to a primary network control method for a multi-heat source heating system including a high-temperature heat pump. Background Art

[0002] Central heating systems are a vital energy infrastructure that utilizes various types of available energy sources, extracts heat from them, and distributes it through a heating network to numerous heat users in urban areas to meet their heating needs. Central heating has rapidly developed and gained widespread application due to its advantages such as energy efficiency, multi-source coordination, and safe and stable heating quality.

[0003] Although heat users in different areas of a city are heated by different heat sources, and these sources can collaborate to provide heat, when some heat sources are responsible for areas with poor heating performance, low temperatures, and a high number of complaints from heat users, the water volume of some heat sources must be kept stable and difficult to adjust, making it impossible for other heat sources to coordinate and control. Therefore, improving the heating performance of the primary network of the heating system can be used to improve the heating effect. However, there is currently little research on improving the heating parameters of the primary network. While improving the heating parameters of surrounding heat sources can improve the heating of some areas of the primary network, it also leads to high energy consumption and low temperature control accuracy, resulting in excessively high temperatures in other areas.

[0004] Based on the above technical problems, it is necessary to design a new primary network control method for a multi-heat source heating system containing a high-temperature heat pump. Summary of the Invention

[0005] The technical problem to be solved by the present invention is to overcome the shortcomings of the existing technology and provide a primary network control method for a multi-heat source heating system containing a high-temperature heat pump. By diverting the low-temperature and large-flow heat source to reduce the water supply flow and coordinately adjust the heating areas of each heat source with other heat sources, and using the high-temperature heat pump to increase the temperature, the coordinated control of the heating of each heat source is achieved and the comfort of the heat users in the heating area is met. In addition, the use of the high-temperature heat pump can achieve clean heating and energy saving; and by adopting the reinforcement learning and model predictive control hybrid algorithm to establish a primary network temperature control model, the control accuracy can be improved.

[0006] In order to solve the above technical problems, the technical solution of the present invention is:

[0007] The present invention provides a primary network control method for a multi-heat source heating system including a high-temperature heat pump, comprising:

[0008] According to the heating indicators of the areas under the jurisdiction of each heat source, the low-temperature, high-flow heat source and the surrounding heat source objects are determined, and a high-temperature heat pump is set for the low-temperature, high-flow heat source to divert the flow and increase the temperature of the primary network. The regional heating area that needs to be reduced in the areas under the jurisdiction of the low-temperature, high-flow heat source is determined, and the surrounding heat sources increase the heating capacity and cooperate with the low-temperature, high-flow heat source to provide heat, forming a multi-heat source heating system including a high-temperature heat pump; the high-temperature heat pump includes an evaporator, a condenser, a compressor and a throttling device;

[0009] Set the temperature control logic:

[0010] The water supply side of the primary network of the low-temperature, high-flow heat source is diverted by switching the diverter valve and the bypass valve group to obtain a first water supply flow and a second water supply flow. By inputting electrical energy into the compressor as a circulating energy source, the refrigerant circulates inside the various components of the high-temperature heat pump and undergoes a phase change. The first water supply flow is then passed through the heat exchanger and evaporator to absorb heat and cool it down before flowing into the primary network return water. The second water supply flow is passed through the condenser to release heat and heat it up before being used as the primary network water supply to provide heat to the heat users in the area under the jurisdiction of the heat source.

[0011] Establish a heating control model:

[0012] The target value of the primary network temperature supply is set based on the heat demand of the area governed by the low-temperature and large-flow heat source. The evaporator inlet temperature, evaporator outlet temperature, evaporator water flow, condenser inlet temperature, condenser outlet temperature, condenser water flow, primary network temperature supply, and high-temperature heat pump design performance are used as model input variables. The compressor operating frequency, diverter valve opening, and bypass valve group switch status are used as model output variables. After model training through a hybrid algorithm of reinforcement learning and model predictive control, a primary network temperature supply control model is established, and the optimal control strategy for the primary network temperature supply is output.

[0013] Furthermore, the primary network control method for a multi-heat source heating system further includes establishing a digital twin model of the multi-heat source heating system including a high-temperature heat pump using a mechanism modeling and data identification method after forming the multi-heat source heating system including a high-temperature heat pump, specifically including:

[0014] Construct the physical entity model, logical model and simulation model of the heating pipeline;

[0015] The establishment of the logic model includes: establishing a controllable closed-loop logic model based on the logical mechanism relationship of the physical entities of the heating system including the heat source, the diverter valve, the plate heat exchanger, the high-temperature heat pump, and the heat users in the jurisdiction area, and mapping the physical model to the logic model; the high-temperature heat pump includes an evaporator, a condenser, a compressor, and a throttling device;

[0016] The establishment of the simulation model includes: building a simulation model of the heating system based on the collected operation data, status data, and physical property data of the heating system, and optimizing the parameters of the simulation model according to the error between the predicted value and the actual value output by the simulation model;

[0017] The physical entity model, the logical model and the simulation model are integrated into a virtual and real model to construct a system-level digital twin model of the physical entity of the heating system in a virtual space;

[0018] Connecting the multi-condition real-time operation data of the heating system to the system-level digital twin model, and using a reverse identification method to adaptively identify and correct the simulation results of the system-level digital twin model to obtain the identified and corrected digital twin model of the heating system;

[0019] And, after outputting the optimal control strategy for the primary network temperature supply, it also includes:

[0020] Verify the control strategy: Use the digital twin model of the heating system to simulate and analyze the optimal control strategy for the primary network temperature supply to verify whether the strategy meets the system heating indicators. If so, the strategy will be issued for execution; otherwise, the strategy will be readjusted until the system heating indicators are met.

[0021] Furthermore, the low-temperature, high-flow heat source and surrounding heat source objects are determined according to the heating indicators of the areas under the jurisdiction of each heat source, and a high-temperature heat pump is set for the low-temperature, high-flow heat source to perform flow diversion and primary network temperature improvement. The regional heating area that needs to be reduced in the area under the jurisdiction of the low-temperature, high-flow heat source is determined, and the surrounding heat sources increase the heating capacity and cooperate with the low-temperature, high-flow heat source to provide heating, forming a multi-heat source heating system with a high-temperature heat pump, including:

[0022] Analyze the water supply flow, water supply temperature and heating value curves of heat users in the jurisdiction of each heat source. When adjusting the water supply flow cannot improve the heating value, and the proportion of days with substandard heating value and water supply temperature below the set value exceeds the set threshold, the corresponding heat source will be used as a low-temperature, high-flow heat source, and the remaining heat sources will be used as peripheral heat sources. For the low-temperature, high-flow heat source, a high-temperature heat pump will be installed to divert the flow and improve the temperature of the primary network to improve the heating value. The heating area that needs to be reduced in the jurisdiction of the low-temperature, high-flow heat source will be determined, the heating area will be adjusted, and the heat user area in the reduced heat source jurisdiction will be heated by increasing the heating capacity of other peripheral heat sources with excess heat, forming a multi-heat source heating system with high-temperature heat pumps.

[0023] Among them, when adjusting the heating area, it also includes conducting a hydraulic analysis of the entire network using a hydraulic simulation model of the heating network to determine whether the heating network after adjusting the heating area meets the hydraulic balance requirements. If not, the area will be readjusted.

[0024] Furthermore, when establishing the heating control model, the primary network heating target value is set based on the regional heating demand under the jurisdiction of the low-temperature, high-flow heat source, including: based on the weather data, regional area, heat source heating capacity, and heating operation data of the regional heating operation under the jurisdiction of the low-temperature, high-flow heat source, a machine learning algorithm is used to establish a regional heating load prediction model, obtain the regional heating demand, and set the primary network heating target value that meets the regional heating demand.

[0025] Furthermore, when establishing the temperature control model, the evaporator inlet temperature, evaporator outlet temperature, evaporator water flow, condenser inlet temperature, condenser outlet temperature, condenser water flow, primary network temperature supply, and high-temperature heat pump design performance are used as model input variables, and the compressor operating frequency, diverter valve opening, and bypass valve group switch state are used as model output variables. After model training through a hybrid algorithm of reinforcement learning and model predictive control, a primary network temperature control model is established, and the optimal control strategy for the primary network temperature supply is output, including:

[0026] Collect influencing parameters of the primary network temperature control of the heating system, including evaporator inlet temperature, evaporator outlet temperature, evaporator water flow, condenser inlet temperature, condenser outlet temperature, condenser water flow, primary network temperature, high-temperature heat pump design performance, compressor operating frequency, diverter valve opening, and bypass valve group switch status;

[0027] A multi-agent deep reinforcement learning algorithm is used to establish a temperature control model for the first-level network of the heating system: two reinforcement learning controllers are set as agents to control the operating frequency of the compressor and the opening of the diverter valve respectively, multi-agent deep reinforcement learning is performed and a neural network is constructed. The input layer is the currently acquired environmental state; the middle layer is fully connected with all possible actions, and the output layer is the value estimation of all actions under the current environmental state. The action output by the agent controlling the operating frequency of the compressor is all the frequencies that the compressor can reach, the action output by the agent controlling the opening of the diverter valve is all the openings that the diverter valve can reach, and the action output by the agent controlling the switch state of the bypass valve group is the open and closed state of the bypass valve group switch; a replay memory unit is set to store all samples (s t ,a t ,r t ,s t+1 ), s t is the current environmental status, including evaporator inlet temperature, evaporator outlet temperature, evaporator water flow, condenser inlet temperature, condenser outlet temperature, condenser water flow, primary network temperature supply, and high-temperature heat pump design performance; a t The action under the current environmental conditions, including the compressor operating frequency, diverter valve opening and bypass valve group switch status; r t In the current environment state s t Next, perform action a tThe payback is expressed by the high-temperature heat pump operating power consumption and the regional heat user comfort; t+1 is the previous environment state s t Next, perform action a t The next state to which it migrates afterward; the reinforcement learning controller obtains the first control strategy through the deep Q learning algorithm;

[0028] A model predictive control method is used to establish a primary network temperature control model for the heating system: a neural network is used as the prediction model of the model predictive control, with the evaporator inlet temperature, evaporator outlet temperature, evaporator water flow, condenser inlet temperature, condenser outlet temperature, condenser water flow, primary network temperature, and high-temperature heat pump design performance as input variables of the neural network model, and the compressor operating frequency, diverter valve opening, and bypass valve group switch state as output variables of the neural network model. After the neural network is trained, the predicted values ​​of the compressor operating frequency, diverter valve opening, and bypass valve group switch state at the current moment are obtained. Through feedback correction and rolling optimization iteration, a second control strategy for future multi-step prediction is obtained;

[0029] The first control strategy and the second control strategy are integrated to obtain a fusion control mechanism. After the reinforcement learning controller is trained according to the fusion control mechanism, the temperature control is performed by taking the action output by the trained reinforcement learning controller as the optimal control strategy for the first-level network temperature supply.

[0030] Furthermore, the training process of the deep Q learning algorithm includes:

[0031] Initialize the replay memory unit to store training samples;

[0032] Initialize the current value network and randomly initialize the weight parameter w; initialize the target value network, whose structure and initialization weight are the same as the current value network;

[0033] The evaporator inlet temperature, evaporator outlet temperature, evaporator water flow, condenser inlet temperature, condenser outlet temperature, condenser water flow, primary network supply temperature and high-temperature heat pump design performance are passed through the current value network to obtain the value function Q(s,a) under any state s. The ∈greedy strategy is used to select the action a with the largest value function Q(s,a). Each state transition is recorded as a time step t. The data (s,a,r,s′) obtained at each time step is stored in the playback memory unit; s, a, r and s′ are the state, action, reward and next state respectively.

[0034] Define the loss function L(w), randomly extract a data (s, a, r, s′) from the replay memory unit, and transmit it to the current value network, target value network and loss function respectively. The loss function L(w) is updated using the stochastic gradient descent method with respect to the initialized weight parameter w.

[0035] Furthermore, when feedback correction is performed in the model predictive control method, after executing the control strategy for the compressor operating frequency, diverter valve opening, and bypass valve group switch state at the current moment, the error between the current primary network temperature actual value and the primary network temperature target value is used as a feedback correction amount to obtain the corrected control strategy for the compressor operating frequency, diverter valve opening, and bypass valve group switch state at the next moment;

[0036] When performing rolling optimization iterations in the model predictive control method, the output value at the current moment is processed using a softening coefficient method, and a rolling optimization performance objective function is established. After solving the function, an optimized control strategy is obtained.

[0037] Furthermore, the softening coefficient method is expressed as:

[0038] p r (k+j)=a r p(k)+(1-a r )p s ;

[0039] p r (k+j) is the reference output value at time (k+j); a r is the softening coefficient; p(k) is the actual output value at the current moment; p s is the output set value; j is the prediction step length;

[0040] The rolling optimization performance objective function is established as follows:

[0041]

[0042] p c (k+j) is the output value after feedback correction; n is the maximum prediction length; m is the control length; λ is the control weighting coefficient; Δu(k) is the increment of the output value at the current moment.

[0043] Furthermore, the first control strategy and the second control strategy are integrated to obtain a fusion control mechanism, and after training the reinforcement learning controller according to the fusion control mechanism, the temperature control is performed by using the action output by the trained reinforcement learning controller as the optimal control strategy for the primary network temperature supply, including:

[0044] The first control strategy and the second control strategy are integrated to obtain a fusion control mechanism, which is expressed as:

[0045] u(t)=g(t)*a1(t)+(1-g(t))*a2(t);

[0046] g(t) is the action weight function at time t; a1(t) is the first control strategy; a2(t) is the second control strategy;

[0047] The reinforcement learning controller interacts and updates according to the fusion control mechanism u(t). During the interaction, the model predictive control and reinforcement learning controller will implement control according to the fusion control mechanism. The process will give state transitions and store the resulting data tuples in the replay memory unit. During the update process, the samples in the replay memory unit are sampled to update the network parameters of the reinforcement learning controller.

[0048] The updated parameters are input into the reinforcement learning controller network, and the output compressor operating frequency and diverter valve opening are used as the optimal control strategy for the primary network temperature supply to control the temperature.

[0049] The present invention further provides a primary network control device for a multi-heat source heating system including a high-temperature heat pump, the primary network control device for the multi-heat source heating system comprising:

[0050] A multi-heat source heating system establishment unit including a high-temperature heat pump is used to determine low-temperature, high-flow heat sources and surrounding heat source objects based on the heating indicators of the areas under the jurisdiction of each heat source, set high-temperature heat pumps for the low-temperature, high-flow heat sources to divert flow and increase the temperature of the primary network, and determine the regional heating area that needs to be reduced in the areas under the jurisdiction of the low-temperature, high-flow heat sources. The surrounding heat sources increase the heating capacity and cooperate with the low-temperature, high-flow heat sources to provide heating, thereby forming a multi-heat source heating system including a high-temperature heat pump; the high-temperature heat pump includes an evaporator, a condenser, a compressor and a throttling device;

[0051] The temperature control logic setting unit is used to divert the water supply side of the primary network of the low-temperature, high-flow heat source through the diverter valve and the bypass valve group switch to obtain a first water supply flow and a second water supply flow; by inputting electrical energy into the compressor as a circulating energy source, the refrigerant circulates inside the various components of the high-temperature heat pump and undergoes a phase change, and then the first water supply flow is passed through the heat exchanger and evaporator to absorb heat and cool it before flowing into the primary network return water, and the second water supply flow is passed through the condenser to release heat and heat it before serving as the primary network water supply to provide heat to the heat users in the area under the jurisdiction of the heat source;

[0052] The temperature control model establishment unit is used to set the primary network temperature target value based on the heat demand of the area governed by the low-temperature and large-flow heat source. The evaporator inlet temperature, evaporator outlet temperature, evaporator water flow, condenser inlet temperature, condenser outlet temperature, condenser water flow, primary network temperature, and high-temperature heat pump design performance are used as model input variables, and the compressor operating frequency, diverter valve opening, and bypass valve group switch status are used as model output variables. After model training through the hybrid algorithm of reinforcement learning and model predictive control, the primary network temperature control model is established, and the optimal control strategy for the primary network temperature is output.

[0053] The beneficial effects of the present invention are:

[0054] The present invention determines low-temperature, high-flow heat sources and surrounding heat source objects based on the heating indicators of the areas under the jurisdiction of each heat source, sets a high-temperature heat pump for the low-temperature, high-flow heat source to perform flow diversion and primary network temperature improvement, and determines the regional heating area that needs to be reduced in the area under the jurisdiction of the low-temperature, high-flow heat source. The surrounding heat sources increase the heating capacity and cooperate with the low-temperature, high-flow heat source to provide heat, thereby forming a multi-heat source heating system including a high-temperature heat pump; the high-temperature heat pump includes an evaporator, a condenser, a compressor and a throttling device; a temperature control logic is set: the water supply side of the primary network of the low-temperature, high-flow heat source is diverted by switching the diverter valve and the bypass valve group to obtain a first water supply flow and a second water supply flow; by inputting electrical energy into the compressor as a circulating energy source, the refrigerant circulates inside the various components of the high-temperature heat pump and undergoes phase change, the first water supply flow is passed through the heat exchanger and evaporator to absorb heat and cool down, and then flows into the primary network return water, and the second water supply flow is passed through the condenser to release heat and heat up, and then serves as the primary network water supply, supplying heat users in the area under the jurisdiction of the heat source. Heat; Establish a temperature control model: Set the target value of the primary network temperature supply based on the heat demand of the area governed by the low-temperature, high-flow heat source, use the evaporator inlet temperature, evaporator outlet temperature, evaporator water flow, condenser inlet temperature, condenser outlet temperature, condenser water flow, primary network temperature supply, and high-temperature heat pump design performance as model input variables, and use the compressor operating frequency, diverter valve opening, and bypass valve group switch state as model output variables. After model training through a hybrid algorithm of reinforcement learning and model predictive control, a primary network temperature control model is established, and the optimal control strategy for the primary network temperature supply is output; by diverting the low-temperature, high-flow heat source to reduce the water flow, and coordinating with other heat sources to adjust the heating areas of each heat source, and using high-temperature heat pumps to increase the temperature, the coordinated control of the heat supply of each heat source is achieved and the comfort of the heat users in the heating area is met. In addition, the use of high-temperature heat pumps can achieve clean heating and save energy; and by adopting a hybrid algorithm of reinforcement learning and model predictive control to establish a primary network temperature control model, the control accuracy can be improved.

[0055] Other features and advantages will be described in the following description, and in part will become apparent from the description, or understood by practicing the invention. The purpose and other advantages of the invention are realized and obtained by the structures particularly pointed out in the description and the drawings.

[0056] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, preferred embodiments are given below and described in detail with reference to the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0057] In order to more clearly illustrate the specific embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the specific embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0058] Figure 1 This is a flow chart of a primary network control method for a multi-heat source heating system including a high-temperature heat pump according to the present invention;

[0059] Figure 2 This is a schematic diagram of the control principle of the primary heating network containing a high-temperature heat pump according to the present invention;

[0060] Figure 3 This is a structural diagram of the primary network control device of a multi-heat source heating system containing a high-temperature heat pump according to the present invention. DETAILED DESCRIPTION

[0061] To make the objectives, technical solutions, and advantages of the embodiments of the present invention more clear, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the embodiments described are only part of the embodiments of the present invention, not all of them. All other embodiments obtained by ordinary technicians in this field based on the embodiments of the present invention without making any creative efforts shall fall within the scope of protection of the present invention.

[0062] Example 1

[0063] Figure 1 The present invention relates to a flow chart of a primary network control method for a multi-heat source heating system including a high-temperature heat pump.

[0064] Figure 2 It is a schematic diagram of the control principle of the primary heating network containing a high-temperature heat pump involved in the present invention.

[0065] like Figure 1 、 2 As shown, this embodiment 1 provides a primary network control method for a multi-heat source heating system including a high-temperature heat pump, comprising:

[0066] According to the heating indicators of the areas under the jurisdiction of each heat source, the low-temperature, high-flow heat source and the surrounding heat source objects are determined, and a high-temperature heat pump is set for the low-temperature, high-flow heat source to divert the flow and increase the temperature of the primary network. The regional heating area that needs to be reduced in the areas under the jurisdiction of the low-temperature, high-flow heat source is determined, and the surrounding heat sources increase the heating capacity and cooperate with the low-temperature, high-flow heat source to provide heat, forming a multi-heat source heating system including a high-temperature heat pump; the high-temperature heat pump includes an evaporator, a condenser, a compressor and a throttling device;

[0067] Set the temperature control logic:

[0068] The water supply side of the primary network of the low-temperature, high-flow heat source is diverted by switching the diverter valve and the bypass valve group to obtain a first water supply flow and a second water supply flow. By inputting electrical energy into the compressor as a circulating energy source, the refrigerant circulates inside the various components of the high-temperature heat pump and undergoes a phase change. The first water supply flow is then passed through the heat exchanger and evaporator to absorb heat and cool it down before flowing into the primary network return water. The second water supply flow is passed through the condenser to release heat and heat it up before being used as the primary network water supply to provide heat to the heat users in the area under the jurisdiction of the heat source.

[0069] Establish a heating control model:

[0070] The target value of the primary network temperature supply is set based on the heat demand of the area governed by the low-temperature and large-flow heat source. The evaporator inlet temperature, evaporator outlet temperature, evaporator water flow, condenser inlet temperature, condenser outlet temperature, condenser water flow, primary network temperature supply, and high-temperature heat pump design performance are used as model input variables. The compressor operating frequency, diverter valve opening, and bypass valve group switch status are used as model output variables. After model training through a hybrid algorithm of reinforcement learning and model predictive control, a primary network temperature supply control model is established, and the optimal control strategy for the primary network temperature supply is output.

[0071] In actual applications, a low-temperature, high-flow heat source can be understood as an externally purchased heat source, but it is not limited to being an externally purchased heat source. Usually, an externally purchased heat source must take into account AGC regulation, resulting in substandard temperature supply, while the own heat source will basically achieve temperature supply standards through gas heating. In addition, an externally purchased heat source does not mean that the temperature supply will not meet the standards under all circumstances. When the standards are met, it means that there is no low-temperature, high-flow heat source. Only when the standards are not met will the temperature be regulated by a high-temperature heat pump. For example, when the heat source temperature meets the standards, valve V2 is opened and V1 is closed; when the heat source temperature does not meet the standards, valve V2 is closed and V1 is opened. At the same time, the diverter valve also needs to be adjusted accordingly to divert the water supply flow.

[0072] In this embodiment, the primary network control method for a multi-heat source heating system further includes establishing a digital twin model of the multi-heat source heating system including a high-temperature heat pump using a mechanism modeling and data identification method after forming the multi-heat source heating system including a high-temperature heat pump, specifically including:

[0073] Construct the physical entity model, logical model and simulation model of the heating pipeline;

[0074] The establishment of the logic model includes: establishing a controllable closed-loop logic model based on the logical mechanism relationship of the physical entities of the heating system including the heat source, the diverter valve, the plate heat exchanger, the high-temperature heat pump, and the heat users in the jurisdiction area, and mapping the physical model to the logic model; the high-temperature heat pump includes an evaporator, a condenser, a compressor, and a throttling device;

[0075] The establishment of the simulation model includes: building a simulation model of the heating system based on the collected operation data, status data, and physical property data of the heating system, and optimizing the parameters of the simulation model according to the error between the predicted value and the actual value output by the simulation model;

[0076] The physical entity model, the logical model and the simulation model are integrated into a virtual and real model to construct a system-level digital twin model of the physical entity of the heating system in a virtual space;

[0077] Connecting the multi-condition real-time operation data of the heating system to the system-level digital twin model, and using a reverse identification method to adaptively identify and correct the simulation results of the system-level digital twin model to obtain the identified and corrected digital twin model of the heating system;

[0078] And, after outputting the optimal control strategy for the primary network temperature supply, it also includes:

[0079] Verify the control strategy: Use the digital twin model of the heating system to simulate and analyze the optimal control strategy for the primary network temperature supply to verify whether the strategy meets the system heating indicators. If so, the strategy will be issued for execution; otherwise, the strategy will be readjusted until the system heating indicators are met.

[0080] In this embodiment, the low-temperature, high-flow heat source and surrounding heat source objects are determined according to the heating indicators of the areas under the jurisdiction of each heat source, a high-temperature heat pump is set for the low-temperature, high-flow heat source to perform flow diversion and primary network temperature improvement, and the regional heating area that needs to be reduced in the areas under the jurisdiction of the low-temperature, high-flow heat source is determined. The surrounding heat sources increase the heating capacity and cooperate with the low-temperature, high-flow heat source to provide heating, forming a multi-heat source heating system including a high-temperature heat pump, including:

[0081] Analyze the water supply flow, water supply temperature and heating value curves of heat users in the jurisdiction of each heat source. When adjusting the water supply flow cannot improve the heating value, and the proportion of days with substandard heating value and water supply temperature below the set value exceeds the set threshold, the corresponding heat source will be used as a low-temperature, high-flow heat source, and the remaining heat sources will be used as peripheral heat sources. For the low-temperature, high-flow heat source, a high-temperature heat pump will be installed to divert the flow and improve the temperature of the primary network to improve the heating value. The heating area that needs to be reduced in the jurisdiction of the low-temperature, high-flow heat source will be determined, the heating area will be adjusted, and the heat user area in the reduced heat source jurisdiction will be heated by increasing the heating capacity of other peripheral heat sources with excess heat, forming a multi-heat source heating system with high-temperature heat pumps.

[0082] Among them, when adjusting the heating area, it also includes conducting a hydraulic analysis of the entire network using a hydraulic simulation model of the heating network to determine whether the heating network after adjusting the heating area meets the hydraulic balance requirements. If not, the area will be readjusted.

[0083] In actual applications, when the heating supply to heat users in some areas does not meet the standards and the number of complaints from heat users increases, it is necessary to regulate the water supply flow in the area. However, sometimes adjusting the water supply flow cannot effectively improve the heating situation, and regulating the heat source is relatively difficult. Reducing the area of ​​the heating area, reducing the water supply flow, and using high-temperature heat pumps to increase the temperature can effectively improve the heating supply.

[0084] After using high-temperature heat pump diversion to improve the primary network heating, the actual hot water flow of the low-temperature, high-flow heat source will drop significantly, and the corresponding regional heat user area will decrease. It is necessary to determine the regional heat user area that needs to be reduced based on the heating indicators of the areas under the jurisdiction of each heat source, and increase the heating capacity of other surrounding heat sources and coordinate heating with low-temperature, high-flow heat sources to form a multi-heat source heating system with high-temperature heat pumps.

[0085] A heat pump is an energy utilization system that consumes a portion of high-quality energy. Through the refrigerant's condensation and evaporation processes, it transfers heat from a low-temperature object to a high-temperature object, converting low-level energy into high-level energy. Heat pump technology raises the temperature of low-temperature waste heat and redistributes it to heat-using equipment, saving energy consumption, such as steam. Heat pumps are broadly categorized into low-temperature, medium-temperature, and high-temperature heat pumps, depending on the temperature range they reach. A high-temperature heat pump consists of four main components: a compressor, a condenser, a throttling device, and an evaporator, all connected in sequence by copper tubing to form a closed circulation system. The compressor is the core component of the entire system and the source of its power. The evaporator and condenser, collectively known as heat exchangers, serve as the medium for heat exchange between the system and the outside world. The throttling device is a flow device within the pipeline. The refrigerant fluid forms a localized constriction at the throttling point, increasing the flow rate and reducing the static pressure, creating a pressure differential within the fluid.

[0086] In this embodiment, when establishing the heating control model, the primary network heating target value is set based on the regional heating demand under the jurisdiction of the low-temperature, high-flow heat source, including: based on the weather data, regional area, heat source heating capacity, and heating operation data of the regional heating operation under the jurisdiction of the low-temperature, high-flow heat source, a machine learning algorithm is used to establish a regional heating load prediction model, obtain the regional heating demand, and set the primary network heating target value that meets the regional heating demand.

[0087] It should be noted that due to the phased changes in regional heat demand, the primary network temperature target value also changes in phases.

[0088] In this embodiment, when establishing the temperature control model, the evaporator inlet temperature, evaporator outlet temperature, evaporator water flow, condenser inlet temperature, condenser outlet temperature, condenser water flow, primary network temperature, and high-temperature heat pump design performance are used as model input variables, and the compressor operating frequency, diverter valve opening, and bypass valve group switch state are used as model output variables. After model training using a hybrid algorithm of reinforcement learning and model predictive control, a primary network temperature control model is established, and the optimal control strategy for the primary network temperature is output, including:

[0089] Collect influencing parameters of the primary network temperature control of the heating system, including evaporator inlet temperature, evaporator outlet temperature, evaporator water flow, condenser inlet temperature, condenser outlet temperature, condenser water flow, primary network temperature, high-temperature heat pump design performance, compressor operating frequency, diverter valve opening, and bypass valve group switch status;

[0090] A multi-agent deep reinforcement learning algorithm is used to establish a temperature control model for the first-level network of the heating system: two reinforcement learning controllers are set as agents to control the operating frequency of the compressor and the opening of the diverter valve respectively, multi-agent deep reinforcement learning is performed and a neural network is constructed. The input layer is the currently acquired environmental state; the middle layer is fully connected with all possible actions, and the output layer is the value estimation of all actions under the current environmental state. The action output by the agent controlling the operating frequency of the compressor is all the frequencies that the compressor can reach, the action output by the agent controlling the opening of the diverter valve is all the openings that the diverter valve can reach, and the action output by the agent controlling the switch state of the bypass valve group is the open and closed state of the bypass valve group switch; a replay memory unit is set to store all samples (s t ,a t ,r t ,s t+1 ), s t is the current environmental status, including evaporator inlet temperature, evaporator outlet temperature, evaporator water flow, condenser inlet temperature, condenser outlet temperature, condenser water flow, primary network temperature supply, and high-temperature heat pump design performance; a t The action under the current environmental conditions, including the compressor operating frequency, diverter valve opening and bypass valve group switch status; r t In the current environment state s t Next, perform action a t The payback is expressed by the high-temperature heat pump operating power consumption and the regional heat user comfort; t+1 is the previous environment state s t Next, perform action a t The next state to which it migrates afterward; the reinforcement learning controller obtains the first control strategy through the deep Q learning algorithm;

[0091] A model predictive control method is used to establish a primary network temperature control model for the heating system: a neural network is used as the prediction model of the model predictive control, with the evaporator inlet temperature, evaporator outlet temperature, evaporator water flow, condenser inlet temperature, condenser outlet temperature, condenser water flow, primary network temperature, and high-temperature heat pump design performance as input variables of the neural network model, and the compressor operating frequency, diverter valve opening, and bypass valve group switch state as output variables of the neural network model. After the neural network is trained, the predicted values ​​of the compressor operating frequency, diverter valve opening, and bypass valve group switch state at the current moment are obtained. Through feedback correction and rolling optimization iteration, a second control strategy for future multi-step prediction is obtained;

[0092] The first control strategy and the second control strategy are integrated to obtain a fusion control mechanism. After the reinforcement learning controller is trained according to the fusion control mechanism, the temperature control is performed by taking the action output by the trained reinforcement learning controller as the optimal control strategy for the first-level network temperature supply.

[0093] It's important to note that the learning process of reinforcement learning is a dynamic, interactive one, and the data required is generated through continuous interaction with the environment. The framework that can solve most reinforcement learning problems is called a Markov decision process. By formulating the primary grid heating control problem as a Markov decision process, it possesses the Markov property required by reinforcement learning models. The Markov property states that the next state of a system depends only on the current state and not on the previous state. Once the current state is known, historical information is discarded. Therefore, a reinforcement learning controller is chosen, and the heating system operation is treated as a Markov decision process. The primary grid heating control problem can then be rationally formulated as a reinforcement learning problem, allowing the reinforcement learning algorithm to improve the control strategy. At each time step in a Markov decision process, the agent's observations include the state, action, and reward function. The system takes an action in the current state and transitions to the next state via a transition function. The agent attempts to maximize the cumulative reward received from the environment. The ultimate goal of reinforcement learning is for the agent to find an action sequence or strategy that maximizes this cumulative reward. Therefore, the problem of temperature control in the primary network is to find the optimal control strategy so that the reward obtained in the entire optimization range is maximized, that is, the high-temperature heat pump has the lowest power consumption and the regional heat user comfort is the highest.

[0094] The calculation of high-temperature heat pump operation power consumption is mainly based on the heat and high-temperature heat pump design performance; the user comfort is determined by the error between the actual value of the primary network temperature supply and the primary network temperature supply target value. Assuming that the primary network temperature supply target value t g The comfort level of users in the lower area is the highest. Since the temperature of the primary network fluctuates within the set range, it will not affect the comfort level. Therefore, when the temperature of the primary network is greater than t g +Δt, the primary network temperature is too high, the user comfort is reduced, and when the primary network temperature is less than tg When -Δt, the temperature supplied by the primary network is too low and the user comfort level is reduced.

[0095] In this embodiment, the training process of the deep Q learning algorithm includes:

[0096] Initialize the replay memory unit to store training samples;

[0097] Initialize the current value network and randomly initialize the weight parameter w; initialize the target value network, whose structure and initialization weight are the same as the current value network;

[0098] The evaporator inlet temperature, evaporator outlet temperature, evaporator water flow, condenser inlet temperature, condenser outlet temperature, condenser water flow, primary network supply temperature and high-temperature heat pump design performance are passed through the current value network to obtain the value function Q(s,a) under any state s. The ∈greedy strategy is used to select the action a with the largest value function Q(s,a). Each state transition is recorded as a time step t. The data (s,a,r,s′) obtained at each time step is stored in the playback memory unit; s, a, r and s′ are the state, action, reward and next state respectively.

[0099] Define the loss function L(w), randomly extract a data (s, a, r, s′) from the replay memory unit, and transmit it to the current value network, target value network and loss function respectively. The loss function L(w) is updated using the stochastic gradient descent method with respect to the initialized weight parameter w.

[0100] In this embodiment, when feedback correction is performed in the model predictive control method, after executing the control strategy for the compressor operating frequency, diverter valve opening, and bypass valve group switch state at the current moment, the error between the current primary network temperature actual value and the primary network temperature target value is used as the feedback correction amount to obtain the corrected control strategy for the compressor operating frequency, diverter valve opening, and bypass valve group switch state at the next moment;

[0101] When performing rolling optimization iterations in the model predictive control method, the output value at the current moment is processed using a softening coefficient method, and a rolling optimization performance objective function is established. After solving the function, an optimized control strategy is obtained.

[0102] In this embodiment, the softening coefficient method is expressed as:

[0103] p r (k+j)=a r p(k)+(1-a r )p s ;

[0104] p r(k+j) is the reference output value at time (k+j); a r is the softening coefficient; p(k) is the actual output value at the current moment; p s is the output set value; j is the prediction step length;

[0105] The rolling optimization performance objective function is established as follows:

[0106]

[0107] p c (k+j) is the output value after feedback correction; n is the maximum prediction length; m is the control length; λ is the control weighting coefficient; Δu(k) is the increment of the output value at the current moment.

[0108] It should be noted that the main principles of model predictive control are prediction model, rolling optimization and feedback correction.

[0109] Predictive Model: Designing a model predictive controller first requires establishing a predictive model for the system. Once this model is established, the controller can predict the future state and output of the controlled system based on the system's current state and the control variables in the future control time domain. Subtracting the predicted output from a given expected reference trajectory can be used for the next step of rolling optimization.

[0110] Rolling Optimization: At each sampling moment, the model prediction algorithm solves a constrained target optimization function. This function is a metric related to the control and prediction horizons, and it rolls over a finite horizon. Model predictive control optimization is not accomplished in a single offline calculation; instead, it requires rolling optimization, requiring multiple, iterative online solutions to achieve the final optimization goal.

[0111] Feedback correction: Correct and optimize the model's predicted value by calculating the error between the target value and the actual object measurement value to eliminate interference from external disturbances and model adaptation.

[0112] In this embodiment, the first control strategy and the second control strategy are integrated to obtain a fusion control mechanism, and after training the reinforcement learning controller according to the fusion control mechanism, temperature control is performed by using the actions output by the trained reinforcement learning controller as the optimal control strategy for primary network temperature supply, including:

[0113] The first control strategy and the second control strategy are integrated to obtain a fusion control mechanism, which is expressed as:

[0114] u(t)=g(t)*a1(t)+(1-g(t))*a2(t);

[0115] g(t) is the action weight function at time t; a1(t) is the first control strategy; a2(t) is the second control strategy;

[0116] The reinforcement learning controller interacts and updates according to the fusion control mechanism u(t). During the interaction, the model predictive control and reinforcement learning controller will implement control according to the fusion control mechanism. The process will give state transitions and store the resulting data tuples in the replay memory unit. During the update process, the samples in the replay memory unit are sampled to update the network parameters of the reinforcement learning controller.

[0117] The updated parameters are input into the reinforcement learning controller network, and the output compressor operating frequency and diverter valve opening are used as the optimal control strategy for the primary network temperature supply to control the temperature.

[0118] It should be noted that a primary network temperature control model is established after model training using a hybrid algorithm of reinforcement learning and model predictive control. During the training phase, model predictive control is used to guide the reinforcement learning actions. By providing high-quality samples, the training is accelerated, while ensuring that the control performance of reinforcement learning is better than that of traditional model predictive control and reinforcement learning control schemes.

[0119] Example 2

[0120] Figure 3 This is a structural diagram of a primary network control device of a multi-heat source heating system containing a high-temperature heat pump involved in the present invention.

[0121] like Figure 3 As shown, this embodiment 2 provides a primary network control device for a multi-heat source heating system including a high-temperature heat pump, and the primary network control device for the multi-heat source heating system includes:

[0122] A multi-heat source heating system establishment unit including a high-temperature heat pump is used to determine low-temperature, high-flow heat sources and surrounding heat source objects based on the heating indicators of the areas under the jurisdiction of each heat source, set high-temperature heat pumps for the low-temperature, high-flow heat sources to divert flow and increase the temperature of the primary network, and determine the regional heating area that needs to be reduced in the areas under the jurisdiction of the low-temperature, high-flow heat sources. The surrounding heat sources increase the heating capacity and cooperate with the low-temperature, high-flow heat sources to provide heating, thereby forming a multi-heat source heating system including a high-temperature heat pump; the high-temperature heat pump includes an evaporator, a condenser, a compressor and a throttling device;

[0123] The temperature control logic setting unit is used to divert the water supply side of the primary network of the low-temperature, high-flow heat source through the diverter valve and the bypass valve group switch to obtain a first water supply flow and a second water supply flow; by inputting electrical energy into the compressor as a circulating energy source, the refrigerant circulates inside the various components of the high-temperature heat pump and undergoes a phase change, and then the first water supply flow is passed through the heat exchanger and evaporator to absorb heat and cool it before flowing into the primary network return water, and the second water supply flow is passed through the condenser to release heat and heat it before serving as the primary network water supply to provide heat to the heat users in the area under the jurisdiction of the heat source;

[0124] The temperature control model establishment unit is used to set the primary network temperature target value based on the heat demand of the area governed by the low-temperature and large-flow heat source. The evaporator inlet temperature, evaporator outlet temperature, evaporator water flow, condenser inlet temperature, condenser outlet temperature, condenser water flow, primary network temperature, and high-temperature heat pump design performance are used as model input variables, and the compressor operating frequency, diverter valve opening, and bypass valve group switch status are used as model output variables. After model training through the hybrid algorithm of reinforcement learning and model predictive control, the primary network temperature control model is established, and the optimal control strategy for the primary network temperature is output.

[0125] In the several embodiments provided in this application, it should be understood that the disclosed systems and methods can also be implemented in other ways. The system embodiments described above are merely illustrative. For example, the flowcharts and block diagrams in the accompanying drawings show the possible architectures, functions, and operations of the systems, methods, and computer program products according to multiple embodiments of the present invention. In this regard, each box in the flowchart or block diagram can represent a module, a program segment, or a portion of code, and the module, program segment, or a portion of code contains one or more executable instructions for implementing the specified logical functions. It should also be noted that in some alternative implementations, the functions marked in the boxes can also occur in an order different from that marked in the accompanying drawings. For example, two consecutive boxes can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram and / or flowchart, and the combination of boxes in the block diagram and / or flowchart, can be implemented using a dedicated hardware-based system that performs the specified functions or actions, or can be implemented using a combination of dedicated hardware and computer instructions.

[0126] In addition, the functional modules in each embodiment of the present invention can be integrated together to form an independent part, or each module can exist separately, or two or more modules can be integrated to form an independent part. If the function is implemented in the form of a software functional module and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, or the part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes several instructions for enabling a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the methods of each embodiment of the present invention. The aforementioned storage medium includes various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk.

[0127] With the above-described preferred embodiments of the present invention as a guide, and with reference to the above description, relevant personnel are fully capable of making various changes and modifications without departing from the technical scope of this invention. The technical scope of this invention is not limited to the contents of the specification and must be determined according to the scope of the claims.

Claims

1. A primary network control method for a multi-heat source heating system including a high-temperature heat pump, characterized in that: include: According to the heating indicators of the areas under the jurisdiction of each heat source, the low-temperature, high-flow heat source and the surrounding heat source objects are determined, and a high-temperature heat pump is set for the low-temperature, high-flow heat source to divert the flow and increase the temperature of the primary network. The regional heating area that needs to be reduced in the areas under the jurisdiction of the low-temperature, high-flow heat source is determined, and the surrounding heat sources increase the heating capacity and cooperate with the low-temperature, high-flow heat source to provide heat, forming a multi-heat source heating system including a high-temperature heat pump; the high-temperature heat pump includes an evaporator, a condenser, a compressor and a throttling device; Set the temperature control logic: The water supply side of the primary network of the low-temperature, high-flow heat source is diverted by switching the diverter valve and the bypass valve group to obtain a first water supply flow and a second water supply flow. By inputting electrical energy into the compressor as a circulating energy source, the refrigerant circulates inside the various components of the high-temperature heat pump and undergoes a phase change. The first water supply flow is then passed through the heat exchanger and evaporator to absorb heat and cool it down before flowing into the primary network return water. The second water supply flow is passed through the condenser to release heat and heat it up before being used as the primary network water supply to provide heat to the heat users in the area under the jurisdiction of the heat source. Establish a heating control model: The target value of the primary network temperature supply is set based on the heat demand of the area governed by the low-temperature, high-flow heat source. The evaporator inlet temperature, evaporator outlet temperature, evaporator water flow, condenser inlet temperature, condenser outlet temperature, condenser water flow, primary network temperature supply, and high-temperature heat pump design performance are used as model input variables. The compressor operating frequency, diverter valve opening, and bypass valve group switch status are used as model output variables. After model training through a hybrid algorithm of reinforcement learning and model predictive control, a primary network temperature supply control model is established, and the optimal control strategy for the primary network temperature supply is output, including: Collect influencing parameters of the primary network temperature control of the heating system, including evaporator inlet temperature, evaporator outlet temperature, evaporator water flow, condenser inlet temperature, condenser outlet temperature, condenser water flow, primary network temperature, high-temperature heat pump design performance, compressor operating frequency, diverter valve opening, and bypass valve group switch status; A multi-agent deep reinforcement learning algorithm is used to establish a temperature control model for the first-level network of the heating system: two reinforcement learning controllers are set as agents to control the operating frequency of the compressor and the opening of the diverter valve respectively, multi-agent deep reinforcement learning is performed and a neural network is constructed. The input layer is the currently acquired environmental state; the middle layer is fully connected with all possible actions, and the output layer is the value estimation of all actions under the current environmental state. The action output by the agent controlling the operating frequency of the compressor is all the frequencies that the compressor can reach, the action output by the agent controlling the opening of the diverter valve is all the openings that the diverter valve can reach, and the action output by the agent controlling the switch state of the bypass valve group is the open and closed state of the bypass valve group switch; a replay memory unit is set to store all samples (s t ,a t ,r t ,s t+1 ), s t is the current environmental status, including evaporator inlet temperature, evaporator outlet temperature, evaporator water flow, condenser inlet temperature, condenser outlet temperature, condenser water flow, primary network temperature supply, and high-temperature heat pump design performance; a t The action under the current environmental conditions, including the compressor operating frequency, diverter valve opening and bypass valve group switch status; r t In the current environment state s t Next, perform action a t The payback is expressed by the high-temperature heat pump operating power consumption and the regional heat user comfort; t+1 is the previous environment state s t Next, perform action a t The next state to which it migrates afterward; the reinforcement learning controller obtains the first control strategy through the deep Q learning algorithm; A model predictive control method is used to establish a primary network temperature control model for the heating system: a neural network is used as the prediction model of the model predictive control, with the evaporator inlet temperature, evaporator outlet temperature, evaporator water flow, condenser inlet temperature, condenser outlet temperature, condenser water flow, primary network temperature, and high-temperature heat pump design performance as input variables of the neural network model, and the compressor operating frequency, diverter valve opening, and bypass valve group switch state as output variables of the neural network model. After the neural network is trained, the predicted values ​​of the compressor operating frequency, diverter valve opening, and bypass valve group switch state at the current moment are obtained. Through feedback correction and rolling optimization iteration, a second control strategy for future multi-step prediction is obtained; The first control strategy and the second control strategy are integrated to obtain a fusion control mechanism. After the reinforcement learning controller is trained according to the fusion control mechanism, the temperature control is performed by taking the action output by the trained reinforcement learning controller as the optimal control strategy for the first-level network temperature supply.

2. The primary network control method for a multi-heat source heating system according to claim 1, characterized in that: The primary network control method for a multi-heat source heating system further includes, after forming a multi-heat source heating system including a high-temperature heat pump, establishing a digital twin model of the multi-heat source heating system including a high-temperature heat pump using a mechanism modeling and data identification method, specifically including: Construct the physical entity model, logical model and simulation model of the heating pipeline; The establishment of the logic model includes: establishing a controllable closed-loop logic model based on the logical mechanism relationship of the physical entities of the heating system including the heat source, the diverter valve, the plate heat exchanger, the high-temperature heat pump, and the heat users in the jurisdiction area, and mapping the physical model to the logic model; the high-temperature heat pump includes an evaporator, a condenser, a compressor, and a throttling device; The establishment of the simulation model includes: building a simulation model of the heating system based on the collected operation data, status data, and physical property data of the heating system, and optimizing the parameters of the simulation model according to the error between the predicted value and the actual value output by the simulation model; The physical entity model, the logical model and the simulation model are integrated into a virtual and real model to construct a system-level digital twin model of the physical entity of the heating system in a virtual space; Connecting the multi-condition real-time operation data of the heating system to the system-level digital twin model, and using a reverse identification method to adaptively identify and correct the simulation results of the system-level digital twin model to obtain the identified and corrected digital twin model of the heating system; And, after outputting the optimal control strategy for the primary network temperature supply, it also includes: Verify the control strategy: Use the digital twin model of the heating system to simulate and analyze the optimal control strategy for the primary network temperature supply to verify whether the strategy meets the system heating indicators. If so, the strategy will be issued for execution; otherwise, the strategy will be readjusted until the system heating indicators are met.

3. The primary network control method for a multi-heat source heating system according to claim 1, characterized in that: The method comprises determining low-temperature, high-flow heat sources and surrounding heat sources according to the heating indicators of the areas under the jurisdiction of each heat source, setting a high-temperature heat pump for the low-temperature, high-flow heat source to divert flow and increase the temperature of the primary network, and determining the regional heating area that needs to be reduced in the areas under the jurisdiction of the low-temperature, high-flow heat source. The surrounding heat sources increase the heating capacity and cooperate with the low-temperature, high-flow heat source to provide heating, thereby forming a multi-heat source heating system including a high-temperature heat pump, including: Analyze the water supply flow, water supply temperature and heating value curves of heat users in the jurisdiction of each heat source. When adjusting the water supply flow cannot improve the heating value, and the proportion of days with substandard heating value and water supply temperature below the set value exceeds the set threshold, the corresponding heat source will be used as a low-temperature, high-flow heat source, and the remaining heat sources will be used as peripheral heat sources. For the low-temperature, high-flow heat source, a high-temperature heat pump will be installed to divert the flow and improve the temperature of the primary network to improve the heating value. The heating area that needs to be reduced in the jurisdiction of the low-temperature, high-flow heat source will be determined, the heating area will be adjusted, and the heat user area in the reduced heat source jurisdiction will be heated by increasing the heating capacity of other peripheral heat sources with excess heat, forming a multi-heat source heating system with high-temperature heat pumps. Among them, when adjusting the heating area, it also includes conducting a hydraulic analysis of the entire network using a hydraulic simulation model of the heating network to determine whether the heating network after adjusting the heating area meets the hydraulic balance requirements. If not, the area will be readjusted.

4. The primary network control method for a multi-heat source heating system according to claim 1, characterized in that: When establishing the heating control model, the primary network heating target value is set based on the heating demand of the area governed by the low-temperature, high-flow heat source, including: based on the weather data, regional area, heat source heating capacity, and heating operation data of the regional heating operation governed by the low-temperature, high-flow heat source, a machine learning algorithm is used to establish a regional heating load prediction model to obtain the regional heating demand, and a primary network heating target value that meets the regional heating demand is set.

5. The primary network control method for a multi-heat source heating system according to claim 1, characterized in that: The training process of the deep Q learning algorithm includes: Initialize the replay memory unit to store training samples; Initialize the current value network and randomly initialize the weight parameter w; initialize the target value network, whose structure and initialization weight are the same as the current value network; The evaporator inlet temperature, evaporator outlet temperature, evaporator water flow, condenser inlet temperature, condenser outlet temperature, condenser water flow, primary network supply temperature and high-temperature heat pump design performance are passed through the current value network to obtain the value function Q(s,a) under any state s. The ∈greedy strategy is used to select the action a with the largest value function Q(s,a). Each state transition is recorded as a time step t. The data (s,a,r,s′) obtained at each time step is stored in the playback memory unit; s, a, r and s′ are the state, action, reward and next state respectively. Define the loss function L(w), and randomly extract a data (s, a, r, s′) from the replay memory unit and transmit it to the current value network, target value network and loss function respectively. The loss function L(w) is updated using the stochastic gradient descent method with respect to the initialization weight parameter w.

6. The primary network control method for a multi-heat source heating system according to claim 1, characterized in that: When performing feedback correction in the model predictive control method, after executing the control strategy of the compressor operating frequency, diverter valve opening and bypass valve group switch state at the current moment, the error between the current primary network temperature actual value and the primary network temperature target value is used as the feedback correction amount to obtain the corrected control strategy of the compressor operating frequency, diverter valve opening and bypass valve group switch state at the next moment; When performing rolling optimization iterations in the model predictive control method, the output value at the current moment is processed using a softening coefficient method, and a rolling optimization performance objective function is established. After solving the function, an optimized control strategy is obtained.

7. The primary network control method for a multi-heat source heating system according to claim 6, characterized in that: The softening coefficient method is expressed as follows: p r (k+j)=a r p(k)+(1-a r )p s ; p r (k+j) is the reference output value at time (k+j); a r is the softening coefficient; p(k) is the actual output value at the current moment; p s is the output set value; j is the prediction step length; The rolling optimization performance objective function is established as follows: p c (k+j) is the output value after feedback correction; n is the maximum prediction length; m is the control length; λ is the control weighting coefficient; Δu(k) is the increment of the output value at the current moment.

8. The primary network control method for a multi-heat source heating system according to claim 1, characterized in that: The first control strategy and the second control strategy are integrated to obtain a fusion control mechanism, and after training the reinforcement learning controller according to the fusion control mechanism, the temperature control is performed by using the action output by the trained reinforcement learning controller as the optimal control strategy for the temperature supply of the primary network, including: The first control strategy and the second control strategy are integrated to obtain a fusion control mechanism, which is expressed as: u(t)=g(t)*a1(t)+1-g(t))*a2(t); g(t) is the action weight function at time t; a1(t) is the first control strategy; a2(t) is the second control strategy; The reinforcement learning controller interacts and updates according to the fusion control mechanism u(t). During the interaction, the model predictive control and reinforcement learning controller will implement control according to the fusion control mechanism. The process will give state transitions and store the resulting data tuples in the replay memory unit. During the update process, the samples in the replay memory unit are sampled to update the network parameters of the reinforcement learning controller. The updated parameters are input into the reinforcement learning controller network, and the output compressor operating frequency and diverter valve opening are used as the optimal control strategy for the primary network temperature supply to control the temperature.

9. A primary network control device for a multi-heat source heating system including a high-temperature heat pump, characterized in that: The multi-heat source heating system primary network control device includes: A multi-heat source heating system establishment unit including a high-temperature heat pump is used to determine low-temperature, high-flow heat sources and surrounding heat source objects based on the heating indicators of the areas under the jurisdiction of each heat source, set high-temperature heat pumps for the low-temperature, high-flow heat sources to divert flow and increase the temperature of the primary network, and determine the regional heating area that needs to be reduced in the areas under the jurisdiction of the low-temperature, high-flow heat sources. The surrounding heat sources increase the heating capacity and cooperate with the low-temperature, high-flow heat sources to provide heating, thereby forming a multi-heat source heating system including a high-temperature heat pump; the high-temperature heat pump includes an evaporator, a condenser, a compressor and a throttling device; The temperature control logic setting unit is used to divert the water supply side of the primary network of the low-temperature, high-flow heat source through the diverter valve and the bypass valve group switch to obtain a first water supply flow and a second water supply flow; by inputting electrical energy into the compressor as a circulating energy source, the refrigerant circulates inside the various components of the high-temperature heat pump and undergoes a phase change, and then the first water supply flow is passed through the heat exchanger and evaporator to absorb heat and cool it before flowing into the primary network return water, and the second water supply flow is passed through the condenser to release heat and heat it before serving as the primary network water supply to provide heat to the heat users in the area under the jurisdiction of the heat source; The temperature control model establishment unit is used to set the primary network temperature target value based on the heat demand of the area governed by the low-temperature, high-flow heat source. The evaporator inlet temperature, evaporator outlet temperature, evaporator water flow, condenser inlet temperature, condenser outlet temperature, condenser water flow, primary network temperature, and high-temperature heat pump design performance are used as model input variables. The compressor operating frequency, diverter valve opening, and bypass valve group switch status are used as model output variables. After model training through a hybrid algorithm of reinforcement learning and model predictive control, a primary network temperature control model is established, and the optimal temperature control strategy for the primary network is output, including: Collect influencing parameters of the primary network temperature control of the heating system, including evaporator inlet temperature, evaporator outlet temperature, evaporator water flow, condenser inlet temperature, condenser outlet temperature, condenser water flow, primary network temperature, high-temperature heat pump design performance, compressor operating frequency, diverter valve opening, and bypass valve group switch status; A multi-agent deep reinforcement learning algorithm is used to establish a temperature control model for the first-level network of the heating system: two reinforcement learning controllers are set as agents to control the operating frequency of the compressor and the opening of the diverter valve respectively, multi-agent deep reinforcement learning is performed and a neural network is constructed. The input layer is the currently acquired environmental state; the middle layer is fully connected with all possible actions, and the output layer is the value estimation of all actions under the current environmental state. The action output by the agent controlling the operating frequency of the compressor is all the frequencies that the compressor can reach, the action output by the agent controlling the opening of the diverter valve is all the openings that the diverter valve can reach, and the action output by the agent controlling the switch state of the bypass valve group is the open and closed state of the bypass valve group switch; a replay memory unit is set to store all samples (s t ,a t ,r t ,s t+1 ), s t is the current environmental status, including evaporator inlet temperature, evaporator outlet temperature, evaporator water flow, condenser inlet temperature, condenser outlet temperature, condenser water flow, primary network temperature supply, and high-temperature heat pump design performance; a t The action under the current environmental conditions, including the compressor operating frequency, diverter valve opening and bypass valve group switch status; r t In the current environment state s t Next, perform action a t The payback is expressed by the high-temperature heat pump operating power consumption and the regional heat user comfort; t+1 is the previous environment state s t Next, perform action a t The next state to which it migrates afterward; the reinforcement learning controller obtains the first control strategy through the deep Q learning algorithm; A model predictive control method is used to establish a primary network temperature control model for the heating system: a neural network is used as the prediction model of the model predictive control, with the evaporator inlet temperature, evaporator outlet temperature, evaporator water flow, condenser inlet temperature, condenser outlet temperature, condenser water flow, primary network temperature, and high-temperature heat pump design performance as input variables of the neural network model, and the compressor operating frequency, diverter valve opening, and bypass valve group switch state as output variables of the neural network model. After the neural network is trained, the predicted values ​​of the compressor operating frequency, diverter valve opening, and bypass valve group switch state at the current moment are obtained. Through feedback correction and rolling optimization iteration, a second control strategy for future multi-step prediction is obtained; The first control strategy and the second control strategy are integrated to obtain a fusion control mechanism. After the reinforcement learning controller is trained according to the fusion control mechanism, the temperature control is performed by taking the action output by the trained reinforcement learning controller as the optimal control strategy for the first-level network temperature supply.

Images