A cold source and terminal energy-saving optimization joint control method and system for a central air conditioner

Abstract

The application provides a cold source and terminal energy-saving optimization joint control method and system for a central air conditioner, the method comprising the following steps: S1, establishing an MDP model about a cold source energy-saving optimization strategy; S2, constructing a Q value function of the MDP model, and simultaneously establishing a deep Q network about the Q value function; S3, training the deep Q network through deep reinforcement learning to fit the Q value function; and S4, regulating and controlling the cold source system and the terminal equipment of the central air conditioner according to the following method: S41, adjusting the cooling water flow of the cold source system and the air supply amount of the terminal equipment according to the temperature difference between the temperature set value and the temperature measured value of the central air conditioner; and S42, collecting the current cooling area environment parameters and external environment parameters in real time, calculating the Q value of each control action in the control action space through the Q value function, and executing the control action with the maximum Q value. The application can realize real-time optimization of the control strategy of the cold source system based on the double targets of thermal comfort and energy saving.

Description

Technical Field

[0001] This invention relates to the field of energy conservation, and specifically to a method and system for optimizing the joint control of energy-saving cold sources and terminals in central air conditioning systems. Background Technology

[0002] In recent years, with the rapid development of urbanization, building energy consumption has increased significantly, making building energy conservation research an important direction in energy conservation studies. Among these, air conditioning systems have the highest energy consumption, accounting for more than 60% of the total building energy consumption, making them the main energy consumer in building operation. As an important component of central air conditioning systems, the cooling source system accounts for 30%-40% of the total energy consumption of the central air conditioning system in actual operation. The optimized operation of the cooling source system greatly affects the energy consumption of the entire central air conditioning system.

[0003] Generally, the equipment that needs to be controlled in a cooling system mainly includes the chiller, cooling water pump, chilled water pump, and cooling tower. Controllable equipment parameters include cooling water return temperature, chilled water outlet temperature, chilled water supply and return temperature difference, chiller start / stop, pump frequency, and cooling tower frequency. How to more scientifically set these equipment parameters to maximize energy efficiency while ensuring comfort is an important direction for improving existing technologies. Summary of the Invention

[0004] The purpose of this invention is to address the problems in the prior art by providing a cold source and terminal energy-saving optimization control system for central air conditioning, which optimizes the control strategy of the cold source system in real time based on the dual objectives of thermal comfort and energy saving.

[0005] To achieve the above objectives, the present invention adopts the following technical solution:

[0006] A method for energy-saving optimization and joint control of cold source and terminal units in central air conditioning systems, comprising:

[0007] S1. Establish an MDP model for energy-saving optimization strategies for cold sources, describing the MDP model as a quadruple {S, A, P, R}; where S is the environmental state space composed of environmental parameters of the cooling area and external environmental parameters; A is the control action space composed of control parameters of the cold source controller; P is the probability distribution of transitioning to other different environmental states after executing control action a in environmental state s; R is the immediate reward obtained by taking different control actions a in environmental state s; s ∈ S, a ∈ A;

[0008] S2. Construct the Q-value function of the MDP model, and simultaneously establish a deep Q-network based on the Q-value function. The expression of the deep Q-network is as follows:

[0009] ;

[0010] In the formula, The value function is related to the environmental state s. Let θ0 be the advantage function related to the environmental state s and the control action a, and let θ0 be the network parameters of the common part of the value function and the advantage function. v θ is the branch-independent parameter of the value function. a These are the branch-independent parameters of the dominance function. To control the size of the action space, a' represents all the control actions within the action space. The value of the dominance function is the average value over the entire control action space.

[0011] S3. Train the deep Q-network using deep reinforcement learning to fit the Q-value function;

[0012] S4. Adjust the cooling source system and terminal equipment of the central air conditioning system according to the following methods:

[0013] S41. Real-time acquisition of actual temperature values ​​in the cooling area; adjustment of cooling water flow rate of the cold source system and air volume of terminal equipment based on the temperature difference between the central air conditioning temperature setpoint and the actual temperature value, so that the actual temperature value approaches the temperature setpoint.

[0014] S42. Real-time acquisition of current environmental parameters of the cooling area and external environment parameters, calculation of the Q value of each control action in the control action space through the Q value function, and execution of the control action with the largest Q value to optimize the energy consumption of the central air conditioning.

[0015] Furthermore, in S1, the environmental parameters of the cooling area that make up the environmental state space S include indoor temperature, indoor relative humidity and current building schedule, while the external environmental parameters include outdoor temperature and outdoor relative humidity.

[0016] Furthermore, in S1, the control parameters that make up the control action space A include: cooling water return temperature, chilled water supply and return pressure difference, and chilled water supply temperature.

[0017] Furthermore, in S1, the reward function of the instant reward R is:

[0018] ;

[0019] In the formula, Let λ be the energy consumption of the cooling system at time t, λ be the thermal comfort penalty coefficient, and PMV be the human thermal comfort evaluation index. and These are the upper and lower limits of PMV, respectively;

[0020] The formula for calculating PMV is:

[0021] ;

[0022] In the formula, M is the human metabolic rate, L is the human heat load, and e is the natural constant.

[0023] Furthermore, S3 specifically includes the following steps:

[0024] S301. Based on the same deep Q-network structure, establish an action selection network, an action evaluation network, and an auxiliary update network respectively; where the network parameters of the action selection network are denoted as θ, and the network parameters of the action evaluation network are denoted as... The network parameters for the auxiliary network update are denoted as ;

[0025] S302. Initialize the network parameters θ of the action selection network and the network parameters of the action evaluation network. And assist in updating network parameters ;

[0026] S303, Real-time acquisition of the environmental state s at the current time t. t The data is then input into the action selection network to calculate the Q value of each control action in the control action space;

[0027] S304. Select a control action a from the control action space according to the ε-greedy strategy. t ;

[0028] S305, Execute the control action selected in S304 a t And obtain the current instant reward r based on the MDP model. t ;

[0029] S306, When executing control action a t Then, collect the environmental state s at the next time step t+1. t+1 , to use empirical samples Store in the experience replay library;

[0030] S307. Repeat S301 to S306 to accumulate a certain number of experience samples in the experience playback library.

[0031] S308. Randomly select m experience samples from the experience replay library to form a mini-batch sample; where the i-th experience sample in the mini-batch sample is defined as... , i=1,2,…,m;

[0032] S309. For each experience sample The next moment of the environment state Candidate actions are generated through an action selection network. :

[0033] ;

[0034] S310. Re-estimate candidate actions using the action evaluation network and the auxiliary update network, respectively. The Q-value, where the Q-value obtained by the action evaluation network is... The Q value obtained by the auxiliary network update is ;

[0035] S311. Weightedly fuse the Q-values ​​obtained from the action evaluation network and the auxiliary update network to obtain the fused Q-value. In the formula, β is the weighted fusion coefficient;

[0036] S312. Calculate the target value for each empirical sample. In the formula, γ is the attenuation factor. For instant rewards;

[0037] S313. Loss function for action selection network based on mean squared error The network parameters θ are selected by updating the gradient through backpropagation; the loss function is:

[0038] ;

[0039] In the formula, This represents the average of m empirical samples in a mini-batch.

[0040] S314. Update the ε value in the ε-greedy policy according to the preset decay rule, and repeat steps S303 to S313 until the accumulated instant reward is reached. Convergence; during the repeated steps S303 to S313, at each interval The network parameters of the action evaluation network will be evaluated in each cycle. Synchronization is achieved by ensuring that the network parameters θ of the action selection network are consistent with each interval. Each cycle will assist in updating the network parameters. Synchronization is achieved by ensuring that the network parameters θ of the action selection network are consistent. .

[0041] Furthermore, the ε-greedy strategy is as follows: select the control action with the largest Q value with a probability of 1-ε, and randomly select any control action uniformly from the control action space with a probability of ε.

[0042] The expression for the ε-greedy strategy is:

[0043] .

[0044] Furthermore, in S41, if the terminal equipment of the central air conditioning system is a fresh air unit, the following strategy is adopted for regulation:

[0045] The supply air temperature of the fresh air unit is used as the actual temperature value. The temperature difference Δt between the actual temperature value and the temperature setpoint is periodically measured and calculated as: Δt = actual temperature value – temperature setpoint.

[0046] When Δt < 0, adjust according to the following procedure: Determine if the frequency of the fresh air unit has reached the lower limit. If the frequency of the fresh air unit has not reached the lower limit, reduce the frequency of the fresh air unit. If the frequency of the fresh air unit has reached the lower limit, determine if the opening degree of the water valve of the cold source system has reached the lower limit. If the opening degree of the water valve has not reached the lower limit, reduce the opening degree of the water valve. If the opening degree of the water valve has reached the lower limit, determine if the number of fresh air units turned on has reached the lower limit. If the number of fresh air units turned on has not reached the lower limit, reduce the number of fresh air units turned on by one.

[0047] When Δt > 0, adjust according to the following procedure: Determine whether the water valve opening of the cold source system has reached the upper limit. If the water valve opening has not reached the upper limit, increase the water valve opening. If the water valve opening has reached the upper limit, determine whether the frequency of the fresh air unit has reached the upper limit. If the frequency of the fresh air unit has not reached the upper limit, increase the frequency of the fresh air unit. If the frequency of the fresh air unit has reached the upper limit, determine whether the number of fresh air units turned on has reached the upper limit. If the number of fresh air units turned on has not reached the upper limit, add one more fresh air unit.

[0048] Furthermore, in S41, if the terminal equipment of the central air conditioning system is an air handling unit, the following strategy is adopted for regulation:

[0049] The return air temperature of the blower unit is used as the actual temperature value. The temperature difference Δt between the actual temperature value and the temperature setpoint is periodically measured and calculated as: Δt = actual temperature value – temperature setpoint.

[0050] When Δt < 0, the following process is followed for adjustment: Determine whether the frequency of the blower unit has reached the lower limit. If the frequency of the blower unit has not reached the lower limit, reduce the frequency of the blower unit. If the frequency of the blower unit has reached the lower limit, determine whether the opening degree of the water valve of the cold source system has reached the lower limit. If the opening degree of the water valve has not reached the lower limit, reduce the opening degree of the water valve. If the opening degree of the water valve has reached the lower limit, determine whether the number of blowers opened has reached the lower limit. If the number of blowers opened has not reached the lower limit, reduce the number of blowers opened by one.

[0051] When Δt > 0, the following process is followed for control: Determine whether the water valve opening of the cold source system has reached the upper limit. If the water valve opening has not reached the upper limit, increase the water valve opening. If the water valve opening has reached the upper limit, determine whether the frequency of the blower unit has reached the upper limit. If the frequency of the blower unit has not reached the upper limit, increase the blower unit frequency. If the frequency of the blower unit has reached the upper limit, determine whether the number of blower units that are turned on has reached the upper limit. If the number of blower units that are turned on has not reached the upper limit, add one blower unit.

[0052] A cold source and terminal energy-saving optimization control system for central air conditioning, used to implement the above-described methods, includes:

[0053] The data acquisition module is used to collect environmental parameters of the cooling area, external environmental parameters, control parameters of the cold source controller, and energy consumption of the cold source system. The environmental parameters of the cooling area include indoor temperature, indoor relative humidity, air supply temperature of the air conditioning terminal, and return air temperature of the air conditioning terminal. The external environmental parameters include outdoor temperature, outdoor relative humidity, and the current building schedule.

[0054] The neural network training module is used to establish an MDP model and Q-value function for energy-saving optimization strategies of cold sources, establish a deep Q-network for the Q-value function, and train the deep Q-network using data collected by the data acquisition module to fit the Q-value function.

[0055] The terminal control module is used to adjust the cooling water flow rate of the cold source system and the air volume of the terminal equipment according to the temperature difference between the central air conditioning setpoint and the actual temperature value during normal operation of the central air conditioning system, so that the actual temperature value approaches the temperature setpoint.

[0056] The energy-saving strategy output module is used to input the environmental parameters of the cooling area and the external environment collected by the data acquisition module into the Q-value function obtained by the neural network training module during the normal operation of the central air conditioning system, and output the control action with the largest Q value.

[0057] The cold source controller is used to adjust control parameters according to the control actions output by the energy-saving strategy output module during the normal operation of the central air conditioning system, so as to optimize the energy consumption of the central air conditioning system.

[0058] This invention provides a method and system for energy-saving optimization and joint control of cold source and terminal units in central air conditioning systems. It boasts advantages such as model-free operation, adaptability, and online learning, eliminating the need for complex neural network models. Its model-free nature avoids cumbersome system modeling processes in practical applications. Furthermore, during the training phase, this invention employs an improved reinforcement learning algorithm based on value function approximation. Compared to existing reinforcement learning algorithms, the Q-value function obtained after training in this invention provides a more accurate estimate of the Q-value. Through the method and system provided by this invention, real-time optimization of the cold source system control strategy can be achieved more effectively based on the dual objectives of thermal comfort and energy saving. Attached Figure Description

[0059] Figure 1 This is a schematic flowchart of a method for energy-saving optimization and joint control of cold source and terminal units in a central air conditioning system, provided by an embodiment of the present invention.

[0060] Figure 2 yes Figure 1The detailed flowchart of step S3 in the process. Detailed Implementation

[0061] The technical solution of the present invention will now be described in detail with reference to the accompanying drawings and specific embodiments.

[0062] like Figure 1 As shown in the figure, an embodiment of the present invention provides a method for energy-saving optimization and joint control of cold source and terminal units in a central air conditioning system, comprising the following steps:

[0063] S1. Establish an MDP (Markov Decision Model) model for energy-saving optimization strategies for cold sources. The MDP model is described as a quadruple {S, A, P, R}; where S is the environmental state space composed of environmental parameters of the cooling area and external environmental parameters; A is the control action space composed of control parameters of the cold source controller; P is the probability distribution of transitioning to other different environmental states after executing control action a in environmental state s; R is the immediate reward obtained by taking different control actions a in environmental state s; s ∈ S, a ∈ A;

[0064] S2. Construct the Q-value function of the MDP model, and simultaneously establish a deep Q-network based on the Q-value function. The expression of the deep Q-network is as follows:

[0065] ;

[0066] In the formula, The value function is related to the environmental state s. Let θ0 be the advantage function related to the environmental state s and the control action a, and let θ0 be the network parameters of the common part of the value function and the advantage function. v θ is the branch-independent parameter of the value function. a These are the branch-independent parameters of the dominance function. To control the size of the action space, a' represents all the control actions within the action space. The value of the dominance function is the average value over the entire control action space.

[0067] S3. Train the deep Q-network using deep reinforcement learning to fit the Q-value function;

[0068] S4. Adjust the cooling source system and terminal equipment of the central air conditioning system according to the following methods:

[0069] S41. Real-time acquisition of actual temperature values ​​in the cooling area; adjustment of cooling water flow rate of the cold source system and air volume of terminal equipment based on the temperature difference between the central air conditioning temperature setpoint and the actual temperature value, so that the actual temperature value approaches the temperature setpoint.

[0070] S42. Real-time acquisition of current environmental parameters of the cooling area and external environment parameters, calculation of the Q value of each control action in the control action space through the Q value function, and execution of the control action with the largest Q value to optimize the energy consumption of the central air conditioning.

[0071] Specifically, in S1, the environmental parameters of the cooling area that make up the environmental state space S include indoor temperature, indoor relative humidity and current building schedule, and the external environmental parameters include outdoor temperature and outdoor relative humidity.

[0072] The control parameters that make up the control action space A include: cooling water return temperature, chilled water supply and return pressure difference, and chilled water supply temperature;

[0073] The reward function for the instant reward R is:

[0074] ;

[0075] In the formula, Let λ be the energy consumption of the cooling system at time t, λ be the thermal comfort penalty coefficient, and PMV be the human thermal comfort evaluation index. and These are the upper and lower limits of PMV, respectively;

[0076] The formula for calculating PMV is:

[0077] ;

[0078] In the formula, M is the human metabolic rate, L is the human heat load, and e is the natural constant.

[0079] Furthermore, S3 specifically includes the following steps:

[0080] S301. Based on the same deep Q-network structure, establish an action selection network, an action evaluation network, and an auxiliary update network respectively; where the network parameters of the action selection network are denoted as θ, and the network parameters of the action evaluation network are denoted as... The network parameters for the auxiliary network update are denoted as ;

[0081] S302. Initialize the network parameters θ of the action selection network and the network parameters of the action evaluation network. And assist in updating network parameters ;

[0082] S303, Real-time acquisition of the environmental state s at the current time t. t The data is then input into the action selection network to calculate the Q value of each control action in the control action space;

[0083] S304. Select a control action a from the control action space according to the ε-greedy strategy. t ;

[0084] S305, Execute the control action selected in S304 a t And obtain the current instant reward r based on the MDP model. t ;

[0085] S306, When executing control action a t Then, collect the environmental state s at the next time step t+1. t+1 , to use empirical samples Store in the experience replay library;

[0086] S307. Repeat S301 to S306 to accumulate a certain number of experience samples in the experience playback library.

[0087] S308. Randomly select m experience samples from the experience replay library to form a mini-batch; where the i-th experience sample in the mini-batch is defined as... , i=1,2,…,m; It should be noted that the empirical sample Definitions and empirical samples of various parameters The definitions of all parameters are the same; the parameter symbols have only been changed here for ease of expression later.

[0088] S309. For each experience sample The next moment of the environment state Candidate actions are generated through an action selection network. :

[0089] ;

[0090] S310. Re-estimate candidate actions using the action evaluation network and the auxiliary update network, respectively. The Q-value, where the Q-value obtained by the action evaluation network is... The Q value obtained by the auxiliary network update is ;

[0091] S311. Weightedly fuse the Q-values ​​obtained from the action evaluation network and the auxiliary update network to obtain the fused Q-value. In the formula, β is the weighted fusion coefficient;

[0092] S312. Calculate the target value for each empirical sample. In the formula, γ is the attenuation factor. For instant rewards;

[0093] S313. Loss function for action selection network based on mean squared error The network parameters θ are selected by updating the gradient through backpropagation; the loss function is:

[0094] ;

[0095] In the formula, This represents the average of m empirical samples in a mini-batch.

[0096] S314. Update the ε value in the ε-greedy strategy according to the preset decay rule (gradually reducing ε to the minimum value), and repeat steps S303 to S313 until the accumulated instant reward is reached. Convergence; during the repeated steps S303 to S313, at each interval The network parameters of the action evaluation network will be evaluated in each cycle. Synchronization is achieved by ensuring that the network parameters θ of the action selection network are consistent with each interval. Each cycle will assist in updating the network parameters. Synchronization is achieved by ensuring that the network parameters θ of the action selection network are consistent. .

[0097] Furthermore, the ε-greedy strategy in S304 is as follows: select the control action with the largest Q value with a probability of 1-ε, and uniformly and randomly select any control action from the control action space with a probability of ε.

[0098] The expression for the ε-greedy strategy is:

[0099] .

[0100] Furthermore, in S41, if the terminal equipment of the central air conditioning system is a fresh air unit, the following strategy is adopted for regulation:

[0101] The supply air temperature of the fresh air unit is used as the actual temperature value. The temperature difference Δt between the actual temperature value and the temperature setpoint is periodically measured and calculated as: Δt = actual temperature value – temperature setpoint.

[0102] When Δt < 0, adjust according to the following procedure: Determine if the frequency of the fresh air unit has reached the lower limit. If the frequency of the fresh air unit has not reached the lower limit, reduce the frequency of the fresh air unit. If the frequency of the fresh air unit has reached the lower limit, determine if the opening degree of the water valve of the cold source system has reached the lower limit. If the opening degree of the water valve has not reached the lower limit, reduce the opening degree of the water valve. If the opening degree of the water valve has reached the lower limit, determine if the number of fresh air units turned on has reached the lower limit. If the number of fresh air units turned on has not reached the lower limit, reduce the number of fresh air units turned on by one.

[0103] When Δt > 0, adjust according to the following procedure: Determine whether the water valve opening of the cold source system has reached the upper limit. If the water valve opening has not reached the upper limit, increase the water valve opening. If the water valve opening has reached the upper limit, determine whether the frequency of the fresh air unit has reached the upper limit. If the frequency of the fresh air unit has not reached the upper limit, increase the frequency of the fresh air unit. If the frequency of the fresh air unit has reached the upper limit, determine whether the number of fresh air units turned on has reached the upper limit. If the number of fresh air units turned on has not reached the upper limit, add one more fresh air unit.

[0104] In S41, if the terminal equipment of the central air conditioning system is an air handling unit, the following strategy is used for regulation:

[0105] The return air temperature of the blower unit is used as the actual temperature value. The temperature difference Δt between the actual temperature value and the temperature setpoint is periodically measured and calculated as: Δt = actual temperature value – temperature setpoint.

[0106] When Δt < 0, the following process is followed for adjustment: Determine whether the frequency of the blower unit has reached the lower limit. If the frequency of the blower unit has not reached the lower limit, reduce the frequency of the blower unit. If the frequency of the blower unit has reached the lower limit, determine whether the opening degree of the water valve of the cold source system has reached the lower limit. If the opening degree of the water valve has not reached the lower limit, reduce the opening degree of the water valve. If the opening degree of the water valve has reached the lower limit, determine whether the number of blowers opened has reached the lower limit. If the number of blowers opened has not reached the lower limit, reduce the number of blowers opened by one.

[0107] When Δt > 0, the following process is followed for control: Determine whether the water valve opening of the cold source system has reached the upper limit. If the water valve opening has not reached the upper limit, increase the water valve opening. If the water valve opening has reached the upper limit, determine whether the frequency of the blower unit has reached the upper limit. If the frequency of the blower unit has not reached the upper limit, increase the blower unit frequency. If the frequency of the blower unit has reached the upper limit, determine whether the number of blower units that are turned on has reached the upper limit. If the number of blower units that are turned on has not reached the upper limit, add one blower unit.

[0108] This invention also provides a cold source and terminal energy-saving optimization control system for central air conditioning, used to implement the methods described above. Specifically, the cold source and terminal energy-saving optimization control system for central air conditioning includes:

[0109] The data acquisition module is used to collect environmental parameters of the cooling area, external environmental parameters, control parameters of the cold source controller, and energy consumption of the cold source system. The environmental parameters of the cooling area include indoor temperature, indoor relative humidity, air supply temperature of the air conditioning terminal, and return air temperature of the air conditioning terminal. The external environmental parameters include outdoor temperature, outdoor relative humidity, and the current building schedule.

[0110] The neural network training module is used to establish an MDP model and Q-value function for energy-saving optimization strategies of cold sources, establish a deep Q-network for the Q-value function, and train the deep Q-network using data collected by the data acquisition module to fit the Q-value function.

[0111] The terminal control module is used to adjust the cooling water flow rate of the cold source system and the air volume of the terminal equipment according to the temperature difference between the central air conditioning setpoint and the actual temperature value during normal operation of the central air conditioning system, so that the actual temperature value approaches the temperature setpoint.

[0112] The energy-saving strategy output module is used to input the environmental parameters of the cooling area and the external environment collected by the data acquisition module into the Q-value function obtained by the neural network training module during the normal operation of the central air conditioning system, and output the control action with the largest Q value.

[0113] The cold source controller is used to adjust control parameters according to the control actions output by the energy-saving strategy output module during the normal operation of the central air conditioning system, so as to optimize the energy consumption of the central air conditioning system.

[0114] In the construction stage of the Q-value function, this invention decomposes the Q-value function into a state value function. and dominance function And by combining the shared feature parameter θ0, we obtain This decouples the action selection from the action evaluation, allowing the network to more accurately assess the value of the environmental state without being affected by the specific action selection.

[0115] During the training phase, traditional tabular reinforcement learning is difficult to directly adapt to the characteristics of continuous state and multi-parameter control in cold source systems. Therefore, this invention employs an improved reinforcement learning algorithm in S3. Specifically, during the training of the deep Q-network, this invention simultaneously updates and maintains three sets of neural network parameters with different functions; that is, based on the same experience replay library, forward inference and parameter updates of the action selection network, action evaluation network, and auxiliary update network are completed simultaneously. Among them, the action selection network is used to receive the environmental state s at the current time t in real time. t It calculates and outputs the Q value of all possible actions, and is responsible for selecting the "best candidate action" (i.e., control action a in S304) to be executed at the next time t+1. t The action evaluation network serves as the target network, used to calculate the target Q-value of the aforementioned "best candidate action," and its parameters... Instead of real-time gradient descent updates, the algorithm periodically synchronizes with the action selection network. The auxiliary update network is used to introduce a third-way estimation mechanism to solve the "overestimation" problem. The output of this network is compared with the output of the action evaluation network and weighted averaged as the final target value to reduce the estimation error generated by a single network and improve convergence stability.

[0116] In summary, compared with the prior art, the present invention has the following advantages:

[0117] 1. When constructing the Q-value function, the structure of the deep Q-network is decomposed by combining the value function and the advantage function, which can more accurately evaluate the value of the state without learning every control action in every environmental state, thereby improving the estimation accuracy in large-scale state spaces.

[0118] 2. During the training phase, the overestimation bias of Q-value estimation is reduced by separating the action selection network and the action evaluation network. Furthermore, the instability of Q-value estimation is further mitigated by updating the network architecture, thereby enabling the trained Q-value function to perform better in the dual-objective optimization of energy consumption and thermal comfort.

[0119] The embodiments described above are merely illustrative of several implementations of the present invention, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of the present invention. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of the present invention, and these modifications and improvements all fall within the scope of protection of the present invention. Therefore, the scope of protection of this patent should be determined by the appended claims.

Claims

1. A method for energy-saving optimization and joint control of cold source and terminal units in central air conditioning, characterized in that, include: S1. Establish an MDP model for energy-saving optimization strategies for cold sources, describing the MDP model as a quadruple {S, A, P, R}; where S is the environmental state space composed of environmental parameters of the cooling area and external environmental parameters; A is the control action space composed of control parameters of the cold source controller; P is the probability distribution of transitioning to other different environmental states after executing control action a in environmental state s; R is the immediate reward obtained by taking different control actions a in environmental state s; s ∈ S, a ∈ A; S2. Construct the Q-value function of the MDP model, and simultaneously establish a deep Q-network based on the Q-value function. The expression of the deep Q-network is as follows: ； In the formula, The value function is related to the environmental state s. Let θ0 be the advantage function related to the environmental state s and the control action a, and let θ0 be the network parameters of the common part of the value function and the advantage function. v θ is the branch-independent parameter of the value function. a These are the branch-independent parameters of the dominance function. To control the size of the action space, a' represents all the control actions within the action space. The value of the dominance function over the entire control action space; S3. Train the deep Q-network using deep reinforcement learning to fit the Q-value function; S4. Adjust the cooling source system and terminal equipment of the central air conditioning system according to the following methods: S41. Real-time acquisition of actual temperature values ​​in the cooling area; adjustment of cooling water flow rate of the cold source system and air volume of terminal equipment based on the temperature difference between the central air conditioning temperature setpoint and the actual temperature value, so that the actual temperature value approaches the temperature setpoint. S42. Real-time acquisition of current environmental parameters of the cooling area and external environment parameters, calculation of the Q value of each control action in the control action space through the Q value function, and execution of the control action with the largest Q value to optimize the energy consumption of the central air conditioning.

2. The method for energy-saving optimization and joint control of cold source and terminal units in central air conditioning according to claim 1, characterized in that, In S1, the environmental parameters of the cooling area that make up the environmental state space S include indoor temperature, indoor relative humidity and current building schedule, while the external environmental parameters include outdoor temperature and outdoor relative humidity.

3. The method for energy-saving optimization and joint control of cold source and terminal units in central air conditioning according to claim 1, characterized in that, In S1, the control parameters that make up the control action space A include: cooling water return temperature, chilled water supply and return pressure difference, and chilled water supply temperature.

4. The method for energy-saving optimization and joint control of cold source and terminal units in central air conditioning according to claim 1, characterized in that, In S1, the reward function of the instant reward R is: ； In the formula, Let λ be the energy consumption of the cooling system at time t, λ be the thermal comfort penalty coefficient, and PMV be the human thermal comfort evaluation index. and These are the upper and lower limits of PMV, respectively; The formula for calculating PMV is: ； In the formula, M is the human metabolic rate, L is the human heat load, and e is the natural constant.

5. The method for energy-saving optimization and joint control of cold source and terminal units in central air conditioning according to claim 1, characterized in that, S3 specifically includes the following steps: S301. Based on the same deep Q-network structure, establish an action selection network, an action evaluation network, and an auxiliary update network respectively; where the network parameters of the action selection network are denoted as θ, and the network parameters of the action evaluation network are denoted as... The network parameters for the auxiliary network update are denoted as ; S302. Initialize the network parameters θ of the action selection network and the network parameters of the action evaluation network. And assist in updating network parameters ; S303, Real-time acquisition of the environmental state s at the current time t. t The data is then input into the action selection network to calculate the Q value of each control action in the control action space; S304. Select a control action a from the control action space according to the ε-greedy strategy. t ; S305, Execute the control action selected in S304 a t And obtain the current instant reward r based on the MDP model. t ; S306, When executing control action a t Then, collect the environmental state s at the next time step t+1. t+1 , to use empirical samples Store in the experience replay library; S307. Repeat S301 to S306 to accumulate a certain number of experience samples in the experience playback library. S308. Randomly select m experience samples from the experience replay library to form a mini-batch sample; where the i-th experience sample in the mini-batch sample is defined as... , i=1,2,…,m; S309. For each experience sample The next moment of the environment state Candidate actions are generated through an action selection network. : ； S310. Re-estimate candidate actions using the action evaluation network and the auxiliary update network, respectively. The Q-value, where the Q-value obtained by the action evaluation network is... The Q value obtained by the auxiliary network update is ; S311. Weightedly fuse the Q-values ​​obtained from the action evaluation network and the auxiliary update network to obtain the fused Q-value. In the formula, β is the weighted fusion coefficient; S312. Calculate the target value for each empirical sample. In the formula, γ is the attenuation factor. For instant rewards; S313. Loss function for action selection network based on mean squared error The network parameters θ are selected by updating the gradient through backpropagation; the loss function is: ； In the formula, This represents the average of m empirical samples in a mini-batch. S314. Update the ε value in the ε-greedy policy according to the preset decay rule, and repeat steps S303 to S313 until the accumulated instant reward is reached. Convergence; during the repeated steps S303 to S313, at each interval The network parameters of the action evaluation network will be evaluated in each cycle. Synchronization is achieved by ensuring that the network parameters θ of the action selection network are consistent with each interval. Each cycle will assist in updating the network parameters. Synchronization is achieved by ensuring that the network parameters θ of the action selection network are consistent. .

6. The method for energy-saving optimization and joint control of cold source and terminal units in central air conditioning according to claim 5, characterized in that, The ε-greedy strategy is as follows: select the control action with the largest Q value with a probability of 1-ε, and randomly select any control action uniformly from the control action space with a probability of ε. The expression for the ε-greedy strategy is: 。 7. The method for energy-saving optimization and joint control of cold source and terminal units in central air conditioning according to claim 1, characterized in that, In S41, if the terminal equipment of the central air conditioning system is a fresh air unit, the following strategy is used for regulation: The supply air temperature of the fresh air unit is used as the actual temperature value. The temperature difference Δt between the actual temperature value and the temperature setpoint is periodically measured and calculated as: Δt = actual temperature value – temperature setpoint. When Δt < 0, adjust according to the following procedure: determine whether the frequency of the fresh air unit has reached the lower limit; if the frequency of the fresh air unit has not reached the lower limit, reduce the frequency of the fresh air unit; if the frequency of the fresh air unit has reached the lower limit, determine whether the opening degree of the water valve of the cold source system has reached the lower limit. If the water valve opening is below the lower limit, reduce the water valve opening; if the water valve opening has reached the lower limit, determine whether the number of fresh air units turned on has reached the lower limit; if the number of fresh air units turned on is below the lower limit, reduce the number of fresh air units turned on by one. When Δt>0, adjust according to the following procedure: determine whether the water valve opening of the cold source system has reached the upper limit. If the water valve opening has not reached the upper limit, increase the water valve opening. If the water valve opening has reached its maximum limit, then determine whether the frequency of the fresh air unit has reached its maximum limit. If the frequency of the fresh air system has not reached the upper limit, then increase the frequency of the fresh air system; if the frequency of the fresh air system has reached the upper limit, then determine whether the number of fresh air systems turned on has reached the upper limit. If the number of fresh air units turned on has not reached the maximum limit, then turn on one more fresh air unit.

8. The method for energy-saving optimization and joint control of cold source and terminal units in central air conditioning according to claim 1, characterized in that, In S41, if the terminal equipment of the central air conditioning system is an air handling unit, the following strategy is used for regulation: The return air temperature of the blower unit is used as the actual temperature value. The temperature difference Δt between the actual temperature value and the temperature setpoint is periodically measured and calculated as: Δt = actual temperature value – temperature setpoint. When Δt < 0, the following procedure is followed for adjustment: Determine whether the frequency of the blower unit has reached the lower limit. If the frequency of the blower unit has not reached the lower limit, reduce the frequency of the blower unit. If the frequency of the blower unit has reached the lower limit, determine whether the opening degree of the water valve of the cold source system has reached the lower limit. If the opening degree of the water valve has not reached the lower limit, reduce the opening degree of the water valve. If the opening degree of the water valve has reached the lower limit, determine whether the number of blowers opened has reached the lower limit. If the number of blowers opened has not reached the lower limit, reduce the number of blowers opened by one. When Δt>0, the following procedure is followed for regulation: Determine whether the water valve opening of the cold source system has reached the upper limit. If the water valve opening has not reached the upper limit, increase the water valve opening. If the water valve opening has reached the upper limit, determine whether the frequency of the blower unit has reached the upper limit. If the frequency of the blower unit has not reached the upper limit, increase the frequency of the blower unit; if the frequency of the blower unit has reached the upper limit, determine whether the number of blower units that can be opened has reached the upper limit; if the number of blower units that can be opened has not reached the upper limit, add one more blower unit.

9. A cold source and terminal energy-saving optimization control system for central air conditioning, used to implement the method described in any one of claims 1 to 8, characterized in that, include: The data acquisition module is used to collect environmental parameters of the cooling area, external environmental parameters, control parameters of the cold source controller, and energy consumption of the cold source system. The environmental parameters of the cooling area include indoor temperature, indoor relative humidity, air supply temperature of the air conditioning terminal and return air temperature of the air conditioning terminal; the external environmental parameters include outdoor temperature, outdoor relative humidity and current building schedule. The neural network training module is used to establish an MDP model and Q-value function for energy-saving optimization strategies of cold sources, establish a deep Q-network for the Q-value function, and train the deep Q-network using data collected by the data acquisition module to fit the Q-value function. The terminal control module is used to adjust the cooling water flow rate of the cold source system and the air volume of the terminal equipment according to the temperature difference between the temperature setpoint and the actual temperature value of the central air conditioning during normal operation, so that the actual temperature value approaches the temperature setpoint. The energy-saving strategy output module is used to input the environmental parameters of the cooling area and the external environment collected by the data acquisition module into the Q-value function obtained by the neural network training module during the normal operation of the central air conditioning system, and output the control action with the largest Q value. The cold source controller is used to adjust control parameters according to the control actions output by the energy-saving strategy output module during the normal operation of the central air conditioning system, so as to optimize the energy consumption of the central air conditioning system.

Images