Cross-regional air conditioner regulation and control strategy modeling method and cross-regional air conditioner regulation and control method and system

Abstract

The invention provides a cross-regional air conditioner regulation and control strategy modeling method and a cross-regional air conditioner regulation and control method and system, and is applied to the technical field of intelligent regulation and control of air conditioners. The method comprises the steps that clustering processing is conducted on air conditioner load data of users in all regions, and a clustering result is obtained; constructing a regional cold supply system simulation model corresponding to the clustering cluster based on the users in the same clustering cluster in the clustering result; the operation safety of an actual regional cooling system is considered, modeling is carried out on the regulation and control process of the simulation model based on the Markov decision process, and an optimization strategy model is constructed by taking expected return maximization of a value function corresponding to an operation strategy of the simulation model as a target; and training an optimization strategy model based on simulation operation data of the simulation model. According to the method, the problems that in the process that the DCS participates in power grid load regulation and control, a trial-and-error training-based model is prone to system damage in practical application, and single-model regulation and control cannot guarantee the load regulation and control effect of a whole power grid are solved.

Description

Technical Field

[0001] The present invention relates to the technical field of intelligent air conditioning control, and in particular to a cross-zone air conditioning control strategy modeling method, a cross-zone air conditioning control method and a system. Background Art

[0002] With the increase of volatile new energy and the widening of load peak-valley difference, the stable and safe operation of urban power grids has been challenged unprecedentedly. Traditional generators maintain the stable operation of the power system by frequently adjusting the output, which is energy-efficient and faces the problem of lack of adjustable resources. Among the types of urban load resources, the power consumption of commercial building air-conditioning systems accounts for a large proportion of the total urban power consumption, especially in the high temperature environment in summer, the load proportion of air-conditioning systems increases significantly, reaching more than 30% of the peak load of the power grid. As an important part of the total power consumption of the urban power grid, the air-conditioning system of commercial buildings has the potential to become a regulating resource to participate in the load regulation of the power grid.

[0003] District Cooling System (DCS), as a large-capacity and energy-saving HVAC system, is becoming the main form of cooling in modern cities. However, due to the complexity of DCS systems and the uncertainty of cooling demand, it is very difficult to control DCSs subject to heterogeneous indoor temperature constraints in multiple buildings. To solve this problem, deep reinforcement learning-based technology is applied to air conditioning control systems. However, since most of the control models based on deep reinforcement learning in simulation environments must undergo a lot of "trial and error" training to obtain better control effects, this means that some wrong decisions may be made during training, which makes it easy for the model to be damaged when deployed in actual environments. System safety issues such as system damage; moreover, most of the corresponding control models are currently built for one DCS, which cannot guarantee the load control effect of the entire power grid. Summary of the invention

[0004] In order to overcome the problems that the model based on trial-and-error training in the process of DCS participating in power grid load regulation is prone to system damage in actual application and single model regulation cannot guarantee the load regulation effect of the entire power grid, the present invention provides a cross-region air-conditioning regulation strategy modeling method, cross-region air-conditioning regulation method and system.

[0005] On the one hand, the present invention provides a method for clustering air conditioning load data of users in various regions to obtain clustering results;

[0006] Building a district cooling system simulation model corresponding to the cluster based on users in the same cluster in the clustering result;

[0007] For each district cooling system simulation model, on the basis of considering the operational safety of the actual district cooling system, the control process of the district cooling system simulation model is modeled based on the Markov decision process, and an optimization strategy model is constructed with the goal of maximizing the expected return of the value function corresponding to the operation strategy of the district cooling system simulation model;

[0008] The optimization strategy model is trained based on the simulated operation data of the regional cooling system simulation model, and the action network corresponding to the trained optimization strategy model is used as the air conditioning control strategy model for users in the cluster corresponding to the regional cooling system simulation model.

[0009] Optionally, clustering is performed on the air conditioning load data of users in each region to obtain clustering results, including:

[0010] Configure different numbers of cluster centers;

[0011] For each configuration of the number of cluster centers, clustering is performed with the goal of minimizing the distance between the air-conditioning load data of users in each region and each preset cluster center, so as to cluster the air-conditioning load data of users in each region into the cluster cluster corresponding to the cluster center with the closest distance, and obtain the clustering results under each configuration of the number of cluster centers;

[0012] Based on the graph convolutional network, the performance of the clustering results under each configuration of the number of cluster centers is evaluated, and based on the evaluation results, the clustering results under the target number of cluster centers are selected as the final clustering results.

[0013] Optionally, the air conditioning load data is an air conditioning load sequence, and the performance evaluation of the clustering results under each configuration of the number of cluster centers is performed based on the graph convolutional network, including:

[0014] For the clustering results under each configuration of the number of cluster centers, a load prediction sub-model corresponding to the cluster is trained based on the air-conditioning load sequence in each cluster;

[0015] Based on fusing the model parameters of the trained load forecasting sub-models corresponding to each cluster, a fusion parameter is obtained;

[0016] The fusion parameters are used as initial model parameters of the graph convolutional network, and the graph convolutional network is trained based on the air conditioning load sequences of users in various regions to obtain a global load forecasting model; each configuration of the number of cluster centers corresponds to a global load forecasting model;

[0017] Based on the prediction error of the global load forecasting model corresponding to each configuration of the number of cluster centers, the performance evaluation of the clustering results under the corresponding configuration of the number of cluster centers is carried out.

[0018] Optionally, performing district cooling system simulation modeling on users in the same cluster in the clustering result to obtain a district cooling system simulation model corresponding to the cluster includes:

[0019] Taking the users in the same cluster in the clustering result as a whole, based on fusing the building conditions of the users in the same cluster, determine the building parameters of the district cooling system simulation model corresponding to the cluster;

[0020] Determine the cooling associated parameters of the regional cooling system simulation model corresponding to the cluster based on the fusion processing of the cooling conditions of each user in the same cluster;

[0021] Based on the building parameters and cooling-related parameters of the district cooling system corresponding to the cluster, a simulation model of the district cooling system corresponding to the cluster is constructed.

[0022] Optionally, the regional cooling system simulation model includes a first water circulation loop from the energy station to each building, a second water circulation loop from each building to the room, and an air circulation loop in each room.

[0023] Optionally, the regulating process of the simulation model of the district cooling system is modeled based on a Markov decision process on the basis of considering the operational safety of the actual district cooling system, including:

[0024] For each district cooling system simulation model, a corresponding intelligent controller is constructed to control the amount of cold source flowing from the energy station in the district cooling system simulation model into each building;

[0025] An intelligent controller is used as an intelligent agent, and the power difference between the actual power corresponding to the simulation model of the regional cooling system and the upper power limit, the amount of cold source flowing into each building in the simulation model of the regional cooling system, the temperature of the cold source flowing into each building in the simulation model of the regional cooling system, and the temperature difference between the room temperature of each building and the user-set temperature are used as observation values ​​of the corresponding intelligent agent;

[0026] The cold source adjustment amount flowing into each building in the district cooling system simulation model is used as the action of the corresponding intelligent agent;

[0027] Considering the operational safety of an actual district cooling system corresponding to the district cooling system simulation model, a security layer is constructed between the intelligent agent and the district cooling system simulation model to perform security processing on actions input into the district cooling system simulation model;

[0028] The reward of the corresponding intelligent agent is determined based on the expectation of the temperature difference between the room temperature of each building and the temperature set by the user, the variance of the temperature difference between the room temperature of each building and the temperature set by the user, and the negative reward penalty corresponding to the safety processing.

[0029] Optionally, an optimization strategy model is constructed with the goal of maximizing the expected return of the value function corresponding to the operation strategy of the district cooling system simulation model, including:

[0030] Taking the expected return of the value function corresponding to the operation strategy of the district cooling system simulation model as the objective function;

[0031] Constructing an action network and a value network for solving the objective function based on a deep deterministic policy gradient algorithm;

[0032] The optimization strategy model is determined based on the action network and the value network.

[0033] Optionally, the control process includes a power supply control process, and before training the optimization strategy model based on the simulated operation data of the district cooling system simulation model, it also includes:

[0034] The simulation model of the district cooling system is used as an environment, and the interaction between the corresponding intelligent agent and the environment is performed by simulating the power supply regulation process of the actual district cooling system to obtain state transition data corresponding to the power supply regulation process;

[0035] The state transition data corresponding to the power supply guarantee and regulation process is stored in the experience buffer.

[0036] Optionally, training the optimization strategy model based on simulated operation data of the district cooling system simulation model includes:

[0037] Based on the data sampling of the experience buffer, a corresponding training sample is obtained;

[0038] Training the optimization strategy model based on the training samples to update model parameters of the action network, the value network, and the security layer;

[0039] Among them, during the training process, before the action corresponding to the training sample is input into the district cooling system simulation model, the operational safety of the current action is checked based on the safety layer. If the check fails, the safety mapping of the current action is performed based on the model parameters of the safety layer to obtain the safety action corresponding to the current action, and the safety action is input into the district cooling system simulation model for model training.

[0040] Optionally, the checking of the running safety of the current action includes:

[0041] By calling the simulation system to simulate the current action running in the simulation model of the district cooling system, the simulated operating power corresponding to the current action is obtained;

[0042] If the simulated operating power corresponding to the current action exceeds the power upper limit of the regional cooling system simulation model, it is determined that the safety inspection of the current action has failed.

[0043] Optionally, the control process further includes a user power recovery process, and before training the optimization strategy model based on the simulated operation data of the district cooling system simulation model, further includes:

[0044] Taking into account the user power recovery process, constructing a reward function corresponding to the user power recovery process based on an adaptive factor;

[0045] Simulating a user power recovery process of an actual district cooling system to perform a corresponding interaction between an intelligent agent and a district cooling system simulation model, and obtaining state transition data corresponding to the user power recovery process;

[0046] The state transition data corresponding to the user power recovery process is stored in the experience buffer.

[0047] Optionally, the expression of the reward function corresponding to the user power recovery process is:

[0048]

[0049] Among them, r t',re is the reward function at time t' in the user power recovery process, i is the building index, I is the building set, t' is the time corresponding to the user power recovery process, t1 is the start time of the user power recovery process, t2 is the end time of the user power recovery process, ΔT i,t' is the temperature difference between the room temperature of the i-th building at time t' and the temperature set by the user, is the power recovery adaptive factor of the i-th building at time t', E represents the expected value, is the temperature difference between the room temperature of the i-th building at time t1 and the user-set temperature, and λ is the recovery parameter.

[0050] On the other hand, the present invention also provides a cross-zone air conditioning control method, comprising:

[0051] Obtain the operating status data of the regional cooling system where the users in each cluster are located at the current moment;

[0052] For each cluster, based on the operating status data of the regional cooling system, the air conditioning control strategy model corresponding to the cluster constructed in the above embodiment is used to determine the cold source adjustment amount of the energy station flowing into the users in the cluster;

[0053] The air-conditioning power of the users in the cluster is regulated based on the cold source regulation amount flowing into the users in the cluster from the energy station.

[0054] Optionally, the operating status data includes the power difference between the actual power of the regional cooling system and the power upper limit, the amount of cold source flowing into each building in the regional cooling system, the temperature of the cold source flowing into each building in the regional cooling system, and the temperature difference between the room temperature of each building and the user-set temperature.

[0055] Optionally, the air conditioning power of the users in the cluster is regulated based on the cold source regulation amount flowing from the energy station to each user in the cluster, including:

[0056] If the current moment is in the power supply guarantee regulation period, power reduction regulation is performed based on the cold source regulation amount flowing into each user in the cluster from the energy station, so that the cooling system in the area where the users in the cluster are located can provide energy reserve for the power system;

[0057] If the current moment is in the user power recovery period, power recovery of users in the cluster is performed based on the cold source adjustment amount flowing into the users in the cluster from the energy station, so that the actual indoor temperature of users in the cluster approaches the user set temperature.

[0058] On the other hand, the present invention also provides a cross-region air conditioning control strategy modeling system, comprising:

[0059] A clustering module is used to perform clustering processing on the air conditioning load data of users in each region to obtain clustering results;

[0060] A simulation module, used for building a district cooling system simulation model corresponding to the cluster based on users in the same cluster in the clustering result;

[0061] A strategy model building module is used for modeling the control process of each district cooling system simulation model based on the Markov decision process on the basis of considering the operation safety of the actual district cooling system, and building an optimization strategy model with the goal of maximizing the expected return of the value function corresponding to the operation strategy of the district cooling system simulation model;

[0062] A strategy model training module is used to train the optimization strategy model based on the simulated operation data of the regional cooling system simulation model, and use the action network corresponding to the trained optimization strategy model as the air conditioning control strategy model for users in the cluster corresponding to the regional cooling system simulation model.

[0063] On the other hand, the present invention also provides a cross-zone air conditioning control system, comprising:

[0064] An acquisition module is used to obtain the operation status data of the regional cooling system where the users in each cluster are located at the current moment;

[0065] A control module is used to determine, for each cluster, based on the operating status data of the regional cooling system, the air-conditioning control strategy model corresponding to the cluster constructed by the above embodiment to determine the cold source adjustment amount of the energy station flowing into the users in the cluster; and to control the air-conditioning power of the users in the cluster based on the cold source adjustment amount of the energy station flowing into the users in the cluster.

[0066] On the other hand, the present invention also provides an electronic device, comprising: at least one processor and a memory; the memory and the processor are connected via a bus;

[0067] The memory is used to store one or more programs;

[0068] When the one or more programs are executed by the at least one processor, the above-mentioned cross-zone air-conditioning control strategy modeling method or cross-zone air-conditioning control method is implemented.

[0069] On the other hand, the present invention also provides a readable storage medium having an execution program stored thereon, and when the execution program is executed, the above-mentioned cross-zone air-conditioning control strategy modeling method or cross-zone air-conditioning control method is implemented.

[0070] Compared with the prior art, the present invention has the following beneficial effects:

[0071] The present invention provides a cross-region air-conditioning control strategy modeling method and system, by constructing a regional cooling system simulation model, and on the basis of considering the operational safety of an actual regional cooling system, the control process of the regional cooling system simulation model is modeled based on a Markov decision process, and an optimization strategy model is constructed with the goal of maximizing the expected return of a value function corresponding to the operational strategy of the regional cooling system simulation model, so that the constructed strategy model fully considers the operational safety of the actual regional cooling system, avoids erroneous decisions that threaten the system safety during the model training process, and further avoids the problem of damage to the air-conditioning system when the model is deployed in an actual environment, thereby improving the operational safety of the actual control process.

[0072] The present invention obtains a clustering result by clustering the air-conditioning load data of users in various regions, and constructs a regional cooling system simulation model corresponding to the clustering cluster based on the air-conditioning load data in the same clustering cluster in the clustering result, thereby realizing unified modeling of users with similar load characteristics in different regions, being able to coordinate the load regulation in various regions and ensuring the regulation effect for the entire power grid; at the same time, the model scale can be reduced, and the computational overhead of the model training process can be greatly reduced. BRIEF DESCRIPTION OF THE DRAWINGS

[0073] Figure 1 A schematic diagram of a flow chart of a cross-region air conditioning control strategy modeling method of the present invention;

[0074] Figure 2 A schematic diagram of the structure and cold source circulation flow of a regional cooling system simulation model of the present invention;

[0075] Figure 3 It is a schematic structural diagram of the electronic device of the present invention. DETAILED DESCRIPTION

[0076] The specific implementation modes of the present invention are further described in detail below with reference to the accompanying drawings.

[0077] Embodiment 1:

[0078] The present invention provides a cross-region air conditioning control strategy modeling method, the schematic diagram is as follows Figure 1 As shown, including:

[0079] Step S110, clustering the air conditioning load data of users in each region to obtain a clustering result;

[0080] Step S120, constructing a district cooling system simulation model corresponding to the cluster based on the users in the same cluster in the clustering result;

[0081] Step S130, for each district cooling system simulation model, based on the consideration of the operational safety of the actual district cooling system, a control process of the district cooling system simulation model is modeled based on a Markov decision process, and an optimization strategy model is constructed with the goal of maximizing the expected return of the value function corresponding to the operation strategy of the district cooling system simulation model;

[0082] Step S140, training the optimization strategy model based on the simulated operation data of the district cooling system simulation model, and using the action network corresponding to the trained optimization strategy model as the air conditioning control strategy model for users in the cluster corresponding to the district cooling system simulation model.

[0083] In this example implementation, each region may include different cities or regions, and the air conditioning load data may be the total air conditioning load time series data of users of the regional cooling system. Clustering is the process of clustering users based on load characteristics. The Markov Decision Process (MDP) is used to simulate the random strategies and rewards that can be achieved by intelligent agents in an environment where the system state has Markov properties. This example constructs a simulation model and an optimization strategy model for each clustering cluster, and each user in the clustering cluster corresponds to an air conditioning control strategy model, that is, the same control strategy is used for users in the same clustering cluster, so as to achieve unified control of users across regions, coordinate the load control of various regions, and ensure the control effect for the entire power grid. The control process of the regional cooling system simulation model includes the power supply control process and the user power recovery process. The power supply control process refers to the control process corresponding to the operation reserve provided by the regional cooling system for the power system, and the user power recovery process refers to the control process of restoring the cooling effect of the regional cooling system after the peak period of power consumption of the power system. The simulated operation data of the simulation model of the regional cooling system is the data obtained by simulating the actual DCS operation process through the interaction between the simulation model and the intelligent agent, and may include state transition data obtained by simulating the power supply control process and the user power recovery process respectively. The state transition data may be the Markov chain of the corresponding model, i.e., the MDP trajectory, and the state transition data may be obtained through the interaction between the intelligent agent and the environment (the simulation model of the regional cooling system). On the basis of considering the operational safety of the actual regional cooling system, the control process of the simulation model of the regional cooling system is modeled based on the Markov decision process, and an optimization strategy model is constructed with the goal of maximizing the expected return of the value function corresponding to the operation strategy of the simulation model of the regional cooling system, so that the constructed strategy model fully considers the operational safety of the actual regional cooling system, avoids erroneous decisions that threaten the system safety during the model training process, and thus avoids the problem of damage to the air-conditioning system when the model is deployed in the actual environment, and improves the operational safety of the actual control process.

[0084] In an example implementation, clustering is performed on the air conditioning load data of users in each region in step S110 to obtain a clustering result, including:

[0085] Configure different numbers of cluster centers;

[0086] For each configuration of the number of cluster centers, clustering is performed with the goal of minimizing the distance between the air-conditioning load data of users in each region and each preset cluster center, so as to cluster the air-conditioning load data of users in each region into the cluster cluster corresponding to the cluster center with the closest distance, and obtain the clustering results under each configuration of the number of cluster centers;

[0087] Based on the graph convolutional network, the performance of the clustering results under each configuration of the number of cluster centers is evaluated, and based on the evaluation results, the clustering results under the target number of cluster centers are selected as the final clustering results.

[0088] In this example implementation, since the total number of clusters has a great influence on the control effect of the final strategy model, this example specifically designs a method for determining the number of clusters. You can first configure different numbers of candidate cluster centers based on experience. For example, set the number of cluster centers to k, where k is 6, 8, 10, etc. Perform clustering processing for each cluster center configuration (i.e., k value) to obtain clustering results under each configuration. Perform performance evaluation on the clustering results under each configuration through a graph neural network, and use the cluster center number configuration with the best performance evaluation result as the target cluster center number configuration. Use the clustering result corresponding to the target cluster center number configuration as the final clustering result. Exemplarily, the clustering process is as follows: the data to be processed is the total air-conditioning load time series of the regional cooling system of users in different regions. Suppose the total number of users of the data to be processed is N, y j,t represents the tth observation value of the jth user air conditioning load time series, and the jth user air conditioning load time series is represented by x j =(y j,1 ,y j,2 ,…,y j,p ) T ,y j,p is the pth observation value of the jth user's air conditioning load time series, p is the total number of time steps of the user, where j = 1, 2, 3, ..., N. Hierarchical clustering: Set the maximum number of clusters M, and set the number of clusters m = 1 for loop iteration , 2,...M. In the first iteration, for each cluster number m in the loop, the centroids (cluster centers) {e1,e2,...,e m},e m is the mth cluster center / centroid. The number of centroids in each cycle is the number of clusters. The objective function formula of clustering processing is used to divide the load time series closest to the centroid into the same cluster. The objective function formula of clustering processing is:

[0089]

[0090] Where D = {D1, D2, ..., D k} represents a clustering result set containing a total of k clusters, D k represents the kth cluster, e k Representative D k The centroid (cluster center), x j Represents cluster D k The j-th user air conditioning load time series, |D k| indicates cluster D k The total number of users. The air conditioning load time series of each user is clustered, that is, after the cluster label of the nearest centroid is obtained, the centroid set {e1, e2, ..., e m}. After multiple iterations, clustering results with different cluster numbers are obtained.

[0091] Exemplarily, the air conditioning load data is an air conditioning load sequence, and the performance evaluation of the clustering results under each configuration of the number of cluster centers is performed based on the graph convolutional network, including the following process:

[0092] For the clustering results under each configuration of the number of cluster centers, a load prediction sub-model corresponding to the cluster is trained based on the air-conditioning load sequence in each cluster;

[0093] Based on fusing the model parameters of the trained load forecasting sub-models corresponding to each cluster, a fusion parameter is obtained;

[0094] The fusion parameters are used as initial model parameters of the graph convolutional network, and the graph convolutional network is trained based on the air conditioning load sequences of users in various regions to obtain a global load forecasting model; each configuration of the number of cluster centers corresponds to a global load forecasting model;

[0095] Based on the prediction error of the global load forecasting model corresponding to each configuration of the number of cluster centers, the performance evaluation of the clustering results under the corresponding configuration of the number of cluster centers is carried out.

[0096] In this example implementation, the load prediction sub-model can be a model based on graph convolution and time convolution (i.e., GCN-TCN), and the global load prediction model adopts graph convolutional networks (GCN). GCN includes two parts: In the spatial dimension, a graph convolutional network is used. Suppose the model output corresponding to the input j-th user air conditioning load sequence is is the predicted value of the air conditioning load of the jth user at time t+1; from the perspective of graph theory learning, The forecast expectation can be expressed as:

[0097]

[0098] Among them, E is the expectation, Ne(v j ) represents the domain user set of the jth user, that is, the node v in the graph theory j The domain node set of x j For node v j Corresponding to the user's time series, v n ∈Ne(v j ), v n is node v jA neighbor node of n is node v n Corresponding to the user's time series, k0 is the node self-loop learnable weight parameter in the corresponding graph theory, k s1 is the learnable weight parameter corresponding to the positive relationship with the domain node s in graph theory, k s2 is the learnable weight parameter corresponding to the negative relationship with the domain node s in graph theory. For the convenience of implementation, this paper converts the relationship k between the node and the domain node into s1 and k s2 Use relationship matrices A0 and A in Indicates that The original formula (2) of the prediction expectation is extended to the prediction expectation x of any user t+1 :

[0099] x t+1 =(A0X+A in X -1 )U (3)

[0100] Among them, A0,A in ∈R N×N and are three learnable parameters, R represents the dimension of the matrix, d0 and d m are the dimensions of node input and output, respectively. N represents the number of all nodes. X represents the data matrix of all nodes. A0 is the positive relationship matrix between nodes and neighboring nodes, and also includes the relationship between nodes and self-loops. X -1 represents the inverse of all node data matrices, A in The matrix representing the negative relationship between nodes and node domains. The learnable weight set is represented by W s ={A0,A in ,U}, and the loss function is represented by the mean absolute error between the true value and the predicted value.

[0101] For each clustering result, 90% of the user air conditioning load sequence data of the same cluster cluster is used as a batch of training data and input into the GCN neural network model to train the corresponding GCN-TCN (Graph Convolutional Network-Temporal Convolutional Networks) sub-model (load prediction sub-model), and the remaining 10% of the data is used as a test set. Different batches of training data are used independently to train the GCN neural network model, so that a GCN-TCN sub-model equal to the number of clusters in the clustering result can be trained under each clustering result. Under each clustering result, the data of the same cluster is used to train and converge to obtain the parameters of the load prediction sub-model. The different sub-model parameters obtained from different clusters can be averaged to obtain the initial model parameters of the global load prediction model; then the global load prediction model is globally trained in all user air conditioning load data until convergence, and the global load prediction model under each cluster center configuration is obtained. By comparing the MAPE (Mean Absolute Percentage Error) index performance of the global load prediction model under each cluster center configuration, the best clustering result, that is, the best clustering number, is obtained, and the clustering result corresponding to the clustering number is used as the final clustering result.

[0102] In an example implementation, step S120 performs district cooling system simulation modeling on users in the same cluster in the clustering result to obtain a district cooling system simulation model corresponding to the cluster, including:

[0103] Taking the users in the same cluster in the clustering result as a whole, based on fusing the building conditions of the users in the same cluster, determine the building parameters of the district cooling system simulation model corresponding to the cluster;

[0104] Determine the cooling associated parameters of the regional cooling system simulation model corresponding to the cluster based on the fusion processing of the cooling conditions of each user in the same cluster;

[0105] Based on the building parameters and cooling-related parameters of the district cooling system corresponding to the cluster, a simulation model of the district cooling system corresponding to the cluster is constructed.

[0106] In this example implementation, each user in each cluster is taken as a whole to construct a regional cooling system simulation model, so that unified regulation of users in the same cluster can be achieved. The fusion processing can be averaging or weighted averaging, etc. The building situation can be the building area, and the building area of ​​users in the same cluster can be averaged as the building parameter of the corresponding simulation model. The cooling situation can be various parameters of the DCS system of each user. The various parameters of the DCS system can specifically include the cooling capacity of the energy station assembly, the inlet and return water temperatures of the two water circulation loops, the efficiency coefficient of the heat exchange process between different loops, the heat transfer coefficient of the water supply pipe, the air mixing ratio, the maximum range of mass flow of different buildings and the corresponding minimum value, the set temperature of each building, the maximum deviation of the required comfortable indoor temperature, the ambient temperature, the heat load of each building and the upper limit of the operating power, etc. For example, for each parameter, the fusion processing can be performed by averaging the users in the same cluster, and the average value of each parameter is used as the parameter of the corresponding simulation model. This example uses the best cluster center configuration to simulate the building air conditioning for users in the same cluster as a whole. The initialization parameters are obtained by integrating the actual conditions of each user, and the building area and various parameters of the DCS system in the area are initialized based on the initialization parameters. The simulation model of the district cooling system can be constructed in the EnergyPlus simulation software.

[0107] like Figure 2 As shown in the figure, it is a control flow chart of the regional cooling system simulation model constructed in this example. The regional cooling system simulation model in the figure includes the first water circulation loop from the energy station to each building, the second water circulation loop from each building to the room, and the air circulation loop in each room, that is, the three parts in the dotted box in the figure. The chiller is the energy station. For the first water circulation loop I, the outlet water temperature of the system chiller is The mass flow rate of the outlet water is The return water temperature is The variable frequency water pump transfers the water output of the chiller to different mass flow rates. Sent to each building to participate in heat exchange, where i is the building number, is the cold source amount flowing into building i at time t, and the return water after exchange is sent back to the chiller for cooling. For the second water circulation loop II, the water supply mass flow rate in building i can be expressed as The water supply temperature is The water supply cools the room through the air handling unit in the terminal room. The return water of the second water circulation loop also transfers heat with the water supply of the first water circulation loop at the heat exchanger to cool down. For the air circulation loop, the temperature of the room is adjusted by the air handling unit in the terminal room, and its air mass flow rate is The air outlet temperature is The room temperature is

[0108] In order to solve the DCS control problem, the DCS control problem is mathematically expressed as a Markov decision process (MDP). In some embodiments, step S130, based on the consideration of the operational safety of the actual district cooling system, models the control process of the district cooling system simulation model based on the Markov decision process, including the following process:

[0109] For each district cooling system simulation model, a corresponding intelligent controller is constructed to control the amount of cold source flowing from the energy station in the district cooling system simulation model into each building;

[0110] An intelligent controller is used as an intelligent agent, and the power difference between the actual power corresponding to the simulation model of the regional cooling system and the upper power limit, the amount of cold source flowing into each building in the simulation model of the regional cooling system, the temperature of the cold source flowing into each building in the simulation model of the regional cooling system, and the temperature difference between the room temperature of each building and the user-set temperature are used as observation values ​​of the corresponding intelligent agent;

[0111] The cold source adjustment amount flowing into each building in the district cooling system simulation model is used as the action of the corresponding intelligent agent;

[0112] Considering the operational safety of an actual district cooling system corresponding to the district cooling system simulation model, a security layer is constructed between the intelligent agent and the district cooling system simulation model to perform security processing on actions input into the district cooling system simulation model;

[0113] The reward of the corresponding intelligent agent is determined based on the expectation of the temperature difference between the room temperature of each building and the temperature set by the user, the variance of the temperature difference between the room temperature of each building and the temperature set by the user, and the negative reward penalty corresponding to the safety processing.

[0114] In this example implementation, based on Figure 2 The DCS control process is parameterized in the MDP framework as follows. The DCS is set to provide operating reserves for the power system during the power supply regulation process, that is, the power supply regulation cycle T1 = [t0, t1]. t0 is the start time of the power supply regulation process, and t1 is the end time of the power supply regulation process. Specifically, the DCS simulation model is regarded as an environment, and a centralized intelligent controller (agent) is designed to send signals to each building to control the mass flow of each building. The agent observes the state. Then, according to s tBased on the information in the DCS, the agent makes a decision on the execution action. Based on the above process, a complete interaction trajectory can be obtained. is the model running state at time t0, is the execution action at time t0+1, is the model running status at time t0+1, is the execution action at time t1, is the model operation state at time t1. After taking action, it transfers from the current state to the next state. In the above DCS control process, the state state and action action are defined as:

[0115]

[0116] Among them, s t is the operating state of the DCS simulation model at time t, a t is the action of the intelligent controller at time t, ΔP t is the power difference between the actual power and the upper power limit corresponding to the simulation model of the district cooling system at time t, is the amount of cold source flowing into building i in the district cooling system simulation model at time t, is the temperature of the cold source flowing into building i in the district cooling system simulation model at time t, ΔT i,t is the temperature difference between the room temperature of building i at time t and the temperature set by the user, S is the operating state set of the DCS simulation model, is the cold source regulation amount flowing into building |I| in the simulation model of the district cooling system at time t, |I| is the total number of buildings, A is the action set, T is the corresponding period of the entire regulation process, and the superscript T is the matrix transpose. is the set temperature of room i in building i at time t, represents the air temperature in room i of building at time t.

[0117] In order to ensure that each executed action will not cause safety problems in the district cooling system, a safety layer is built between the agent and the simulation model of the district cooling system to perform pre-safety processing on the actions input into the simulation model, thereby ensuring that the training process will not cause system damage and other problems. In order to evaluate the performance of the strategy, this example designs a reward function for the action, where, combined with the negative reward penalty provided by the safety layer to the agent, the final reward function can be expressed as:

[0118]

[0119] In the formula, the first term E i∈I [|ΔT i,t|] represents the average temperature of the temperature difference between the indoor temperature of all buildings and the corresponding set value at time t. The second term represents the variance of the temperature difference between the indoor temperature of all buildings at time t and the corresponding set value. The third term θ safe |μ t +υ t | represents the negative reward penalty provided by the security layer to the agent; r t,se is the reward function at time t of the power supply regulation process, T1 is the corresponding period of the power supply regulation process, θ r To determine the weight factor of the first and second terms, E is the expectation, σ 2 To find the variance, μ t ,υ t are the model parameters of the security layer, i.e., the correction coefficient, θ safe is the weight factor of the safety layer. In this example, the control problem of the district cooling system is transformed into a Markov decision process, realizing the transformation of the actual control problem into a mathematical model, applying deep reinforcement learning to DCS control, and preventing the model from making wrong decisions by setting a safety layer.

[0120] In some embodiments, based on the above problem transformation, step S130 constructs an optimization strategy model with the goal of maximizing the expected return of the value function corresponding to the operation strategy of the district cooling system simulation model, including:

[0121] Taking the expected return of the value function corresponding to the operation strategy of the district cooling system simulation model as the objective function;

[0122] Constructing an action network and a value network for solving the objective function based on a deep deterministic policy gradient algorithm;

[0123] The optimization strategy model is determined based on the action network and the value network.

[0124] In this example implementation, based on the mathematical model of conversion, the discount return is determined based on the reward function, and the discount return corresponding to the reward function is defined as:

[0125]

[0126] Where γ∈[0,1] is the discount factor, G t represents the expected return of the action reward function at time t, r t is the action reward at time t, r t+1 is the action reward at time t+1, r t+Tis the action reward at time t+1, T is the regulation period, and te is the end time of the regulation process; for example, when in the power supply regulation process, T = T1, te = t1. The goal of the intelligent agent is to maximize the expected return of the optimal action value function, and the constructed objective function is:

[0127]

[0128] Among them, π is the policy function, J π is the expected return of the policy function π, Q π is the action value function, π(s t ) is state s t The following action strategy.

[0129] In order to solve the objective function J π The optimal policy function is obtained by using the Deep Deterministic Policy Gradient (DDPG) algorithm to optimize the policy model. Specifically, two neural networks are used to approximate the action value function Q π and policy function π. Approximate Q π The value network is called the critic network, and the other network that outputs actions (the network that approximates the strategy π) is called the actor network. The optimization strategy model is composed of the action network (actor network) and the value network (critic network).

[0130] In some embodiments, the control process includes a power supply control process, and before training the optimization strategy model based on the simulated operation data of the district cooling system simulation model, it also includes:

[0131] The simulation model of the district cooling system is used as an environment, and the interaction between the corresponding intelligent agent and the environment is performed by simulating the power supply regulation process of the actual district cooling system to obtain state transition data corresponding to the power supply regulation process;

[0132] The state transition data corresponding to the power supply guarantee and regulation process is stored in the experience buffer.

[0133] In this example implementation, for the power supply and regulation process, the interaction between the corresponding intelligent agent and the environment is carried out by simulating the power supply and regulation process of the actual regional cooling system, that is, the agent interacts with the DCS environment, and the state transition data corresponding to the power supply and regulation process is obtained, which can be the state transition trajectory of the interaction process, and is stored in the experience buffer as training data. The experience buffer is used to store training data.

[0134] After obtaining the training data, in some embodiments, step S140 of training the optimization strategy model based on the simulated operation data of the district cooling system simulation model includes:

[0135] Based on the data sampling of the experience buffer, a corresponding training sample is obtained;

[0136] Training the optimization strategy model based on the training samples to update model parameters of the action network, the value network, and the security layer;

[0137] Among them, during the training process, before the action corresponding to the training sample is input into the district cooling system simulation model, the operational safety of the current action is checked based on the safety layer. If the check fails, the safety mapping of the current action is performed based on the model parameters of the safety layer to obtain the safety action corresponding to the current action, and the safety action is input into the district cooling system simulation model for model training.

[0138] In this example implementation, data is sampled from the experience buffer to obtain training samples. Each training sample may include the current state, action, reward, and state at the next moment. The four data constitute a training sample. For example, the agent randomly samples a small batch of data from the experience buffer. To update the actor network and the critic network. The loss function of the critic network can be the mean square error loss function, which is expressed as:

[0139]

[0140] Where L1 is the loss function of the critic network, Q(s u ,a u ) is the value function value corresponding to the u-th sample, y u is the target value to be estimated, and u is the sample index; the critic network can be expressed as follows based on the expected equation of the Bellman action value function:

[0141]

[0142] In the formula, Q π (s t ,a t ) is the action value function at time t, Q π (s t+1 ,a t+1 ) is the action value at time t+1. Two target networks can be used to estimate the critic network to avoid unstable training, which can be expressed as:

[0143]

[0144] Among them, Q′, π′ represent two target networks, r u is the reward for the u-th sample, s u+1 is the state of the u+1th sample, and the network parameters can be updated at a smaller frequency than the critic network.

[0145] The loss function of the actor network is the gradient estimate of the current policy, expressed as:

[0146]

[0147] Where L2 is the loss function of the actor network, π(s u ) is the action strategy of the u-th sample.

[0148] This example obtains training samples by sampling from the experience buffer, trains the optimization strategy model through the training samples and equations (9)-(11), and updates the action network and the value network. In addition, in order to improve the exploration efficiency, an independent noise term that obeys the normal distribution can be added to each action. In order to ensure that the actions generated by the actor network meet the operational safety constraints during training, in this example, before the actions corresponding to the training samples are input into the simulation model of the district cooling system, the operational safety of the current action is checked based on the safety layer during the model training process. If the check fails, the safety mapping of the current action is performed based on the model parameters of the safety layer to obtain the safety action corresponding to the current action, and the safety action is input into the simulation model of the district cooling system for model training; if the check passes, the current action is input into the simulation model for training. Exemplarily, by calling the simulation system to simulate the current action running in the simulation model of the district cooling system, the simulated operating power corresponding to the current action is obtained; if the simulated operating power corresponding to the current action exceeds the power upper limit of the simulation model of the district cooling system, it is determined that the safety check of the current action fails. That is, before the action is passed to the EnergyPlus simulation model, the simulation interface on EnergyPlus can be used to simulate the current action, obtain the simulated operation power, and determine whether the simulated operation power exceeds the functional upper limit of the simulation model. If so, the action that causes the system simulated operation power to exceed the power upper limit is mapped to a safe action, otherwise the current action is directly input. The safety mapping rule can be expressed as:

[0149]

[0150] in, is the safety action obtained after safety mapping at time t, that is, the safety action that will eventually be executed in the DCS. The action before the safe mapping at time t, is the total cold source flowing into all buildings at time t. After the safety mapping of the safety layer, the safety layer will provide negative reward penalties to the agent for learning. The update parameters in the model training process include the model parameters of the safety layer, that is, the correction coefficient.

[0151] In some other embodiments, for the user power recovery process, before training the optimization strategy model based on the simulated operation data of the district cooling system simulation model, the process further includes:

[0152] Taking into account the user power recovery process, constructing a reward function corresponding to the user power recovery process based on an adaptive factor;

[0153] Simulating a user power recovery process of an actual district cooling system to perform a corresponding interaction between an intelligent agent and a district cooling system simulation model, and obtaining state transition data corresponding to the user power recovery process;

[0154] The state transition data corresponding to the user power recovery process is stored in the experience buffer.

[0155] In this example implementation, there are two stages in the process of cooperating with the power system regulation, namely the power supply regulation stage during the peak power consumption period and the user power recovery stage during the non-peak power consumption period. This example designs an adaptive reward function for the power recovery stage after the user participates in the regulation to achieve smooth power recovery and avoid peak power rebound that brings secondary impact to the power system. In the user power recovery stage, the reward function is modified to achieve the effect of smooth recovery. The expression of the reward function corresponding to the user power recovery process is:

[0156]

[0157] Among them, r t',re is the reward function at time t' in the user power recovery process, i is the building index, I is the building set, t' is the time corresponding to the user power recovery process, t1 is the starting time of the user power recovery process, which is also the end time of the power supply control phase, t2 is the end time of the user power recovery process, ΔT i,t' is the temperature difference between the room temperature of the i-th building at time t' and the temperature set by the user, is the power recovery adaptive factor of the i-th building at time t', E represents the expected value, is the temperature difference between the room temperature of the i-th building at time t1 and the user-set temperature, λ is the recovery parameter, which can be determined according to the degree of recovery of the indoor temperature at time t2. For example, when λ is set to 6, the deviation between the indoor temperature and the target comfort temperature (user-set temperature) can reach Within 5% of It is the deviation between the indoor temperature of the i-th building at time t1 and the temperature set by the user.

[0158] From the above, it can be seen that the present invention designs different reward functions for the two stages of the control process, that is, the reward function for the power supply control stage is formula (5) and the reward function for the user power recovery stage is formula (13) and (14). Considering that the interactive characteristics of the power recovery process are different from the supply control process, this example simulates the user power recovery process of the actual district cooling system to interact with the corresponding intelligent agent and the district cooling system simulation model, obtains the state transfer data corresponding to the user power recovery process, and stores the state transfer data corresponding to the user power recovery process in the experience buffer. In other words, the experience buffer stores the state transfer data corresponding to the two control stages. Through the sample sampling of the experience buffer and the training process using the samples, the training process is also divided into two stages, namely the model training stage based on the reward function of formula (5) and the model training stage based on the reward function of formula (13). In the above two-stage training process, the actor network of the DDPG algorithm estimates the value of the current state-action pair. Since the reward functions in the power supply regulation and restoration stages are different, the Critic network learns the value of each stage through different samples in the experience buffer, and finally gradually optimizes the network parameters, that is, an optimization strategy model suitable for both stages is obtained. The action network (actor network) in the trained optimization strategy model is used as the air conditioning regulation strategy model for the clustering cluster users corresponding to the corresponding simulation model.

[0159] Example 2

[0160] Based on the same inventive concept, the present invention also provides a cross-zone air conditioning control method, comprising the following steps:

[0161] Obtain the operating status data of the regional cooling system where the users in each cluster are located at the current moment;

[0162] For each cluster, based on the operating status data of the regional cooling system, the air conditioning control strategy model corresponding to the cluster constructed in Example 1 is used to determine the inflow cold source adjustment amount of each user in the cluster;

[0163] The air-conditioning power of the users in the cluster is regulated based on the inflow cold source regulation amount of each user in the cluster.

[0164] In this example implementation, the air conditioning control strategy model can be deployed in the actual DCS where the user is located, that is, after the model training converges, the corresponding model of the same cluster is deployed in the actual DCS of all users in the cluster. The operating status data in the actual DCS at the current moment is obtained and input into the corresponding air conditioning control strategy model. The model will output the corresponding action, that is, the adjustment amount of the cold source flowing into each user in the cluster. Based on the model output, the air conditioning power of the users in the cluster is controlled, that is, the corresponding parameters in the DCS are changed, such as the mass flow rate of the water pumps of each building in the area in the first water cycle, and the actual DCS system of each user in the cluster is controlled to complete the power reduction regulation and user power recovery process during the power supply guarantee period. Exemplarily, the operating status data includes the power difference between the actual power of the regional cooling system and the power upper limit, the amount of cold source flowing into each building in the regional cooling system, the temperature of the cold source flowing into each building in the regional cooling system, and the temperature difference between the room temperature of each building and the user set temperature. The state s in the corresponding model t .

[0165] In an example implementation, air conditioning power control of users in the cluster is performed based on the cold source adjustment amount flowing from the energy station to each user in the cluster, including:

[0166] If the current moment is in the power supply guarantee regulation period, power reduction regulation is performed based on the cold source regulation amount flowing into each user in the cluster from the energy station, so that the cooling system in the area where each user in the cluster is located provides energy reserve for the power system;

[0167] If the current moment is in the user power recovery period, power recovery of users in the cluster is performed based on the cold source adjustment amount of the energy station flowing into each user in the cluster, so that the actual indoor temperature of the users in the cluster approaches the user set temperature.

[0168] In this example implementation, the control process is divided into a power supply control period and a user power recovery period. The corresponding time configuration can be made for different control periods in advance, and DCS control can be performed based on the control period at the current moment to provide operation reserves for the power system or provide comfortable temperature for users. Different control effects can be obtained in different periods, so that operation reserves can be provided for the power system during the peak period of power consumption of the power system to ensure the stable operation of the power system, and the user experience can be guaranteed during the non-peak period.

[0169] Urban buildings are the main area of ​​energy consumption in the power grid, and the economical and efficient operation of the urban power grid is the key to achieving the goal of "carbon peak and carbon neutrality". As an important part of the total power consumption of the urban power grid, the air conditioning system of commercial buildings has the potential to become a regulating resource to participate in the load regulation of the power grid. At the same time, with the help of the energy storage characteristics of the building, it can be regulated while meeting the user's indoor temperature comfort needs, and its potential is huge. In order to better explore the regulation capabilities of commercial building air conditioning systems, the regulation technology of commercial building air conditioning systems has received increasing attention and research. District cooling systems (DCS), as a large-capacity and energy-saving HVAC system (up to 100 MW), are emerging in modern cities and have great potential for regulation as flexible loads. However, due to the complexity of the DCS system and the uncertainty of cooling demand, controlling DCS is quite challenging. Among them, complexity refers to the fact that in order to provide operating reserve services, an accurate thermodynamic model of the DCS is usually developed to describe the relationship between operating power and mass flow. However, this is challenging because the thermodynamics of the DCS includes cooling power generation, transportation and consumption, which is usually very complex. Traditional model-based HVAC system control methods are difficult to use in DCS. In addition, the power consumption of DCS is usually automatically adjusted according to its operating status, and its power cannot be directly controlled to provide operating reserves like other demand-side resources. Uncertainty refers to the fact that the power consumption of DCS and the indoor temperature of the building are related to the ambient temperature and indoor human behavior. Higher ambient temperatures and more indoor human activities require more cooling supply and higher power consumption. However, ambient temperature may have different effects on heterogeneous buildings (buildings with different materials, structures, etc.). Indoor human behavior is also random and difficult to accurately predict. Therefore, it is not easy to control DCS subject to heterogeneous indoor temperature constraints in multiple buildings. To solve the above problems, technology based on deep reinforcement learning has received widespread attention and has been applied to air conditioning control systems. This technology does not require a priori thermodynamic models of urban buildings. It learns the optimal control strategy through continuous exploratory interaction with the building environment. Once the learning process is completed, the learned strategy can be used for end-to-end real-time decision control. Typical reinforcement learning methods include deep deterministic policy gradient algorithm, asynchronous advantage actor critic (A3C) algorithm, trust region policy optimization algorithm (TRPO), etc. Although these methods are effective in building HVAC (Heating, Ventilation, and Air Conditioning) control systems, they are less studied in DCS control, and generally speaking, controllers based on deep reinforcement learning must undergo a lot of trial and error training before they can become intelligent.This means that some “wrong” decisions may be made during training, which can easily cause system corruption.

[0170] Based on the above technical problems, the present invention proposes a method for cross-regional DCS control and recovery based on deep reinforcement learning. In the context of considering the power demand of each region, in order to maintain the consistency of geographical aggregation, a cross-regional hierarchical clustering technology based on deep learning is adopted. The clustering analysis method can make full use of the common characteristics of similar users in different geographical regions, make full use of regional air-conditioning load resources to participate in regulation, and provide a data model with more data utilization efficiency for subsequent intelligent regulation. In order to target commercial buildings, which are the main control objects of power grid air-conditioning resources, for DCS system modeling and simulation, in order to solve the complexity of DCS system control and the uncertainty of cooling demand, to provide operation reserves to the power trading market for power grid regulation, an intelligent control and recovery strategy based on deep reinforcement learning is adopted, the system control process structure is expressed as a Markov decision process, key constraints are added in the system control stage and the recovery stage, and a safety layer is designed in the control stage to improve the safety and feasibility of the model during DCS system training, thereby improving the system safety of the control process after the model is deployed. The present invention provides a method for cross-regional aggregation of multi-DCS system operation reserves under the conditions of meeting the constraints of indoor temperature comfort, no illegal and destructive operations, and power smooth recovery, so as to provide operation capacity for the power trading market.

[0171] Embodiment 3:

[0172] Based on the same inventive concept, the present invention also provides a cross-zone air conditioning control strategy modeling system, comprising:

[0173] A clustering module is used to perform clustering processing on the air conditioning load data of users in each region to obtain clustering results;

[0174] A simulation module, used for building a district cooling system simulation model corresponding to the cluster based on users in the same cluster in the clustering result;

[0175] A strategy model building module is used for modeling the control process of each district cooling system simulation model based on the Markov decision process on the basis of considering the operation safety of the actual district cooling system, and building an optimization strategy model with the goal of maximizing the expected return of the value function corresponding to the operation strategy of the district cooling system simulation model;

[0176] A strategy model training module is used to train the optimization strategy model based on the simulated operation data of the regional cooling system simulation model, and use the action network corresponding to the trained optimization strategy model as the air conditioning control strategy model for users in the cluster corresponding to the regional cooling system simulation model.

[0177] In a possible implementation, the clustering module includes:

[0178] Configuration submodule, used to configure different numbers of cluster centers;

[0179] The clustering submodule is used to perform clustering processing for each configuration of the number of cluster centers, with the goal of minimizing the distance between the air-conditioning load data of users in each region and each preset cluster center, so as to cluster the air-conditioning load data of users in each region into the cluster cluster corresponding to the cluster center with the closest distance, and obtain the clustering result under each configuration of the number of cluster centers;

[0180] The performance evaluation submodule is used to perform performance evaluation of clustering results under each cluster center number configuration based on the graph convolutional network, and screen out the clustering results under the target cluster center number configuration as the final clustering results based on the evaluation results.

[0181] In a possible implementation manner, the air conditioning load data is an air conditioning load sequence, and the performance evaluation submodule is specifically used for:

[0182] For the clustering results under each configuration of the number of cluster centers, a load prediction sub-model corresponding to the cluster is trained based on the air-conditioning load sequence in each cluster;

[0183] Based on fusing the model parameters of the trained load forecasting sub-models corresponding to each cluster, a fusion parameter is obtained;

[0184] The fusion parameters are used as initial model parameters of the graph convolutional network, and the graph convolutional network is trained based on the air conditioning load sequences of users in various regions to obtain a global load forecasting model; each configuration of the number of cluster centers corresponds to a global load forecasting model;

[0185] Based on the prediction error of the global load forecasting model corresponding to each configuration of the number of cluster centers, the performance evaluation of the clustering results under the corresponding configuration of the number of cluster centers is carried out.

[0186] In a possible implementation, the simulation module includes:

[0187] A building parameter determination submodule, for treating the users in the same cluster in the clustering result as a whole, and determining the building parameters of the district cooling system simulation model corresponding to the cluster based on the fusion processing of the building conditions of the users in the same cluster;

[0188] A cooling parameter determination submodule, used for determining the cooling associated parameters of the regional cooling system simulation model corresponding to the cluster based on the fusion processing of the cooling conditions of each user in the same cluster;

[0189] The simulation modeling submodule is used to construct a simulation model of the district cooling system corresponding to the cluster based on the building parameters and cooling-related parameters of the district cooling system corresponding to the cluster.

[0190] In a possible implementation, the regional cooling system simulation model includes a first water circulation loop from the energy station to each building, a second water circulation loop from each building to the room, and an air circulation loop in each room.

[0191] In a possible implementation, the policy model building module includes a problem conversion submodule, and the problem conversion submodule is used to:

[0192] For each district cooling system simulation model, a corresponding intelligent controller is constructed to control the amount of cold source flowing from the energy station in the district cooling system simulation model into each building;

[0193] An intelligent controller is used as an intelligent agent, and the power difference between the actual power corresponding to the simulation model of the regional cooling system and the upper power limit, the amount of cold source flowing into each building in the simulation model of the regional cooling system, the temperature of the cold source flowing into each building in the simulation model of the regional cooling system, and the temperature difference between the room temperature of each building and the user-set temperature are used as observation values ​​of the corresponding intelligent agent;

[0194] The cold source adjustment amount flowing into each building in the district cooling system simulation model is used as the action of the corresponding intelligent agent;

[0195] Considering the operational safety of an actual district cooling system corresponding to the district cooling system simulation model, a security layer is constructed between the intelligent agent and the district cooling system simulation model to perform security processing on actions input into the district cooling system simulation model;

[0196] The reward of the corresponding intelligent agent is determined based on the expectation of the temperature difference between the room temperature of each building and the temperature set by the user, the variance of the temperature difference between the room temperature of each building and the temperature set by the user, and the negative reward penalty corresponding to the safety processing.

[0197] In a possible implementation, the policy model construction module includes a policy model construction submodule, and the policy model construction submodule is used to:

[0198] Taking the expected return of the value function corresponding to the operation strategy of the district cooling system simulation model as the objective function;

[0199] Constructing an action network and a value network for solving the objective function based on a deep deterministic policy gradient algorithm;

[0200] The optimization strategy model is determined based on the action network and the value network.

[0201] In a possible implementation, the control process includes a power supply control process, and the system further includes a first state transfer module, and the first state transfer module is used to:

[0202] The simulation model of the district cooling system is used as an environment, and the interaction between the corresponding intelligent agent and the environment is performed by simulating the power supply regulation process of the actual district cooling system to obtain state transition data corresponding to the power supply regulation process;

[0203] The state transition data corresponding to the power supply guarantee and regulation process is stored in the experience buffer.

[0204] In a possible implementation, the policy model training module includes:

[0205] A sampling submodule, configured to obtain corresponding training samples based on sampling of data in the experience buffer;

[0206] A training submodule, which trains the optimization strategy model based on the training samples to update the model parameters of the action network, the value network and the security layer;

[0207] Among them, during the training process, before the action corresponding to the training sample is input into the district cooling system simulation model, the operational safety of the current action is checked based on the safety layer. If the check fails, the safety mapping of the current action is performed based on the model parameters of the safety layer to obtain the safety action corresponding to the current action, and the safety action is input into the district cooling system simulation model for model training.

[0208] In a possible implementation, the training submodule is specifically used for:

[0209] By calling the simulation system to simulate the current action running in the simulation model of the district cooling system, the simulated operating power corresponding to the current action is obtained;

[0210] If the simulated operating power corresponding to the current action exceeds the power upper limit of the regional cooling system simulation model, it is determined that the safety inspection of the current action has failed.

[0211] In a possible implementation manner, the control process further includes a user power recovery process, and the system further includes a second state transfer module, wherein the second state transfer module is used to:

[0212] Taking into account the user power recovery process, constructing a reward function corresponding to the user power recovery process based on an adaptive factor;

[0213] Simulating a user power recovery process of an actual district cooling system to perform a corresponding interaction between an intelligent agent and a district cooling system simulation model, and obtaining state transition data corresponding to the user power recovery process;

[0214] The state transition data corresponding to the user power recovery process is stored in the experience buffer.

[0215] In a possible implementation, the reward function corresponding to the user power recovery process is expressed as:

[0216]

[0217] Among them, r t',re is the reward function at time t' during the user power recovery process, i is the building index, I is the building set, t' is the time corresponding to the user power recovery process, t1 is the start time of the user power recovery process, t2 is the end time of the power recovery process, ΔT i,t' is the temperature difference between the room temperature of the i-th building at time t' and the temperature set by the user, is the power recovery adaptive factor of the i-th building at time t', E represents the expected value, is the temperature difference between the room temperature of the i-th building at time t1 and the user-set temperature, and λ is the recovery parameter.

[0218] Example 4

[0219] Based on the same inventive concept, the present invention also provides a cross-zone air conditioning control system, comprising:

[0220] An acquisition module is used to obtain the operation status data of the regional cooling system where the users in each cluster are located at the current moment;

[0221] A control module is used to determine, for each cluster, based on the operating status data of the regional cooling system, the air-conditioning control strategy model corresponding to the cluster constructed by any implementation method of Example 1 to determine the cold source adjustment amount of the energy station flowing into the users in the cluster; and to control the air-conditioning power of the users in the cluster based on the cold source adjustment amount of the energy station flowing into the users in the cluster.

[0222] In a possible implementation manner, the operation status data includes a power difference between an actual power of the regional cooling system and a power upper limit, an amount of cold source flowing into each building in the regional cooling system, a temperature of the cold source flowing into each building in the regional cooling system, and a temperature difference between a room temperature of each building and a user-set temperature;

[0223] In a possible implementation manner, the control module is specifically used to:

[0224] If the current moment is in the power supply guarantee regulation period, power reduction regulation is performed based on the cold source regulation amount flowing into the users in the cluster from the energy station, so that the cooling system in the area where each user in the cluster is located provides energy reserve for the power system;

[0225] If the current moment is in the user power recovery period, power recovery of users in the cluster is performed based on the cold source adjustment amount flowing into the users in the cluster from the energy station, so that the actual indoor temperature of users in the cluster approaches the user set temperature.

[0226] Example 5

[0227] like Figure 3 As shown, the present invention also provides an electronic device, which may be a computer device, a single-chip device, an intelligent mobile device, etc. The electronic device in this embodiment may include a processor, a memory, a transceiver component, etc. The memory, the processor, and the transceiver component are connected via a bus; the memory may be used to store an execution program, and an exemplary execution program may include instructions; the processor is used to execute the instructions stored in the memory. The memory may also be used to store data, which may be called and / or modified when the instructions are executed.

[0228] The processor may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components, etc. It is the computing core and control core of the terminal, which is suitable for implementing one or more instructions, specifically suitable for loading and executing one or more instructions in a storage medium to implement the corresponding method flow or corresponding function, so as to implement a cross-zone air-conditioning control strategy modeling method or the steps of the cross-zone air-conditioning control method in the above-mentioned embodiment.

[0229] Example 6

[0230] Based on the same inventive concept, the present invention also provides a readable storage medium implementation, specifically an electronic device readable storage medium (Memory), which is a memory device in an electronic device for storing programs and data. It can be understood that the storage medium here can include both built-in storage media in electronic devices and, of course, extended storage media supported by electronic devices. The storage medium provides a storage space that stores the operating system of the terminal. In addition, one or more instructions suitable for being loaded and executed by a processor are also stored in the storage space, and these instructions can be one or more execution programs (including program codes). It should be noted that the storage medium here can be a high-speed RAM memory or a non-volatile memory, such as at least one disk storage. The processor loads and executes one or more instructions stored in the storage medium, which can implement a step of a cross-zone air conditioning control strategy modeling method or a cross-zone air conditioning control method in the above-mentioned embodiment.

[0231] Those skilled in the art will appreciate that embodiments of the present invention may be provided as methods, systems, or computer program products. Therefore, the present invention may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0232] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0233] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1A function specified in one or more boxes.

[0234] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0235] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention rather than to limit its protection scope. Although the present invention has been described in detail with reference to the above embodiments, ordinary technicians in the field should understand that after reading the present invention, those skilled in the art can still make various changes, modifications or equivalent substitutions to the specific implementation methods of the application, but these changes, modifications or equivalent substitutions are all within the protection scope of the claims to be approved.

Claims

1. A cross-region air conditioning control strategy modeling method, characterized in that: include: Based on clustering the air conditioning load data of users in each region, the clustering results are obtained; Building a district cooling system simulation model corresponding to the cluster based on users in the same cluster in the clustering result; For each district cooling system simulation model, on the basis of considering the operational safety of the actual district cooling system, the control process of the district cooling system simulation model is modeled based on the Markov decision process, and an optimization strategy model is constructed with the goal of maximizing the expected return of the value function corresponding to the operation strategy of the district cooling system simulation model; The optimization strategy model is trained based on the simulated operation data of the regional cooling system simulation model, and the action network corresponding to the trained optimization strategy model is used as the air conditioning control strategy model for users in the cluster corresponding to the regional cooling system simulation model.

2. The method according to claim 1, characterized in that Based on clustering of the air conditioning load data of users in various regions, the clustering results are obtained, including: Configure different numbers of cluster centers; For each configuration of the number of cluster centers, clustering is performed with the goal of minimizing the distance between the air-conditioning load data of users in each region and each preset cluster center, so as to cluster the air-conditioning load data of users in each region into the cluster cluster corresponding to the cluster center with the closest distance, and obtain the clustering results under each configuration of the number of cluster centers; Based on the graph convolutional network, the performance of the clustering results under each configuration of the number of cluster centers is evaluated, and based on the evaluation results, the clustering results under the target number of cluster centers are selected as the final clustering results.

3. The method according to claim 2, characterized in that The air conditioning load data is an air conditioning load sequence. The performance evaluation of the clustering results under each cluster center number configuration is performed based on the graph convolutional network, including: For the clustering results under each configuration of the number of cluster centers, a load prediction sub-model corresponding to the cluster is trained based on the air-conditioning load sequence in each cluster; Based on fusing the model parameters of the trained load forecasting sub-models corresponding to each cluster, a fusion parameter is obtained; The fusion parameters are used as initial model parameters of the graph convolutional network, and the graph convolutional network is trained based on the air conditioning load sequences of users in various regions to obtain a global load forecasting model; each configuration of the number of cluster centers corresponds to a global load forecasting model; Based on the prediction error of the global load forecasting model corresponding to each configuration of the number of cluster centers, the performance evaluation of the clustering results under the corresponding configuration of the number of cluster centers is carried out.

4. The method according to claim 1, characterized in that: Performing district cooling system simulation modeling for users in the same cluster in the clustering result to obtain a district cooling system simulation model corresponding to the cluster, including: Taking the users in the same cluster in the clustering result as a whole, based on fusing the building conditions of the users in the same cluster, determine the building parameters of the district cooling system simulation model corresponding to the cluster; Determine the cooling associated parameters of the regional cooling system simulation model corresponding to the cluster based on the fusion processing of the cooling conditions of each user in the same cluster; Based on the building parameters and cooling-related parameters of the district cooling system corresponding to the cluster, a simulation model of the district cooling system corresponding to the cluster is constructed.

5. The method according to any one of claims 1 to 4, characterized in that: The simulation model of the regional cooling system includes a first water circulation loop from the energy station to each building, a second water circulation loop from each building to the room, and an air circulation loop in each room.

6. The method according to claim 5, characterized in that The method of modeling the control process of the simulation model of the district cooling system based on the Markov decision process, taking into account the operational safety of the actual district cooling system, includes: For each district cooling system simulation model, a corresponding intelligent controller is constructed to control the amount of cold source flowing from the energy station in the district cooling system simulation model into each building; An intelligent controller is used as an intelligent agent, and the power difference between the actual power corresponding to the simulation model of the regional cooling system and the upper power limit, the amount of cold source flowing into each building in the simulation model of the regional cooling system, the temperature of the cold source flowing into each building in the simulation model of the regional cooling system, and the temperature difference between the room temperature of each building and the user-set temperature are used as observation values ​​of the corresponding intelligent agent; The cold source adjustment amount flowing into each building in the district cooling system simulation model is used as the action of the corresponding intelligent agent; Considering the operational safety of an actual district cooling system corresponding to the district cooling system simulation model, a security layer is constructed between the intelligent agent and the district cooling system simulation model to perform security processing on actions input into the district cooling system simulation model; The reward of the corresponding intelligent agent is determined based on the expectation of the temperature difference between the room temperature of each building and the temperature set by the user, the variance of the temperature difference between the room temperature of each building and the temperature set by the user, and the negative reward penalty corresponding to the safety processing.

7. The method according to claim 6, characterized in that An optimization strategy model is constructed with the goal of maximizing the expected return of the value function corresponding to the operation strategy of the district cooling system simulation model, including: Taking the expected return of the value function corresponding to the operation strategy of the district cooling system simulation model as the objective function; Constructing an action network and a value network for solving the objective function based on a deep deterministic policy gradient algorithm; The optimization strategy model is determined based on the action network and the value network.

8. The method according to claim 7, characterized in that The control process includes a power supply control process, the simulation operation data includes state transition data corresponding to the power supply control process, and before training the optimization strategy model based on the simulation operation data of the district cooling system simulation model, it also includes: The simulation model of the district cooling system is used as an environment, and the interaction between the corresponding intelligent agent and the environment is performed by simulating the power supply regulation process of the actual district cooling system to obtain state transition data corresponding to the power supply regulation process; The state transition data corresponding to the power supply guarantee and regulation process is stored in the experience buffer.

9. The method according to claim 8, characterized in that Training the optimization strategy model based on simulated operation data of the district cooling system simulation model includes: Based on the data sampling of the experience buffer, a corresponding training sample is obtained; Training the optimization strategy model based on the training samples to update model parameters of the action network, the value network, and the security layer; Among them, during the training process, before the action corresponding to the training sample is input into the district cooling system simulation model, the operational safety of the current action is checked based on the safety layer. If the check fails, the safety mapping of the current action is performed based on the model parameters of the safety layer to obtain the safety action corresponding to the current action, and the safety action is input into the district cooling system simulation model for model training.

10. The method according to claim 9, characterized in that The checking of the running safety of the current action includes: By calling the simulation system to simulate the current action running in the simulation model of the district cooling system, the simulated operating power corresponding to the current action is obtained; If the simulated operating power corresponding to the current action exceeds the power upper limit of the regional cooling system simulation model, it is determined that the safety inspection of the current action has failed.

11. The method according to claim 9, characterized in that The control process also includes a user power recovery process, the simulated operation data includes state transition data corresponding to the user power recovery process, and before training the optimization strategy model based on the simulated operation data of the district cooling system simulation model, it also includes: Taking into account the user power recovery process, constructing a reward function corresponding to the user power recovery process based on an adaptive factor; Simulating a user power recovery process of an actual district cooling system to perform a corresponding interaction between an intelligent agent and a district cooling system simulation model, and obtaining state transition data corresponding to the user power recovery process; The state transition data corresponding to the user power recovery process is stored in the experience buffer.

12. The method according to claim 11, characterized in that The expression of the reward function corresponding to the user power recovery process is: Among them, r t',re is the reward function at time t' in the user power recovery process, i is the building index, I is the building set, t' is the time corresponding to the user power recovery process, t1 is the start time of the user power recovery process, t2 is the end time of the user power recovery process, ΔT i,t' is the temperature difference between the room temperature of the i-th building at time t' and the temperature set by the user, is the power recovery adaptive factor of the i-th building at time t', E represents the expected value, is the temperature difference between the room temperature of the i-th building at time t1 and the user-set temperature, and λ is the recovery parameter.

13. A cross-zone air conditioning control method, characterized in that: include: Obtain the operating status data of the regional cooling system where the users in each cluster are located at the current moment; For each cluster, based on the operating status data of the regional cooling system, the air conditioning control strategy model corresponding to the cluster constructed according to any one of claims 1 to 12 is used to determine the cold source adjustment amount flowing from the energy station to the users in the cluster; The air-conditioning power of the users in the cluster is regulated based on the cold source regulation amount flowing into the users in the cluster from the energy station.

14. The method according to claim 13, characterized in that The operating status data includes the power difference between the actual power of the regional cooling system and the power upper limit, the amount of cold source flowing into each building in the regional cooling system, the temperature of the cold source flowing into each building in the regional cooling system, and the temperature difference between the room temperature of each building and the user-set temperature.

15. The method according to claim 13, characterized in that The air conditioning power of the users in the cluster is regulated based on the cold source regulation amount flowing from the energy station to each user in the cluster, including: If the current moment is in the power supply guarantee regulation period, power reduction regulation is performed based on the cold source regulation amount flowing into each user in the cluster from the energy station, so that the cooling system in the area where the users in the cluster are located can provide energy reserve for the power system; If the current moment is in the user power recovery period, power recovery of users in the cluster is performed based on the cold source adjustment amount flowing into the users in the cluster from the energy station, so that the actual indoor temperature of users in the cluster approaches the user set temperature.

16. A cross-region air conditioning control strategy modeling system, characterized in that: include: A clustering module is used to perform clustering processing on the air conditioning load data of users in each region to obtain clustering results; A simulation module, used for constructing a district cooling system simulation model corresponding to the cluster based on users in the same cluster in the clustering result; A strategy model building module is used for modeling the control process of each district cooling system simulation model based on the Markov decision process on the basis of considering the operation safety of the actual district cooling system, and building an optimization strategy model with the goal of maximizing the expected return of the value function corresponding to the operation strategy of the district cooling system simulation model; A strategy model training module is used to train the optimization strategy model based on the simulated operation data of the regional cooling system simulation model, and use the action network corresponding to the trained optimization strategy model as the air conditioning control strategy model for users in the cluster corresponding to the regional cooling system simulation model.

17. The system according to claim 16, characterized in that The clustering module comprises: Configuration submodule, used to configure different numbers of cluster centers; The clustering submodule is used to perform clustering processing for each configuration of the number of cluster centers, with the goal of minimizing the distance between the air-conditioning load data of users in each region and each preset cluster center, so as to cluster the air-conditioning load data of users in each region into the cluster cluster corresponding to the cluster center with the closest distance, and obtain the clustering result under each configuration of the number of cluster centers; The performance evaluation submodule is used to perform performance evaluation of clustering results under each cluster center number configuration based on the graph convolutional network, and screen out the clustering results under the target cluster center number configuration as the final clustering results based on the evaluation results.

18. The system according to claim 17, characterized in that The air conditioning load data is an air conditioning load sequence, and the performance evaluation submodule is specifically used for: For the clustering results under each configuration of the number of cluster centers, a load prediction sub-model corresponding to the cluster is trained based on the air-conditioning load sequence in each cluster; Based on fusing the model parameters of the trained load forecasting sub-models corresponding to each cluster, a fusion parameter is obtained; The fusion parameters are used as initial model parameters of the graph convolutional network, and the graph convolutional network is trained based on the air conditioning load sequences of users in various regions to obtain a global load forecasting model; each configuration of the number of cluster centers corresponds to a global load forecasting model; Based on the prediction error of the global load forecasting model corresponding to each configuration of the number of cluster centers, the performance evaluation of the clustering results under the corresponding configuration of the number of cluster centers is carried out.

19. The system according to claim 16, characterized in that The simulation module comprises: A building parameter determination submodule, for treating the users in the same cluster in the clustering result as a whole, and determining the building parameters of the district cooling system simulation model corresponding to the cluster based on the fusion processing of the building conditions of the users in the same cluster; A cooling parameter determination submodule, configured to determine the cooling associated parameters of the regional cooling system simulation model corresponding to the cluster based on the fusion processing of the cooling conditions of each user in the same cluster; The simulation modeling submodule is used to construct a simulation model of the district cooling system corresponding to the cluster based on the building parameters and cooling-related parameters of the district cooling system corresponding to the cluster.

20. The system according to any one of claims 16 to 19, characterized in that: The simulation model of the regional cooling system includes a first water circulation loop from the energy station to each building, a second water circulation loop from each building to the room, and an air circulation loop in each room.

21. The system according to claim 20, characterized in that The strategy model building module includes a problem conversion submodule, and the problem conversion submodule is used to: For each district cooling system simulation model, a corresponding intelligent controller is constructed to control the amount of cold source flowing from the energy station in the district cooling system simulation model into each building; An intelligent controller is used as an intelligent agent, and the power difference between the actual power corresponding to the simulation model of the regional cooling system and the upper power limit, the amount of cold source flowing into each building in the simulation model of the regional cooling system, the temperature of the cold source flowing into each building in the simulation model of the regional cooling system, and the temperature difference between the room temperature of each building and the user-set temperature are used as observation values ​​of the corresponding intelligent agent; The cold source adjustment amount flowing into each building in the district cooling system simulation model is used as the action of the corresponding intelligent agent; Considering the operational safety of an actual district cooling system corresponding to the district cooling system simulation model, a security layer is constructed between the intelligent agent and the district cooling system simulation model to perform security processing on actions input into the district cooling system simulation model; The reward of the corresponding intelligent agent is determined based on the expectation of the temperature difference between the room temperature of each building and the temperature set by the user, the variance of the temperature difference between the room temperature of each building and the temperature set by the user, and the negative reward penalty corresponding to the safety processing.

22. The system according to claim 21, characterized in that The policy model construction module includes a policy model construction submodule, and the policy model construction submodule is used to: Taking the expected return of the value function corresponding to the operation strategy of the district cooling system simulation model as the objective function; Constructing an action network and a value network for solving the objective function based on a deep deterministic policy gradient algorithm; The optimization strategy model is determined based on the action network and the value network.

23. The system according to claim 22, characterized in that The control process includes a power supply control process, and the system further includes a first state transfer module, and the first state transfer module is used to: The simulation model of the district cooling system is used as an environment, and the interaction between the corresponding intelligent agent and the environment is performed by simulating the power supply regulation process of the actual district cooling system to obtain state transition data corresponding to the power supply regulation process; The state transition data corresponding to the power supply guarantee and regulation process is stored in the experience buffer.

24. The system according to claim 23, characterized in that The strategy model training module includes: A sampling submodule, configured to obtain corresponding training samples based on sampling of data in the experience buffer; A training submodule, which trains the optimization strategy model based on the training samples to update the model parameters of the action network, the value network and the security layer; Among them, during the training process, before the action corresponding to the training sample is input into the district cooling system simulation model, the operational safety of the current action is checked based on the safety layer. If the check fails, the safety mapping of the current action is performed based on the model parameters of the safety layer to obtain the safety action corresponding to the current action, and the safety action is input into the district cooling system simulation model for model training.

25. The system according to claim 24, characterized in that The training submodule is specifically used for: By calling the simulation system to simulate the current action running in the simulation model of the district cooling system, the simulated operating power corresponding to the current action is obtained; If the simulated operating power corresponding to the current action exceeds the power upper limit of the regional cooling system simulation model, it is determined that the safety inspection of the current action has failed.

26. The system according to claim 24, characterized in that The control process also includes a user power recovery process, and the system also includes a second state transfer module, and the second state transfer module is used to: Taking into account the user power recovery process, constructing a reward function corresponding to the user power recovery process based on an adaptive factor; Simulating a user power recovery process of an actual district cooling system to perform a corresponding interaction between an intelligent agent and a district cooling system simulation model, and obtaining state transition data corresponding to the user power recovery process; The state transition data corresponding to the user power recovery process is stored in the experience buffer.

27. The system according to claim 26, characterized in that The expression of the reward function corresponding to the user power recovery process is: Among them, r t',re is the reward function at time t' during the user power recovery process, i is the building index, I is the building set, t' is the time corresponding to the user power recovery process, t1 is the start time of the user power recovery process, t2 is the end time of the power recovery process, ΔT i,t' is the temperature difference between the room temperature of the i-th building at time t' and the temperature set by the user, is the power recovery adaptive factor of the i-th building at time t', E represents the expected value, is the temperature difference between the room temperature of the i-th building at time t1 and the user-set temperature, and λ is the recovery parameter.

28. A cross-zone air conditioning control system, characterized in that: include: An acquisition module is used to obtain the operation status data of the regional cooling system where the users in each cluster are located at the current moment; A control module is used to determine, for each cluster, based on the operating status data of the regional cooling system, the air-conditioning control strategy model corresponding to the cluster constructed by any one of claims 1 to 12 to determine the cold source adjustment amount of the energy station flowing into the users in the cluster; and to control the air-conditioning power of the users in the cluster based on the cold source adjustment amount of the energy station flowing into the users in the cluster.

29. The system according to claim 28, characterized in that The operating status data includes the power difference between the actual power of the regional cooling system and the power upper limit, the amount of cold source flowing into each building in the regional cooling system, the temperature of the cold source flowing into each building in the regional cooling system, and the temperature difference between the room temperature of each building and the user-set temperature.

30. The system according to claim 28, characterized in that The control module is specifically used for: If the current moment is in the power supply guarantee regulation period, power reduction regulation is performed based on the cold source regulation amount flowing into the users in the cluster from the energy station, so that the cooling system in the area where each user in the cluster is located provides energy reserve for the power system; If the current moment is in the user power recovery period, power recovery of users in the cluster is performed based on the cold source adjustment amount flowing into the users in the cluster from the energy station, so that the actual indoor temperature of users in the cluster approaches the user set temperature.

31. An electronic device comprising: at least one processor and memory; The memory and the processor are connected via a bus; The memory is used to store one or more programs; When the one or more programs are executed by the at least one processor, the method according to any one of claims 1 to 15 is implemented.

32. A readable storage medium having an execution program stored thereon, wherein when the execution program is executed, the method according to any one of claims 1 to 15 is implemented.

Images