Temperature control load cluster characteristic analysis method, system, device and readable storage medium

Abstract

A temperature-controlled load cluster characteristic analysis method, system, device and readable storage medium, the method comprises the following steps: based on the pre-established air conditioning load thermodynamic model, a temperature-controlled load numerical calculation model is constructed, and a temperature-controlled load cluster controller is designed to construct a temperature-controlled load cluster simulation environment; selecting the observation data of the temperature-controlled load cluster to form an observation vector, designing the action vector and the reward function of the temperature-controlled load cluster regulation characteristic analysis, and constructing the temperature-controlled load cluster adjustable feature analysis learning environment; using a deep reinforcement learning algorithm, generating a adjustable range according to the observation vector through a strategy neural network, interacting the decision result with the learning environment within the adjustable range, obtaining action experience, and updating the strategy neural network; the trained temperature-controlled load cluster feature analysis neural network is used for temperature-controlled load cluster characteristic analysis. The present application adopts model and data double driving, realizes the analysis of the overall adjustable characteristics by using the measurement data of part of the temperature-controlled load equipment.

Description

Technical Field

[0001] This invention belongs to the field of power system dispatching technology, specifically relating to a method, system, equipment, and readable storage medium for analyzing the characteristics of temperature-controlled load clusters. Background Technology

[0002] With the integration of a high proportion of new energy sources, especially distributed photovoltaic and wind power, power balance faces significant challenges. Distributed power sources are characterized by decentralized access, full grid connection, and low-voltage bare connection, leading to difficulties in coordination and supply-demand imbalances. Electric vehicles, controllable loads, and distributed energy storage, among other controllable resources, possess enormous potential, providing additional regulation capabilities for grid energy and power balance. Cooling electricity consumption accounts for over 15% of total electricity consumption, and air conditioning load in large cities accounts for approximately 60% of peak summer load, making them crucial future regulation resources. However, these resources are numerous and mostly connected to the public grid, limiting their ability to directly interact with the dispatch center on a large scale due to limitations in communication, security, and data processing capabilities.

[0003] The adjustable aggregation characteristics of temperature-controlled load clusters are currently a major research hotspot. In recent years, artificial intelligence computing for massive load resources has gradually emerged. Some researchers have used generative adversarial networks to address the problem of insufficient sample data for massive resources, laying a foundation for the learning and training of temperature-controlled load clusters to some extent. However, they still focus on aggregation based on all resource data and do not yet have practical engineering applications. Some researchers have also used the A3C algorithm to solve the problem of grid voltage control. These studies have made preliminary explorations into the application of artificial intelligence technology in power system dispatching. Since existing technologies mainly achieve statistical analysis of external characteristics of the cluster by analytically modeling temperature-controlled loads and then aggregating measurements of all devices in the cluster, it is necessary to measure, identify, and statistically analyze all devices. However, analytical modeling of temperature-controlled loads suffers from the problem of difficulty in obtaining building parameters. The power consumption characteristics of temperature-controlled loads are related to space size and building materials. In engineering applications, it is impossible to collect building parameters for all temperature-controlled loads, which leads to the low accuracy of existing model-based external characteristic aggregation technologies. The analytical method for obtaining the adjustable characteristics of temperature-controlled load clusters has the problem of not adapting to changes in users. The analytical model depends on static parameters such as user space and air conditioning equipment type. Once a new user is added or a user changes equipment, a new user model or model parameters need to be created or modified. If user changes are not updated in time, the obtained adjustable characteristics will be inaccurate. Summary of the Invention

[0004] The purpose of this invention is to address the problem in the prior art that the characteristics of massive temperature-controlled load clusters cannot be obtained through accurate modeling of all devices. This invention provides a method, system, device, and readable storage medium for analyzing the characteristics of temperature-controlled load clusters. It designs an offline learning model, samples the cluster, and uses measurement data from some temperature-controlled load devices to analyze the overall adjustability characteristics of the temperature-controlled load cluster, supporting the dispatch center or load aggregator to rationally regulate temperature-controlled load resources.

[0005] To achieve the above objectives, the present invention provides the following technical solution:

[0006] Firstly, a method for analyzing the characteristics of temperature-controlled load clusters is provided, including:

[0007] Based on the pre-established thermodynamic model of air conditioning load, a numerical calculation model of temperature control load is constructed, and a temperature control load cluster controller is designed to build a temperature control load cluster simulation environment.

[0008] Based on the temperature-controlled load cluster simulation environment, the observation data of the temperature-controlled load cluster is selected to form the observation vector, and the action vector and reward function for the temperature-controlled load cluster regulation characteristic analysis are designed to construct the temperature-controlled load cluster adjustable feature analysis learning environment.

[0009] Based on the adjustable feature analysis learning environment of temperature-controlled load clusters, a deep reinforcement learning algorithm is adopted. According to the observation vector, a schedulable range is generated through a policy neural network. Within the schedulable range, the decision results are interacted with the learning environment to obtain action experience, update the policy neural network, and complete the training of the feature analysis neural network of temperature-controlled load clusters.

[0010] The characteristics of temperature-controlled load clusters are analyzed using a trained neural network for cluster feature analysis.

[0011] As a preferred embodiment, the air conditioning load thermodynamic model is established as follows:

[0012] Assuming the temperature-controlled load chamber has a uniform temperature field, the heat conduction formula for the temperature-controlled load in the medium is:

[0013]

[0014] In the formula, Q h Where λ is the heat exchange power and λ is the thermal conductivity. Let A be the temperature gradient along the heat transfer direction, and A be the heat transfer area.

[0015]

[0016] In the formula, T in At room temperature, T out The ambient temperature;

[0017] Considering only the heat storage process of air, the formula for calculating the heat storage from t0 to t is:

[0018] ΔQ air =c air V room ρ air (T in (t)-T in (t0))

[0019] In the formula, ΔQ air c represents the change in heat storage in the air. air V is the specific heat capacity of air. room Let ρ be the volume of space. air air density;

[0020] Therefore, the thermodynamic equation for temperature-controlled load is derived as follows:

[0021]

[0022] In the formula, N represents the rate of change of room temperature. m denoted as the number of different media types in the space heat-conducting surface, where m is the sequence number of each different media type.

[0023] The following formula is derived for the room temperature evolution under the conditions of uniform medium and uniform room temperature field under temperature-controlled load:

[0024]

[0025] In the formula, C room =c air V room This parameter describes the heat storage performance of the load;

[0026] This parameter describes the performance of heat exchange between the load and the external environment.

[0027] As a preferred embodiment, the numerical calculation model for the temperature control load is constructed using the improved Euler method. The model construction process includes the following steps:

[0028] An improved Euler method is used to approximate the solution of the first-order differential equation for room temperature variation. The iterative formula for the solution is as follows:

[0029]

[0030] In the formula, T pre T is the predicted room temperature at the next time point calculated based on the rate of change of the initial temperature. cor T is the predicted room temperature at the next time point calculated based on the estimated rate of change of the endpoint temperature. i+1 This is the room temperature for the next time period after correction.

[0031] As a preferred embodiment, the design method of the temperature-controlled load cluster controller is as follows:

[0032] Cluster control is performed using a proportional controller, and the feedback loop of the proportional controller is as follows:

[0033] ΔP=k ag (P target -P rt )

[0034] In the formula, ΔP is the polymer power adjustment amount, and k ag This represents the cluster ratio control coefficient, where ag is the cluster number and P is the cluster index. target For the target power, P rt The real-time power of the cluster is measured; target power tracking is performed through the feedback loop of the proportional controller.

[0035] The temperature-controlled load cluster controller transforms the dispatch instructions issued by the dispatch center into temperature regulation signals.

[0036] The power-temperature conversion formula is as follows:

[0037] ΔT=ΔP*k P2T

[0038] In the formula, k P2T This is the power-to-temperature conversion factor;

[0039] The set temperature adjustment value of the cluster is obtained by using linear transformation.

[0040] As a preferred embodiment, in the step of selecting observation data from the temperature-controlled load cluster to construct the observation vector, the expression of the observation vector is as follows:

[0041] S t :(T env P cur P target T in_mean T in_std )

[0042] In the formula, S t Let T be the observation vector. env For ambient temperature, P cur P is the gate power; target The target power at the gateway; T in_mean The mean room temperature, T in_std The room temperature variance is denoted as . The ambient temperature is the air temperature 2 meters above the ground where the temperature-controlled load cluster is located. The gate power and gate target power refer to the metering gate data of the feeder substation where the temperature-controlled load is located. The room temperature mean and room temperature variance are obtained by sampling and statistically analyzing the room temperature of users in the temperature-controlled cluster.

[0043] As a preferred embodiment, in the step of designing the action vector and reward function for the temperature-controlled load cluster regulation characteristic analysis, the expression for the action vector is as follows:

[0044] a t :(P mx P mn )

[0045] In the formula, P mx For the schedulable upper boundary, P mn This is the lower limit of the schedulable boundary.

[0046] As a preferred embodiment, in the step of designing the action vector and reward function for the temperature-controlled load cluster regulation characteristic analysis, the reward function includes:

[0047] The boundary violation penalty is expressed as follows:

[0048]

[0049] In the formula, P overlimit This indicates the magnitude of the actual power exceeding the boundary;

[0050] Boundary conflict penalty, expressed as follows:

[0051]

[0052] In the formula, P conflict Indicates the degree of conflict in the output results;

[0053] Adjustable capacity benefit, expressed as follows:

[0054]

[0055] In the formula, P range For the adjustable range of the air conditioning cluster, the larger the range, the higher the benefit, provided that the boundaries do not conflict.

[0056] The tracking deviation penalty is expressed as follows:

[0057]

[0058] In the formula, P err P represents the target tracking error. target For scheduling and control objectives;

[0059] The stability penalty is expressed as follows:

[0060]

[0061] In the formula, P chng This refers to the adjustable range of fluctuation. and These are the adjustable boundaries analyzed 5 minutes ago. This penalty term only takes effect when the boundary conflict for a given action is 0.

[0062] The above reward functions are normalized using the following formula:

[0063]

[0064]

[0065] In the formula, r is the total reward, r i For the reward function components, P base This is a per-unit value, taken as the sum of the rated power of the registered air conditioners.

[0066] As a preferred embodiment, in the step of generating a schedulable range using a policy neural network based on the observation vector using a deep reinforcement learning algorithm, the deep reinforcement learning algorithm employs the SAC algorithm, and the expression for the schedulable range is as follows:

[0067]

[0068] In the formula, For a policy neural network, a t Let S be the action vector. t This is the observation vector.

[0069] As a preferred embodiment, when the decision results interact with the learning environment, the adjustable feature analysis learning environment of the temperature-controlled load cluster randomly generates a target power based on the schedulable range. The temperature-controlled load cluster tracks the target power, completes air conditioning control and room temperature evolution, updates the temperature-controlled load state, and outputs the observations for the next time step. The expression for the interaction between the decision results and the learning environment is as follows:

[0070] S t+1 ~p(S t+1 |S t a t )

[0071] Where p(·) describes the state transition process of the learning environment;

[0072] When gaining action experience, calculate the action reward and save the decision, observation, and reward to the experience pool, as shown in the following expression:

[0073]

[0074] in, The experience pool consists of sample data generated through interaction with the environment for reinforcement learning.

[0075] The expression for the policy update neural network is as follows:

[0076]

[0077]

[0078]

[0079] In the formula, θ i The parameters of the Q-function are the network parameters, and i represents the network index. The SAC algorithm uses two network parameters, which are obtained through iterative training based on the Bellman equation. For the stochastic gradient; λ Q φ is the learning rate; φ is the policy network parameter, which is learned through KL divergence expectation. λ is the gradient of the policy network; π The learning rate; τ represents the target network parameters of the valuation network, obtained through soft updates of the valuation network; τ is the learning rate of the target network.

[0080] Secondly, a temperature-controlled load cluster characteristic analysis system is provided, including:

[0081] The simulation environment construction module is used to build a numerical calculation model of temperature control load based on a pre-established thermodynamic model of air conditioning load, and to design a temperature control load cluster controller to build a temperature control load cluster simulation environment.

[0082] The learning environment construction module is used to select the observation data of the temperature-controlled load cluster to form the observation vector based on the temperature-controlled load cluster simulation environment, design the action vector and reward function for the temperature-controlled load cluster regulation characteristic analysis, and construct the temperature-controlled load cluster adjustable feature analysis learning environment.

[0083] The neural network training module is used to analyze the adjustable features of temperature-controlled load clusters in a learning environment. It employs a deep reinforcement learning algorithm and generates a schedulable range through a policy neural network based on the observation vector. Within the schedulable range, the decision results are interacted with the learning environment to obtain action experience, update the policy neural network, and complete the training of the neural network for analyzing the features of temperature-controlled load clusters.

[0084] The characteristic analysis module is used to perform characteristic analysis of temperature-controlled load clusters using a trained neural network for characteristic analysis of temperature-controlled load clusters.

[0085] As a preferred embodiment, the temperature-controlled load cluster characteristic analysis system further includes an air conditioning load thermodynamic model establishment module, which establishes the air conditioning load thermodynamic model in the following manner:

[0086] Assuming the temperature-controlled load chamber has a uniform temperature field, the heat conduction formula for the temperature-controlled load in the medium is:

[0087]

[0088] In the formula, Q h Where λ is the heat exchange power and λ is the thermal conductivity. Let A be the temperature gradient along the heat transfer direction, and A be the heat transfer area.

[0089]

[0090] In the formula, T in At room temperature, T out The ambient temperature;

[0091] Considering only the heat storage process of air, the formula for calculating the heat storage from t0 to t is:

[0092] ΔQ air =c air V room ρ air (T in (t)-T in (t0))

[0093] In the formula, ΔQ air c represents the change in heat storage in the air. air V is the specific heat capacity of air. room Let ρ be the volume of space. air air density;

[0094] Therefore, the thermodynamic equation for temperature-controlled load is derived as follows:

[0095]

[0096] In the formula, N represents the rate of change of room temperature. m denoted as the number of different media types in the space heat-conducting surface, where m is the sequence number of each different media type.

[0097] The following formula is derived for the room temperature evolution under the conditions of uniform medium and uniform room temperature field under temperature-controlled load:

[0098]

[0099] In the formula, C room =c air V room This parameter describes the heat storage performance of the load;

[0100] This parameter describes the performance of heat exchange between the load and the external environment.

[0101] As a preferred embodiment, the simulation environment construction module uses the improved Euler method to construct a numerical calculation model for the temperature-controlled load. The model construction process includes the following steps:

[0102] An improved Euler method is used to approximate the solution of the first-order differential equation for room temperature variation. The iterative formula for the solution is as follows:

[0103]

[0104] In the formula, T pre T is the predicted room temperature at the next time point calculated based on the rate of change of the initial temperature. cor T is the predicted room temperature at the next time point calculated based on the estimated rate of change of the endpoint temperature. i+1 This is the room temperature for the next time period after correction.

[0105] As a preferred embodiment, the method for designing a temperature-controlled load cluster controller using the simulation environment construction module is as follows:

[0106] Cluster control is performed using a proportional controller, and the feedback loop of the proportional controller is as follows:

[0107] ΔP=k ag (P target -P rt )

[0108] In the formula, ΔP is the polymer power adjustment amount, and k ag This represents the cluster ratio control coefficient, where ag is the cluster number and P is the cluster index. target For the target power, P rt The real-time power of the cluster is measured; target power tracking is performed through the feedback loop of the proportional controller.

[0109] The temperature-controlled load cluster controller transforms the dispatch instructions issued by the dispatch center into temperature regulation signals.

[0110] The power-temperature conversion formula is as follows:

[0111] ΔT=ΔP*k P2T

[0112] In the formula, k P2T This is the power-to-temperature conversion factor;

[0113] The set temperature adjustment value of the cluster is obtained by using linear transformation.

[0114] As a preferred embodiment, when the learning environment construction module selects observation data from the temperature-controlled load cluster to form an observation vector, the expression of the observation vector is as follows:

[0115] S t :(T env Pcur P target T in_mean T in_std )

[0116] In the formula, S t Let T be the observation vector. env For ambient temperature, P cur P is the gate power; target The target power at the gateway; T in_mean The mean room temperature, T in_std The room temperature variance is denoted as . The ambient temperature is the air temperature 2 meters above the ground where the temperature-controlled load cluster is located. The gate power and gate target power refer to the metering gate data of the feeder substation where the temperature-controlled load is located. The room temperature mean and room temperature variance are obtained by sampling and statistically analyzing the room temperature of users in the temperature-controlled cluster.

[0117] As a preferred embodiment, when designing the action vector for analyzing the temperature-controlled load cluster regulation characteristics in the learning environment construction module, the expression of the action vector is as follows:

[0118] a t :(P mx P mn )

[0119] In the formula, P mx For the schedulable upper boundary, P mn This is the lower limit of the schedulable boundary.

[0120] As a preferred embodiment, when the learning environment construction module designs the reward function for analyzing the temperature-controlled load cluster regulation characteristics, the reward function includes:

[0121] The boundary violation penalty is expressed as follows:

[0122]

[0123] In the formula, P overlimit This indicates the magnitude of the actual power exceeding the boundary;

[0124] Boundary conflict penalty, expressed as follows:

[0125]

[0126] In the formula, P conflict Indicates the degree of conflict in the output results;

[0127] Adjustable capacity benefit, expressed as follows:

[0128]

[0129] In the formula, P rangeFor the adjustable range of the air conditioning cluster, the larger the range, the higher the benefit, provided that the boundaries do not conflict.

[0130] The tracking deviation penalty is expressed as follows:

[0131]

[0132] In the formula, P err P represents the target tracking error. target For scheduling and control objectives;

[0133] The stability penalty is expressed as follows:

[0134]

[0135] In the formula, P chng This refers to the adjustable range of fluctuation. and These are the adjustable boundaries analyzed 5 minutes ago. This penalty term only takes effect when the boundary conflict for a given action is 0.

[0136] The above reward functions are normalized using the following formula:

[0137]

[0138]

[0139] In the formula, r is the total reward, r i For the reward function components, P base This is a per-unit value, taken as the sum of the rated power of the registered air conditioners.

[0140] As a preferred embodiment, the neural network training module employs a deep reinforcement learning algorithm. When generating the schedulable range through a policy neural network based on the observation vector, the deep reinforcement learning algorithm uses the SAC algorithm, and the expression for the schedulable range is as follows:

[0141]

[0142] In the formula, For a policy neural network, a t Let S be the action vector. t This is the observation vector.

[0143] As a preferred embodiment, when the neural network training module interacts with the learning environment, the adjustable feature analysis learning environment of the temperature-controlled load cluster randomly generates a target power based on the schedulable range. The temperature-controlled load cluster tracks the target power, completes air conditioning control and room temperature evolution, updates the temperature-controlled load state, and outputs the observations for the next time step. The expression for the interaction between the decision result and the learning environment is as follows:

[0144] S t+1 ~p(S t+1 |S t a t )

[0145] Where p(·) describes the state transition process of the learning environment;

[0146] When gaining action experience, calculate the action reward and save the decision, observation, and reward to the experience pool, as shown in the following expression:

[0147]

[0148] in, The experience pool consists of sample data generated through interaction with the environment for reinforcement learning.

[0149] The expression for the policy update neural network is as follows:

[0150]

[0151]

[0152]

[0153] In the formula, θ i The parameters of the Q-function are the network parameters, and i represents the network index. The SAC algorithm uses two network parameters, which are obtained through iterative training based on the Bellman equation. For the stochastic gradient; λ Q φ is the learning rate; φ is the policy network parameter, which is learned through KL divergence expectation. λ is the gradient of the policy network; π The learning rate; τ represents the target network parameters of the valuation network, obtained through soft updates of the valuation network; τ is the learning rate of the target network.

[0154] Thirdly, an electronic device is provided, comprising:

[0155] Memory, storing at least one instruction; and

[0156] The processor executes the instructions stored in the memory to implement the temperature-controlled load cluster characteristic analysis method.

[0157] Fourthly, a computer-readable storage medium is provided, the computer-readable storage medium storing a computer program, which, when executed by a processor, implements the temperature-controlled load cluster characteristic analysis method.

[0158] Compared with the prior art, the first aspect of the present invention has at least the following beneficial effects:

[0159] The temperature-controlled load cluster characteristic analysis method proposed in this invention is a model- and data-driven intelligent analysis method. Model-driven analysis involves analyzing the operating mechanism of the research object and expressing its input-output relationship using analytical formulas. Data-driven analysis involves training a large amount of input data to represent the input-output mapping relationship using a neural network. Based on the operating mechanism of temperature-controlled loads, this invention can sample intuitive and easily accessible information such as temperature and power, and sample a subset of users to analyze the adjustable characteristics of the group, avoiding high dependence on building parameters and effectively improving analysis accuracy. The method proposed in this invention can accumulate experience from characteristic analysis results and automatically adjust neural network parameters. When user changes occur, the method can determine whether offline learning of recent analysis results is needed based on the quality of the analysis results, thereby updating the neural network to adapt to user changes.

[0160] It is understood that the beneficial effects of the second to fourth aspects mentioned above can be found in the relevant descriptions in the first aspect mentioned above, and will not be repeated here. Attached Figure Description

[0161] To more clearly illustrate the technical solutions in the embodiments of this application, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0162] Figure 1 Flowchart of the method for analyzing the characteristics of temperature-controlled load clusters according to an embodiment of the present invention;

[0163] Figure 2 A schematic diagram of a deep reinforcement learning framework for analyzing the characteristics of temperature-controlled load clusters in an embodiment of the present invention;

[0164] Figure 3 A structural block diagram of the temperature-controlled load cluster characteristic analysis system according to an embodiment of the present invention. Detailed Implementation

[0165] In the following description, specific details such as particular system architectures and techniques are set forth for illustrative purposes and not for limitation, in order to provide a thorough understanding of the embodiments of this application. However, those skilled in the art will understand that this application may also be implemented in other embodiments without these specific details. In other instances, detailed descriptions of well-known systems, apparatuses, circuits, and methods have been omitted so as not to obscure the description of this application with unnecessary detail.

[0166] Furthermore, in the description of this application and the appended claims, the terms "first," "second," "third," etc., are used only to distinguish descriptions and should not be construed as indicating or implying relative importance.

[0167] Electric vehicles, controllable loads, and distributed energy storage represent significant potential controllable resources, providing additional regulation capabilities for grid energy and power balance. Cooling electricity consumption accounts for over 15% of total electricity consumption, and air conditioning load in large cities accounts for approximately 60% of peak summer load, making them crucial future regulation resources. However, these resources are numerous and mostly connected to the public grid, limiting their ability to directly interact with the dispatch center on a large scale due to limitations in communication, security, and data processing capabilities. Therefore, it is necessary to analyze the adjustability characteristics of these massive controllable resource clusters to provide a basis for the grid to utilize these resources.

[0168] The purpose of this invention is to propose a data-driven method for analyzing the adjustable characteristics of temperature-controlled load clusters based on gate power and cluster sampling information. This provides a model-data dual-driven intelligent analysis method for temperature-controlled load cluster characteristics, enabling dispatch centers or load resource aggregators to effectively regulate temperature-controlled load resources.

[0169] The basic approach of this invention is to construct a training environment for temperature-controlled load clusters based on a refined equipment-level thermodynamic model, targeting massive temperature-controlled loads. Secondly, considering scheduling requirements, user comfort, and aggregator revenue, a deep reinforcement learning architecture is designed. Thirdly, the neural network for analyzing the characteristics of the temperature-controlled load clusters is trained, and finally, it is deployed in a real-world operating scenario. Based on sampling data from a small number of air conditioning users, the adjustable characteristics of the temperature-controlled load clusters are determined.

[0170] Example 1

[0171] Please see Figure 1 An embodiment of the present invention provides a method for analyzing the characteristics of a temperature-controlled load cluster, comprising:

[0172] Step 1: Based on the thermodynamic model of air conditioning load, a numerical calculation model of temperature-controlled load is established using the improved Euler method, and a cluster proportional controller is designed to construct a temperature-controlled load cluster simulation environment.

[0173] Step 2: Based on the temperature-controlled load cluster simulation environment, considering actual measurement data, establish observation vectors, consider scheduling requirements, design adjustable characteristic analysis action vectors and reward functions, thereby constructing a temperature-controlled load cluster characteristic analysis learning environment.

[0174] Step 3: Based on the temperature-controlled load cluster characteristic analysis learning environment, a deep reinforcement learning algorithm is used to generate a schedulable range through a policy network, and then interact with the learning environment to obtain action experience and update the neural network, thereby completing the training of the temperature-controlled load cluster characteristic analysis neural network.

[0175] Step 4: Analyze the characteristics of temperature-controlled load clusters using the trained neural network for feature analysis.

[0176] In one possible implementation, the method for constructing the temperature-controlled load cluster simulation environment in step 1 includes:

[0177] Step 1.1: Establish a thermodynamic model of the air conditioning load;

[0178] Constructing an accurate thermodynamic model of the temperature-controlled load is crucial for the learning environment of the temperature-controlled load cluster. This invention assumes that the temperature-controlled load room has a uniform temperature field, and that the walls, frame structure, and windows are the mediums for heat exchange with the outside environment. The heat conduction formula for the temperature-controlled load in the medium is as follows:

[0179]

[0180] Among them, Q h Where λ is the heat exchange power and λ is the thermal conductivity. Let be the temperature gradient along the heat transfer direction, and A be the heat transfer area. Under homogeneous medium conditions, the temperature gradient is constant.

[0181]

[0182] Among them, T in At room temperature, T out The ambient temperature.

[0183] Indoor air and other substances serve as heat storage media, but considering that user comfort is usually only related to air temperature, this embodiment of the invention only considers the heat storage process of air. The formula for calculating the heat storage from t0 to t is as follows:

[0184] ΔQ air =c air V room ρ air (T in (t)-T in (t0)) (3)

[0185] Where, ΔQ air c represents the change in heat storage in the air. air V is the specific heat capacity of air. room Let ρ be the volume of space. air This refers to air density.

[0186] Therefore, the thermodynamic equation for temperature-controlled load can be derived as follows:

[0187]

[0188] in, N represents the rate of change of room temperature. m Let m be the number of different media types in the space heat-conducting surface, and m be the index of each media type. From this, we can derive the following formula for the room temperature evolution under conditions of uniform media and uniform room temperature field under temperature-controlled load:

[0189]

[0190] Among them, C room =c air V room This parameter describes the heat storage performance of the load;

[0191] This parameter describes the performance of heat exchange between the load and the external environment.

[0192] Step 1.2: Construct a numerical calculation model for temperature-controlled load using the improved Euler method;

[0193] Based on the thermodynamic model of temperature-controlled load, an improved Euler method is used in the computer simulation to approximate the solution of the first-order differential equation for room temperature change. The iterative formula for the solution is as follows:

[0194]

[0195] Among them, T pre T is the predicted room temperature at the next time point calculated based on the rate of change of the initial temperature. cor T is the predicted room temperature at the next time point calculated based on the estimated rate of change of the endpoint temperature. i+1 This is the room temperature for the next time period after correction.

[0196] Step 1.3: Design a temperature-controlled load cluster controller;

[0197] The aggregator transforms the scheduling instructions issued by the scheduling center into temperature control signals from the centralized controller through the cluster controller. To simplify the problem and verify the reliability of the proposed external characteristic analysis method, a simple proportional controller is used for cluster control.

[0198] ΔP=k ag (P target -P rt (7)

[0199] The above equation represents the feedback loop of the proportional controller, used for aggregate target power tracking. Where ΔP is the aggregate power adjustment, and k... agThis represents the cluster ratio control coefficient, where ag is the cluster number and P is the cluster index. target For the target power, P rt This represents the real-time power of the cluster.

[0200] ΔT=ΔP*k P2T (8)

[0201] The above formula is the power-temperature conversion formula, k P2T The power-temperature conversion coefficient is used. Through natural response data regression statistics, the power of the temperature-controlled load cluster is linearly related to the ambient temperature. Therefore, the present invention can use linear transformation to obtain the cluster's set temperature adjustment amount.

[0202] In one possible implementation, step 2, constructing an adjustable feature analysis learning environment for temperature-controlled load clusters, includes:

[0203] Step 2.1: Observation and statistics of temperature-controlled load clusters;

[0204] Observations serve as input data for intelligent analysis, requiring comprehensive consideration of the difficulty of actual acquisition and their relevance to the decision-making problem. In this embodiment of the invention, the following observation data from a temperature-controlled load cluster constitutes the observation vector.

[0205] S t :(T env P cur P target T in_mean T in _ std )

[0206] Among them, S t The observation vector includes: ambient temperature (T) env ), gate power (P) cur ), target power at the gateway (P) target ), mean room temperature (T) in_mean ) and room temperature variance (T in_std Because the cluster controls a large number of devices, the mean and variance of room temperature are sampled and statistically analyzed for some devices, which is one of the reasons why the method of this invention has good practicality.

[0207] Among them, the ambient temperature is the air temperature 2 meters above the ground where the temperature-controlled load cluster is located; the gate power and gate target power refer to the metering gate data of the feeder substation where the temperature-controlled load is located; the room temperature mean and room temperature variance are obtained by sampling and statistically analyzing the room temperature of users in the temperature-controlled cluster.

[0208] Step 2.2: Intelligent analysis and action quantity design of temperature-controlled load cluster regulation characteristics;

[0209] Action quantity is the output data of intelligent analysis and is a necessary parameter for the dispatch center to control the temperature control cluster.

[0210] The method of this invention designs the following action vector:

[0211] a t :(P mx P mn )

[0212] Among them, P mx For the schedulable upper boundary, P mn This is the lower limit of the schedulable boundary.

[0213] Step 2.3: Analysis of the regulation characteristics of temperature-controlled load clusters and design of the reward function;

[0214] Reward design is crucial to the performance of intelligent analysis results and requires comprehensive consideration of the goals and constraints of all parties. To reduce the convergence difficulty of learning and training, the method of this invention normalizes the relevant rewards.

[0215] Boundary violation penalty:

[0216]

[0217] Among them, P overlimit This indicates the magnitude of the actual power exceeding the boundary.

[0218] Border conflict penalties:

[0219]

[0220] Among them, P conflict Indicates the degree of conflict in the output results.

[0221] Adjustable capacity benefits:

[0222]

[0223] Among them, P tange For the adjustable range of the air conditioning cluster, the larger the range, the higher the benefit, provided that the boundaries do not conflict.

[0224] Tracking deviation penalty:

[0225]

[0226] Among them, P err P represents the target tracking error. targetThe control objective is set as follows. It should be noted that, to simplify the problem and improve the control reliability of the air conditioning cluster, this embodiment of the invention uses a random sampling method with a uniform distribution within the adjustable range to provide the control objective. Through this penalty, the adjustment range given in this embodiment is relatively conservative, and the cluster exhibits good tracking performance for any target value within the range.

[0227] Stability penalty:

[0228]

[0229] Among them, P chng This refers to the adjustable range of fluctuation. and These are the adjustable boundaries analyzed 5 minutes ago. This penalty term only takes effect when the boundary conflict for a given action is 0.

[0230] A unified normalization method will be used for the above rewards and punishments:

[0231]

[0232]

[0233] In the formula, r is the total reward, r i For the reward function components, P base This is a per-unit value, taken as the sum of the rated power of the registered air conditioners.

[0234] In one possible implementation, step 3, constructing the neural network training architecture for temperature-controlled load cluster feature analysis, includes:

[0235] The temperature-controlled load cluster control problem is a Markov decision problem, suitable for the SAC algorithm. This algorithm demonstrates relatively fast convergence and high generalization performance in the testing environment provided by the gym platform, making it a mainstream reinforcement learning algorithm. Figure 2 As shown, the algorithm uses three neural networks, namely the evaluation networks for the two Q-functions, C1 and C2, respectively. tar (The critic) and its target network and one policy network A (the performer).

[0236] Step 3.1: Based on the observation vector, generate the schedulable range as follows using the policy neural network:

[0237]

[0238] in, For a policy neural network, a t Let S be the action vector. t This is the observation vector.

[0239] Step 3.2: Interaction between decision results and the learning environment;

[0240] The temperature-controlled load cluster characteristic analysis learning environment randomly generates target power based on the schedulable range. The temperature-controlled load cluster tracks the target power, completes air conditioning control and room temperature evolution, updates the temperature-controlled load status, and outputs the observations for the next moment.

[0241] S t+1 ~p(S t+1 |S t a t )

[0242] Here, p(·) describes the state transition process of the learning environment.

[0243] Step 3.3: Calculate the action reward and save the decision, observations, and reward to the experience pool;

[0244]

[0245] in, The experience pool is essentially the sample data that reinforcement learning generates through interaction with the environment.

[0246] Step 3.4: Update the neural network;

[0247]

[0248]

[0249]

[0250] In the formula, θ i The parameters of the Q-function are the network parameters, and i represents the network index. The SAC algorithm uses two network parameters, which are obtained through iterative training based on the Bellman equation. For the stochastic gradient; λ Q φ is the learning rate; φ is the policy network parameter, which is learned through KL divergence expectation. λ is the gradient of the policy network; π The learning rate; τ represents the target network parameters of the valuation network, obtained through soft updates of the valuation network; τ is the learning rate of the target network.

[0251] Please see Figure 3 An embodiment of the present invention provides a temperature-controlled load cluster characteristic analysis system, comprising:

[0252] Simulation environment construction module 2 is used to construct a temperature control load numerical calculation model based on a pre-established air conditioning load thermodynamic model, design a temperature control load cluster controller, and construct a temperature control load cluster simulation environment.

[0253] Learning environment construction module 3 is used to construct an observation vector by selecting the observation data of the temperature-controlled load cluster based on the temperature-controlled load cluster simulation environment, designing the action vector and reward function for the temperature-controlled load cluster regulation characteristic analysis, and constructing a learning environment for the temperature-controlled load cluster adjustable feature analysis.

[0254] The neural network training module 4 is used to analyze the adjustable features of temperature-controlled load clusters in a learning environment. It adopts a deep reinforcement learning algorithm, generates a schedulable range through a policy neural network based on the observation vector, and interacts with the learning environment within the schedulable range to obtain action experience, update the policy neural network, and complete the training of the neural network for analyzing the features of temperature-controlled load clusters.

[0255] Characteristic analysis module 5 is used to perform characteristic analysis of temperature-controlled load clusters using a trained neural network for characteristic analysis of temperature-controlled load clusters.

[0256] In one possible implementation, the temperature-controlled load cluster characteristic analysis system of the present invention further includes an air conditioning load thermodynamic model establishment module 1, which establishes the air conditioning load thermodynamic model in the following manner:

[0257] Assuming the temperature-controlled load chamber has a uniform temperature field, the heat conduction formula for the temperature-controlled load in the medium is:

[0258]

[0259] In the formula, Q h Where λ is the heat exchange power and λ is the thermal conductivity. Let A be the temperature gradient along the heat transfer direction, and A be the heat transfer area.

[0260]

[0261] In the formula, T in At room temperature, T out The ambient temperature;

[0262] Considering only the heat storage process of air, the formula for calculating the heat storage from t0 to t is:

[0263] ΔQ air =c air V room ρ air (T in (t)-T in (t0))

[0264] In the formula, ΔQ air c represents the change in heat storage in the air. air V is the specific heat capacity of air. room Let ρ be the volume of space. airair density;

[0265] Therefore, the thermodynamic equation for temperature-controlled load is derived as follows:

[0266]

[0267] In the formula, N represents the rate of change of room temperature. m denoted as the number of different media types in the space heat-conducting surface, where m is the sequence number of each different media type.

[0268] The following formula is derived for the room temperature evolution under the conditions of uniform medium and uniform room temperature field under temperature-controlled load:

[0269]

[0270] In the formula, C room =c air V room This parameter describes the heat storage performance of the load;

[0271] This parameter describes the performance of heat exchange between the load and the external environment.

[0272] In one possible implementation, simulation environment construction module 2 uses the improved Euler method to construct a numerical calculation model of the temperature-controlled load. The model construction process includes the following steps:

[0273] An improved Euler method is used to approximate the solution of the first-order differential equation for room temperature variation. The iterative formula for the solution is as follows:

[0274]

[0275] In the formula, T pre T is the predicted room temperature at the next time point calculated based on the rate of change of the initial temperature. cor T is the predicted room temperature at the next time point calculated based on the estimated rate of change of the endpoint temperature. i+1 This is the room temperature for the next time period after correction.

[0276] In one possible implementation, the simulation environment construction module 2 designs the temperature-controlled load cluster controller as follows:

[0277] Cluster control is performed using a proportional controller, and the feedback loop of the proportional controller is as follows:

[0278] ΔP=k ag (P target -P rt )

[0279] In the formula, ΔP is the polymer power adjustment amount, and k ag This represents the cluster ratio control coefficient, where ag is the cluster number and P is the cluster index.target For the target power, P rt The real-time power of the cluster is measured; target power tracking is performed through the feedback loop of the proportional controller.

[0280] The temperature-controlled load cluster controller transforms the dispatch instructions issued by the dispatch center into temperature regulation signals.

[0281] The power-temperature conversion formula is as follows:

[0282] ΔT=ΔP*k P2T

[0283] In the formula, k P2T This is the power-temperature conversion factor;

[0284] The set temperature adjustment value of the cluster is obtained by using linear transformation.

[0285] In one possible implementation, when the learning environment construction module 3 selects observation data from the temperature-controlled load cluster to form an observation vector, the expression of the observation vector is as follows:

[0286] S t :(T env P cur P target T in_mean T in_std )

[0287] In the formula, S t Let T be the observation vector. env For ambient temperature, P cur P is the gate power; target The target power at the gateway; T in_mean The mean room temperature, T in_std The room temperature variance is denoted as . The ambient temperature is the air temperature 2 meters above the ground where the temperature-controlled load cluster is located. The gate power and gate target power refer to the metering gate data of the feeder substation where the temperature-controlled load is located. The room temperature mean and room temperature variance are obtained by sampling and statistically analyzing the room temperature of users in the temperature-controlled cluster.

[0288] In one possible implementation, when the learning environment construction module 3 designs the action vector for analyzing the temperature-controlled load cluster regulation characteristics, the expression of the action vector is as follows:

[0289] a t :(P mx P mn )

[0290] In the formula, P mx For the schedulable upper boundary, P mn This is the lower limit of the schedulable boundary.

[0291] In one possible implementation, when the learning environment construction module 3 designs the reward function for analyzing the temperature-controlled load cluster regulation characteristics, the reward function includes:

[0292] The boundary violation penalty is expressed as follows:

[0293]

[0294] In the formula, P overlimit This indicates the magnitude of the actual power exceeding the boundary;

[0295] Boundary conflict penalty, expressed as follows:

[0296]

[0297] In the formula, P conflict Indicates the degree of conflict in the output results;

[0298] Adjustable capacity benefit, expressed as follows:

[0299]

[0300] In the formula, P range For the adjustable range of the air conditioning cluster, the larger the range, the higher the benefit, provided that the boundaries do not conflict.

[0301] The tracking deviation penalty is expressed as follows:

[0302]

[0303] In the formula, P err P represents the target tracking error. target For scheduling and control objectives;

[0304] The stability penalty is expressed as follows:

[0305]

[0306] In the formula, P chng This refers to the adjustable range of fluctuation. and These are the adjustable boundaries analyzed 5 minutes ago. This penalty term only takes effect when the boundary conflict for a given action is 0.

[0307] The above reward functions are normalized using the following formula:

[0308]

[0309]

[0310] In the formula, r is the total reward, r i For the reward function components, Pbase This is a per-unit value, taken as the sum of the rated power of the registered air conditioners.

[0311] In one possible implementation, the neural network training module 4 employs a deep reinforcement learning algorithm. When generating a schedulable range through a policy neural network based on the observation vector, the deep reinforcement learning algorithm uses the SAC algorithm, and the expression for the schedulable range is as follows:

[0312]

[0313] In the formula, For a policy neural network, a t Let S be the action vector. t This is the observation vector.

[0314] In one possible implementation, when the neural network training module 4 interacts with the learning environment using the decision results, the temperature-controlled load cluster adjustable feature analysis learning environment randomly generates a target power based on the schedulable range. The temperature-controlled load cluster tracks the target power, completes air conditioning control and room temperature evolution, updates the temperature-controlled load state, and outputs the observations for the next time step. The expression for the interaction between the decision results and the learning environment is as follows:

[0315] S t+1 ~p(S t+1 |S t a t )

[0316] Where p(·) describes the state transition process of the learning environment;

[0317] When gaining action experience, calculate the action reward and save the decision, observation, and reward to the experience pool, as shown in the following expression:

[0318]

[0319] in, The experience pool consists of sample data generated through interaction with the environment for reinforcement learning.

[0320] The expression for the policy update neural network is as follows:

[0321]

[0322]

[0323]

[0324] In the formula, θ i The parameters of the Q-function are the network parameters, and i represents the network index. The SAC algorithm uses two network parameters, which are obtained through iterative training based on the Bellman equation. For the stochastic gradient; λ Q φ is the learning rate; φ is the policy network parameter, which is learned through KL divergence expectation. λ is the gradient of the policy network; π The learning rate; τ represents the target network parameters of the valuation network, obtained through soft updates of the valuation network; τ is the learning rate of the target network.

[0325] Example 3

[0326] Another embodiment of the present invention also provides an electronic device, comprising:

[0327] Memory, storing at least one instruction; and

[0328] The processor executes the instructions stored in the memory to implement the temperature-controlled load cluster characteristic analysis method.

[0329] Example 4

[0330] Another embodiment of the present invention provides a computer-readable storage medium storing a computer program, which, when executed by a processor, implements the temperature-controlled load cluster characteristic analysis method.

[0331] The computer program includes computer program code, which can be in the form of source code, object code, executable file, or some intermediate form. The computer-readable storage medium can include any entity or device capable of carrying the computer program code, a medium, a USB flash drive, a portable hard drive, a magnetic disk, an optical disk, a computer memory, a read-only memory, a random access memory, an electrical carrier signal, a telecommunication signal, and a software distribution medium, etc. It should be noted that the content included in the computer-readable medium can be appropriately added or removed according to the requirements of legislation and patent practice in the jurisdiction. For example, in some jurisdictions, according to legislation and patent practice, the computer-readable medium does not include electrical carrier signals and telecommunication signals. For ease of explanation, the above content only shows the parts related to the embodiments of the present invention; for specific technical details not disclosed, please refer to the method section of the embodiments of the present invention. This computer-readable storage medium is non-transitory and can be stored in storage devices formed by various electronic devices, enabling the execution process described in the method of the embodiments of the present invention.

[0332] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, systems, or computer program products. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0333] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It should be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0334] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0335] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0336] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and not to limit it. Although the present invention has been described in detail with reference to the above embodiments, those skilled in the art should understand that modifications or equivalent substitutions can still be made to the specific implementation of the present invention. Any modifications or equivalent substitutions that do not depart from the spirit and scope of the present invention should be covered within the scope of protection of the claims of the present invention.

Claims

1. A method for analyzing the characteristics of temperature-controlled load clusters, characterized in that, include: Based on the pre-established thermodynamic model of air conditioning load, a numerical calculation model of temperature control load is constructed, and a temperature control load cluster controller is designed to build a temperature control load cluster simulation environment. Based on the temperature-controlled load cluster simulation environment, the observation data of the temperature-controlled load cluster is selected to form the observation vector, and the action vector and reward function for the temperature-controlled load cluster regulation characteristic analysis are designed to construct the temperature-controlled load cluster adjustable feature analysis learning environment. Based on the adjustable feature analysis learning environment of temperature-controlled load clusters, a deep reinforcement learning algorithm is adopted. According to the observation vector, a schedulable range is generated through a policy neural network. Within the schedulable range, the decision results are interacted with the learning environment to obtain action experience, update the policy neural network, and complete the training of the feature analysis neural network of temperature-controlled load clusters. The characteristics of temperature-controlled load clusters are analyzed using a trained neural network for cluster feature analysis. The air conditioning load thermodynamic model is established as follows: Assuming the temperature-controlled load chamber has a uniform temperature field, the heat conduction formula for the temperature-controlled load in the medium is: In the formula, For heat exchange power, For thermal conductivity, The temperature gradient is along the heat transfer direction. For heat transfer area; In the formula, At room temperature The ambient temperature; Considering only the heat storage process of air, arrive The formula for calculating the heat storage at a given time is: In the formula, This represents the change in heat storage in the air. The specific heat capacity of air, For spatial volume, air density; Therefore, the thermodynamic equation for temperature-controlled load is derived as follows: In the formula, The rate of change of room temperature, The number of types of heat-conducting media in the space. These are serial numbers for different media types; The following formula is derived for the room temperature evolution under the conditions of uniform medium and uniform room temperature field under temperature-controlled load: In the formula, = This parameter describes the heat storage performance of the load; This parameter describes the performance of heat exchange between the load and the external environment; The numerical calculation model for the temperature control load is constructed using the improved Euler method. The model construction process includes the following steps: An improved Euler method is used to approximate the solution of the first-order differential equation for room temperature variation. The iterative formula for the solution is as follows: In the formula, This is the predicted room temperature value at the next time point calculated based on the rate of change of the initial temperature. This is the predicted room temperature at the next time point, calculated based on the estimated rate of change of the endpoint temperature. The corrected room temperature for the next time period; The design method of the temperature-controlled load cluster controller is as follows: Cluster control is performed using a proportional controller, and the feedback loop of the proportional controller is as follows: In the formula, This refers to the polymer power adjustment amount. This is the cluster ratio control coefficient. Cluster number For the target power, The real-time power of the cluster is measured; target power tracking is performed through the feedback loop of the proportional controller. The temperature-controlled load cluster controller transforms the dispatch instructions issued by the dispatch center into temperature regulation signals. The power-temperature conversion formula is as follows: In the formula, This refers to the power-to-temperature conversion factor. The set temperature adjustment value of the cluster is obtained by using linear transformation.

2. The method for analyzing the characteristics of temperature-controlled load clusters according to claim 1, characterized in that, In the step of selecting observation data from the temperature-controlled load cluster to construct the observation vector, the expression of the observation vector is as follows: :( , , , , ) In the formula, For the observation vector, For ambient temperature, Power at the gate; The target power at the gateway; This is the average room temperature. The room temperature variance is denoted as . The ambient temperature is the air temperature 2 meters above the ground where the temperature-controlled load cluster is located. The gate power and gate target power refer to the metering gate data of the feeder substation where the temperature-controlled load is located. The room temperature mean and room temperature variance are obtained by sampling and statistically analyzing the room temperature of users in the temperature-controlled cluster.

3. The method for analyzing the characteristics of temperature-controlled load clusters according to claim 2, characterized in that, In the step of designing the action vector and reward function for the temperature-controlled load cluster regulation characteristic analysis, the action vector expression is as follows: :( , ) In the formula, As the upper limit of schedulability, This is the lower limit of the schedulable boundary.

4. The method for analyzing the characteristics of temperature-controlled load clusters according to claim 3, characterized in that, In the step of designing the action vector and reward function for the temperature-controlled load cluster regulation characteristic analysis, the reward function includes: The boundary violation penalty is expressed as follows: In the formula, This indicates the magnitude of the actual power exceeding the boundary; The boundary conflict penalty is expressed as follows: In the formula, Indicates the degree of conflict in the output results; Adjustable capacity benefit, expressed as follows: In the formula, For the adjustable range of the air conditioning cluster, the larger the range, the higher the benefit, provided that the boundaries do not conflict. The tracking deviation penalty is expressed as follows: In the formula, For target tracking error, For scheduling and control objectives; The stability penalty is expressed as follows: In the formula, This refers to the adjustable range of fluctuation. and These are the adjustable boundaries analyzed 5 minutes ago. This penalty term only takes effect when the boundary conflict for a given action is 0. The above reward functions are normalized using the following formula: In the formula, For the total reward, For the reward function components, This is a per-unit value, taken as the sum of the rated power of the registered air conditioners.

5. The method for analyzing the characteristics of temperature-controlled load clusters according to claim 1, characterized in that, In the step of generating a schedulable range using a policy neural network based on the observation vector using a deep reinforcement learning algorithm, the deep reinforcement learning algorithm employs the SAC algorithm, and the expression for the schedulable range is as follows: In the formula, For policy neural networks, For action vectors, This is the observation vector.

6. The method for analyzing the characteristics of temperature-controlled load clusters according to claim 5, characterized in that, When the decision results interact with the learning environment, the adjustable feature analysis learning environment of the temperature-controlled load cluster randomly generates a target power based on the schedulable range. The temperature-controlled load cluster tracks the target power, completes air conditioning control and room temperature evolution, updates the temperature-controlled load state, and outputs the observations for the next time step. The expression for the interaction between the decision results and the learning environment is as follows: in, This describes the process of state transition in the learning environment; When gaining action experience, calculate the action reward and save the decision, observation, and reward to the experience pool, as shown in the following expression: in, The experience pool consists of sample data generated through interaction with the environment for reinforcement learning. The expression for the policy update neural network is as follows: In the formula, Estimate network parameters for the Q function. This indicates the index of the valuation network. The SAC algorithm uses two valuation networks, which are obtained through iterative training based on the Bellman equation. For stochastic gradients; The learning rate; The policy network parameters are obtained through KL divergence expectation learning; The gradient of the policy network; The learning rate; The target network parameters of the valuation network are obtained through soft updates of the valuation network. The target network learning rate.

7. A system for analyzing the characteristics of temperature-controlled load clusters, characterized in that, include: The simulation environment construction module is used to build a numerical calculation model of temperature control load based on a pre-established thermodynamic model of air conditioning load, and to design a temperature control load cluster controller to build a temperature control load cluster simulation environment. The learning environment construction module is used to select the observation data of the temperature-controlled load cluster to form the observation vector based on the temperature-controlled load cluster simulation environment, design the action vector and reward function for the temperature-controlled load cluster regulation characteristic analysis, and construct the temperature-controlled load cluster adjustable feature analysis learning environment. The neural network training module is used to analyze the adjustable features of temperature-controlled load clusters in a learning environment. It employs a deep reinforcement learning algorithm and generates a schedulable range through a policy neural network based on the observation vector. Within the schedulable range, the decision results are interacted with the learning environment to obtain action experience, update the policy neural network, and complete the training of the neural network for analyzing the features of temperature-controlled load clusters. The characteristic analysis module is used to perform characteristic analysis of temperature-controlled load clusters using a trained neural network for characteristic analysis of temperature-controlled load clusters. It also includes an air conditioning load thermodynamic model establishment module, which establishes the air conditioning load thermodynamic model in the following manner: Assuming the temperature-controlled load chamber has a uniform temperature field, the heat conduction formula for the temperature-controlled load in the medium is: In the formula, For heat exchange power, For thermal conductivity, The temperature gradient is along the heat transfer direction. For heat transfer area; In the formula, At room temperature The ambient temperature; Considering only the heat storage process of air, arrive The formula for calculating the heat storage at a given time is: In the formula, This represents the change in heat storage in the air. The specific heat capacity of air, For spatial volume, air density; Therefore, the thermodynamic equation for temperature-controlled load is derived as follows: In the formula, The rate of change of room temperature, The number of types of heat-conducting media in the space. These are serial numbers for different media types; The following formula is derived for the room temperature evolution under the conditions of uniform medium and uniform room temperature field under temperature-controlled load: In the formula, = This parameter describes the heat storage performance of the load; This parameter describes the performance of heat exchange between the load and the external environment; The simulation environment construction module uses the improved Euler method to construct a numerical calculation model for temperature-controlled loads. The model construction process includes the following steps: An improved Euler method is used to approximate the solution of the first-order differential equation for room temperature variation. The iterative formula for the solution is as follows: In the formula, This is the predicted room temperature value at the next time point calculated based on the rate of change of the initial temperature. This is the predicted room temperature at the next time point, calculated based on the estimated rate of change of the endpoint temperature. The corrected room temperature for the next time period; The method for designing a temperature-controlled load cluster controller using the simulation environment construction module is as follows: Cluster control is performed using a proportional controller, and the feedback loop of the proportional controller is as follows: In the formula, This refers to the polymer power adjustment amount. This is the cluster ratio control coefficient. Cluster number For the target power, The real-time power of the cluster is measured; target power tracking is performed through the feedback loop of the proportional controller. The temperature-controlled load cluster controller transforms the dispatch instructions issued by the dispatch center into temperature regulation signals. The power-temperature conversion formula is as follows: In the formula, This refers to the power-to-temperature conversion factor. The set temperature adjustment value of the cluster is obtained by using linear transformation.

8. The temperature-controlled load cluster characteristic analysis system according to claim 7, characterized in that, When the learning environment construction module selects observation data from the temperature-controlled load cluster to form an observation vector, the expression of the observation vector is as follows: :( , , , , ) In the formula, For the observation vector, For ambient temperature, Power at the gate; The target power at the gateway; This is the average room temperature. The room temperature variance is denoted as . The ambient temperature is the air temperature 2 meters above the ground where the temperature-controlled load cluster is located. The gate power and gate target power refer to the metering gate data of the feeder substation where the temperature-controlled load is located. The room temperature mean and room temperature variance are obtained by sampling and statistically analyzing the room temperature of users in the temperature-controlled cluster.

9. The temperature-controlled load cluster characteristic analysis system according to claim 8, characterized in that, When designing the action vector for analyzing the temperature control load cluster regulation characteristics in the learning environment construction module, the expression for the action vector is as follows: :( , ) In the formula, As the upper limit of schedulability, This is the lower limit of the schedulable boundary.

10. The temperature-controlled load cluster characteristic analysis system according to claim 9, characterized in that, When designing the reward function for the temperature-controlled load cluster regulation characteristic analysis in the learning environment construction module, the reward function includes: The boundary violation penalty is expressed as follows: In the formula, This indicates the magnitude of the actual power exceeding the boundary; The boundary conflict penalty is expressed as follows: In the formula, Indicates the degree of conflict in the output results; Adjustable capacity benefit, expressed as follows: In the formula, For the adjustable range of the air conditioning cluster, the larger the range, the higher the benefit, provided that the boundaries do not conflict. The tracking deviation penalty is expressed as follows: In the formula, For target tracking error, For scheduling and control objectives; The stability penalty is expressed as follows: In the formula, This refers to the adjustable range of fluctuation. and These are the adjustable boundaries analyzed 5 minutes ago. This penalty term only takes effect when the boundary conflict for a given action is 0. The above reward functions are normalized using the following formula: In the formula, For the total reward, For the reward function components, This is a per-unit value, taken as the sum of the rated power of the registered air conditioners.

11. The temperature-controlled load cluster characteristic analysis system according to claim 7, characterized in that, The neural network training module employs a deep reinforcement learning algorithm. When generating a schedulable range based on the observation vector through a policy neural network, the deep reinforcement learning algorithm uses the SAC algorithm. The expression for the schedulable range is as follows: In the formula, For policy neural networks, For action vectors, This is the observation vector.

12. The temperature-controlled load cluster characteristic analysis system according to claim 11, characterized in that, When the neural network training module interacts with the learning environment, the adjustable feature analysis learning environment of the temperature-controlled load cluster randomly generates a target power based on the schedulable range. The temperature-controlled load cluster tracks the target power, completes air conditioning control and room temperature evolution, updates the temperature-controlled load state, and outputs the observations for the next time step. The expression for the interaction between the decision result and the learning environment is as follows: in, This describes the process of state transition in the learning environment; When gaining action experience, calculate the action reward and save the decision, observation, and reward to the experience pool, as shown in the following expression: in, The experience pool consists of sample data generated through interaction with the environment for reinforcement learning. The expression for the policy update neural network is as follows: In the formula, Estimate network parameters for the Q function. This indicates the index of the valuation network. The SAC algorithm uses two valuation networks, which are obtained through iterative training based on the Bellman equation. For stochastic gradients; The learning rate; The policy network parameters are obtained through KL divergence expectation learning; The gradient of the policy network; The learning rate; The target network parameters of the valuation network are obtained through soft updates of the valuation network. The target network learning rate.

13. An electronic device, characterized in that, include: Memory, storing at least one instruction; and The processor executes the instructions stored in the memory to implement the temperature-controlled load cluster characteristic analysis method as described in any one of claims 1 to 6.

14. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by the processor, it implements the temperature-controlled load cluster characteristic analysis method as described in any one of claims 1 to 6.

Images