Air conditioning system self-adaptive regulation and control method based on information matching

Abstract

The invention discloses a self-adaptive regulation and control method for an air conditioning system based on information matching. The self-adaptive regulation and control method mainly comprises the following steps that firstly, a data set used for building cold load prediction is divided into a low-completeness data set and a high-completeness data set through a proposed information completeness judgment method; and for the high-completeness data set, regulating and controlling the air conditioning system by adopting a model prediction control method according to the load prediction model and the air conditioning equipment model. And for a low-completeness data set, a proposed migration imitation learning regulation and control method based on deep reinforcement learning is adopted. According to the method, the influence of data quality on the model prediction precision and the regulation and control effect is considered, the influence of fluctuation of historical data quality on the regulation and control effect can be overcome, the relative advantages of the model prediction control method and the reinforcement learning method are integrated, compared with a single regulation and control method, the energy-saving and cost-saving effects can be improved, and good robustness is achieved.

Description

Technical Field

[0001] The present invention belongs to the cross - field of building energy management and artificial intelligence, and particularly relates to a regulation method for integrating and switching model predictive control and reinforcement learning control of an air - conditioning system based on the matching of load prediction information. Background Art

[0002] Model predictive control and reinforcement learning control, as two advanced regulation methods, have been widely studied and applied in the energy - saving and demand - response management of air - conditioning systems. Among them, model predictive control obtains the current state of the system through sampling means, and combines the established model to complete the prediction of the system state in the future time window. Based on the predicted values and the set boundary conditions, an optimal control strategy is obtained using an optimization algorithm. Different from the model - based regulation method, reinforcement learning, as a model - free algorithm, directly interacts with the operating environment to explore and evaluates the control strategy, automatically explores the optimal control actions, avoids the work of model establishment, and can adapt to system changes for learning and adjustment.

[0003] Compared with reinforcement learning, model predictive control can use the model and optimization algorithm for efficient optimization, avoiding the disadvantages of unstable training, difficult convergence, and low optimization efficiency of reinforcement learning. However, model predictive control depends on a high - precision prediction model, and the establishment of the model depends on historical data. Due to the fluctuation of the quality of historical data, the accuracy of the prediction model is affected, which in turn affects the effect of model predictive control.

[0004] Facing complex historical data, a single regulation method is difficult to cope with the change of data quality. At present, there is a lack of a method for judging the quality of historical data sets, and there is also a lack of a method for flexibly deploying control means according to different data conditions and operating conditions to integrate the advantages of the two regulation methods.

[0005] To solve the above problems, the present invention proposes an adaptive regulation method for an air - conditioning system based on information matching. The proposed information - matching method judges the reliability of the load prediction model one day in advance according to data completeness, divides the data used for prediction into high - completeness and low - completeness data, and simultaneously constructs a model predictive control method and a deep reinforcement learning regulation method for the air - conditioning system. When the prediction data is of high completeness, that is, the load prediction accuracy is high, the model predictive control method is adopted; otherwise, the deep reinforcement learning regulation method is adopted. Summary of the Invention

[0007] The technical solution of the present invention includes the following steps:

[0008] S1 Information Completeness Judgment: For the building load prediction, the present invention first establishes a training set - prediction set data completeness judgment index for external disturbances, internal disturbances, and historical similar day factors. The index includes quantifying the difference in the external disturbance distribution between the prediction day and the training set through the JS divergence; reconstructing the load curve through wavelet decomposition to obtain the high-frequency signal reflecting the internal disturbance in the training set data, and calculating the coefficient of variation value to reflect the signal fluctuation and the uncertainty of the internal disturbance; calculating the cosine distance of the normalized feature vectors of the external and internal disturbances between the prediction day and the historical day. The historical day with the minimum cosine distance is the most similar to the prediction day, and this value is used as the historical similar day difference index. The three types of indexes are comprehensively combined to form an information completeness index vector, and the data set is clustered according to the information completeness index by spectral clustering, and the load prediction data set is divided into low and high completeness.

[0009] S2 Construction of Model Predictive Control Method: 1) Based on the high-completeness data set, establish a data-driven load prediction model to predict the building cooling load within a time window of length T. 2) According to the prediction results and the constructed air-conditioning system model, optimize the control variables for the next T time intervals to minimize the objective function and satisfy the constraint conditions. 3) Use the optimized results of the control variables in the first time window as the scheduling control signal and apply it to the air-conditioning system. 4) Update the operating data according to the actual air-conditioning system, and repeat steps 1) to 4) to complete the rolling prediction and optimization. S3 Construction of Deep Reinforcement Learning Control Method: The present invention proposes a transfer imitation learning control method based on deep reinforcement learning: 1) First, construct the state space, action space, and reward function of the Markov decision process according to the air-conditioning system, and construct the algorithm framework and training process of the deep Q network. 2) Offline train the value function Q neural network in the pre-constructed source domain simulation building air-conditioning system. At the same time, introduce imitation learning during the training process to increase the training and parameter adjustment efficiency of the reinforcement learning model: introduce the expert strategy into the gradient calculation of the agent network to provide reference for the agent's actions under specific working conditions. 3) Conduct the domain similarity test between the target building air-conditioning system and the source domain building air-conditioning system. According to the test results, select the offline trained deep reinforcement learning network of the source domain most similar to the target domain and transfer it to the target domain. When transferring, freeze the parameters of some neural network hidden layers, and further online train the remaining neural network parameters in the target domain air-conditioning system and provide control actions.

[0010] S4 Adaptive Regulation Method Switching: One day before the controlled day, combined with historical data, pre-judge the reliability of the load prediction on the current day according to the data completeness judgment method described in S1 and select the regulation method. For the high-completeness prediction data, select the model predictive control method established in S2; for the low-completeness prediction data, select the transfer imitation learning control method established in S3 to achieve the adaptive recognition of different working conditions and the flexible switching of the regulation methods.

[0011] The information completeness judgment index of the load forecasting described in S1 includes external disturbances, internal disturbances, and historical similar day difference indexes.

[0012] External disturbance difference index: Select the Kullback-Leibler divergence to quantify the difference between distributions:

[0013]

[0014] P(x i ) and Q(x i ) are the probability densities of distributions P and Q respectively. To make the index symmetric and limit the value range between 0 and 1. Further introduce the Jensen-Shannon divergence:

[0015] D JS (P|Q) = 0.5·D KL (P|M) + 0.5·D KL (Q|M) (2)

[0016] The distribution M is as follows:

[0017]

[0018] Select the external disturbance variable distribution P i , P i as the Gaussian distribution with the weather forecast of a certain external disturbance (such as room temperature, solar radiation) on the forecast day as the mean as the probability density; the external disturbance variable distribution Q i , Q i The probability density of a certain external disturbance distribution is obtained according to historical data statistics, then the external disturbance matching index JS i of the i-th forecast day is calculated as follows:

[0019]

[0020] where j represents different external disturbance variables and m is the number of external disturbance variables.

[0021] Internal disturbance difference index: Due to internal disturbance influencing factors such as the occupancy rate of people in the room being difficult to directly measure, perform wavelet transform on the time-domain load curve to obtain low-frequency signals related to periodic loads and high-frequency signals related to sudden loads, and the high-frequency signals are related to indoor human activities. First, perform wavelet decomposition on the training set load curve and reconstruct to obtain the high-frequency signal curve. The wavelet transform formula is as follows:

[0022]

[0023] where WT f(a, τ) is the result of wavelet transform, and ψ(t) is called the wavelet basis function; f(t) is the load curve in the time domain, a is the scale variable that controls the dilation and contraction of the wavelet basis function and reflects the magnitude of frequency; τ is the translation variable that controls the movement of the wavelet basis function in the time scale. By translating, stretching, and compressing the wavelet basis function, signal components with different frequencies are obtained. The original signal is decomposed into a low-frequency signal a1 and a first-order high-frequency signal d1. Then, a1 is regarded as a new signal for further decomposition until a set of high-frequency signals {d1, d2,..., d n} and a low-frequency signal a that reflects the periodic law are formed. n When performing wavelet decomposition on the building load signal, the load fluctuation trend signal a1 within a day and the load fluctuation signal d1 caused by internal disturbances are obtained.

[0024] For the obtained high-frequency signal d1, its volatility is related to the change in the occupancy rate of people. The coefficient of variation index is selected to reflect the fluctuation degree of the training set d1 sequence. The coefficient of variation CV is calculated as follows:

[0025]

[0026] where σ is the variance of the training set sequence, is the average value of the training set sequence. The value of CV ranges from 0 to 1. The smaller the CV value, the smaller the standard deviation of the data set relative to the mean, the smaller the volatility of the sequence, and the smaller the uncertainty of the internal disturbance.

[0027] Historical similar-day difference index: The change law of building load can be reflected by the load on historical similar days, and the load on historical similar days is used as an input variable for the data-driven model. To ensure that the load on similar days and the predicted day has a similar change law, the cosine similarity cos(i, j) is established to quantify the similarity of the operating conditions of the two. The calculation formula for the cosine similarity between the i-th day and the j-th day is as follows:

[0028]

[0029] where the feature vector X i = {x i，1 , x i，2 ,..., x i，m} is composed of normalized data of features such as meteorology, date, and occupancy rate of people. The value range of cos(i, j) is between -1 and 1. When the value of cos(i, j) is close to 1, it indicates that the two vectors are similar. When the value of cos(i, j) is close to 0, it indicates that the two vectors are perpendicular and there is no similarity. When the value of cos(i, j) is close to -1, it means that the two vectors are negatively correlated. Introducing the cosine distance can ensure that the similarity and the metric index have the same change trend. The calculation formula for the cosine distance between the i-th day and the j-th day is shown in the formula.

[0030] H ij = 1 - cos(i, j) (8)

[0031] Select historical similar days within the range of k days before the i-th prediction day to ensure the minimum cosine distance between the historical similar days and the prediction day, that is:

[0032] H i = min{H i，1 , H i，2 ,..., H i，j ..., H i，k} (9)

[0033] Among them, H i，j is the cosine distance between the j-th day before the i-th prediction day and the prediction day. Select H i as the historical similar day matching index, which can judge whether the data-driven model can induce the correct load change law from historical similar days.

[0034] Combined with three types of indicators, the information completeness index c i = (JS i , CV i , H i ). Select an unsupervised learning method to judge the information matching, and obtain the matching distribution law from the distribution of c i . Select the spectral clustering algorithm as the judgment method. The clustering results divide the data into low-completeness data and high-completeness data.

[0035] For the model predictive control method described in S2, in addition to the established load prediction model, an air-conditioning system equipment model should also be constructed according to the actual building system, and an optimization problem is constructed as follows:

[0036]

[0037] In the formula, f(X) represents the optimization objective function of the air-conditioning system, which needs to be set according to the actual building air-conditioning system, and should take into account comfort as well as minimizing operating costs and energy consumption. g j (X) is the inequality constraint in the optimization, for example, the charging and discharging constraints of the energy storage system in the air-conditioning system, the indoor personnel comfort constraint, etc.; h k (X) is the equality constraint in the optimization, for example, the energy supply and demand balance in the air-conditioning system optimization process, etc. The predicted value of the cooling load in the energy constraint comes from the load prediction model. In the specific implementation of the present invention, the specific expression is determined according to the example.

[0038] The construction process of the reinforcement learning Markov decision process described in S3 should select the state space, action space, and reward function according to the actual building air conditioning system. In the specific implementation of the present invention, specific expressions and agent construction methods are determined based on examples. At the same time, a deep Q-network algorithm is constructed to train the agent, and the framework of the deep Q-network algorithm can refer to existing literature.

[0039] In the training process of the present invention, imitation learning is introduced to improve the problem of sparse rewards in the reinforcement learning running environment. A loss function related to the expert action is introduced into the gradient calculation of the agent network. The loss function of the batch training samples is as follows:

[0040]

[0041] In the formula, f() describes the gap between the agent policy and the expert policy, and N D is the number of samples taken from the experience pool, and π(s i |θ π ) is the agent policy of the i-th sample, and a i is the expert policy of the i-th sample.

[0042] The update process of the agent network parameter θ π is as follows;

[0043]

[0044] In the formula, α is the learning rate, λ1 and λ2 are weight coefficients, and J is the deep reinforcement learning loss function. L lfd is the loss function of imitation learning. Therefore, in the above formula, in addition to the reinforcement learning loss function being optimized, the loss function of imitation learning is also optimized, that is, the difference between the agent and the expert control actions is reduced, which enables the expert policy to be added to the update of the agent network. Due to the existence of the expert policy, the update gradient of the agent network parameter θ π will not decrease due to the influence of the sparse reward.

[0045] The significance and process of the domain similarity test described in S3 are as follows: If there are differences in the characteristics of the source domain and the target domain, the mismatched knowledge will be transferred to the target domain, which will have an adverse impact on the construction of the target domain model. This is called negative transfer. Therefore, before implementing transfer learning, it is necessary to judge the similarity between the source domain and the target domain to avoid the occurrence of negative transfer. The comprehensive effect of factors such as meteorological conditions, personnel usage behavior patterns, and the heat transfer and heat storage performance of building envelopes can be reflected by the typical patterns of building cooling and heating load curves. Therefore, first, the typical load patterns of the source domain and the target domain are identified through a clustering algorithm. Further, the differences between the typical load patterns of the source domain and the target domain are quantified to complete the similarity judgment between the source domain and the target domain. Finally, the source domain building closest to the characteristics of the target domain is selected according to the calculation results of the difference quantification index.

[0046] Establishment of shape difference index: After obtaining the typical load curves of the source domain and the target domain through clustering, the similarity is judged by the shape difference index. The index establishment process is as follows:

[0047] To reflect the magnitude, correlation degree, and phase relationship of two time series at different frequencies, according to the spectral analysis theory, it can be represented by the cross-spectral density function. For two load time series, X(t)Y(t)

[0048] The cross-spectral density h XY (f) is defined as:

[0049] R XY (τ) = E[(X(t) - μ X )·(Y(t + τ) - μ Y )]

[0050]

[0051] where t is time, μ is the mean; R XY (τ) is the cross-covariance function, τ is the time delay; f is the frequency.

[0052] In the calculation, h XY (f) is a complex function. For the convenience of further calculation, it can be transformed into a new form according to the polar coordinate representation method, as shown in Equations (14) to (16).

[0053] h XY (f) = a XY (f) exp{2πb XY (f)} (14)

[0054]

[0055] b XY (f) = arctan[-q XY (f) / c XY (f)] (16)

[0056] where c XY (f) is the real part of h XY (f), q XY (f) is the imaginary part of h XY (f); a XY (f) is the cross-amplitude spectrum; b XY (f) is the phase spectrum.

[0057] The magnitude of the coherence spectrum is determined by the cross-amplitude spectrum and the phase spectrum. When the phase change is not considered, the coherence spectrum reflects the relationship between two time series in the frequency domain. Generally, the calculation results of the coherence spectrum need to be standardized, and the calculation method is as follows:

[0058]

[0059] where h X (f) and h Y (f) are the spectral density functions of two sequences respectively; w XY (f) is the coherence spectrum, which is the normalized mean of the product of the amplitudes of the components at frequency f in the two sequences, reflecting the degree of linear correlation between the two sequences at different frequencies. The closer the coherence spectrum value is to 1, the greater the correlation between the two sequences at that frequency.

[0060] Calculation of difference measurement index

[0061] By normalizing the coherence spectrum value and the phase spectrum value and calculating the difference, the similarity between the two curves can be judged. The calculation method is as follows:

[0062] Δ = norm(w i ) - norm(b i ) (18)

[0063] where Δ is the normalized difference of cross-spectrum analysis, Δ ∈ [0, 1]; norm(w i ) is the normalized coherence spectrum value of the i-th typical curve in the target domain relative to the source domain curve, w i ∈ [0, 1]; b i is the normalized phase spectrum value of the i-th typical curve in the target domain relative to the source domain curve, b i ∈ [0, 1]. The greater and closer the normalized difference Δ is to 1, the higher the similarity degree of the two curves.

[0064] When performing domain similarity test, it is necessary to first remove the outliers and fill in the missing values in the measured cooling load data set to avoid the influence of abnormal and missing values on the test results. Using the CalinskiHarabasz method, calculate the number of typical load patterns in the target domain and each source domain, and use the K-means clustering method to obtain the set of typical load curves in the target domain and the set of typical load curves in each source domain

[0065] where and are the i-th typical load curve in the target domain and the j-th typical load curve in the p-th source domain. n t and n s,p are the numbers of typical load patterns in the target domain and the p-th source domain respectively.

[0066] To avoid the influence of load magnitude on the calculation of load shape similarity, the typical load curve set is normalized. According to the normalized difference index of cross-spectrum analysis, the load pattern most similar to the target domain can be selected from each source domain. For the i-th typical pattern in the target domain, the similar load pattern selected from the p-th source domain satisfies the condition in Equation (19):

[0067]

[0068]

[0069] Equation (19) indicates that the i-th typical curve in the target domain and the m-th i typical load curve in the p-th source domain have the largest normalized difference in cross-spectrum analysis, that is, the similarity between the two typical load patterns is the largest, and the similarity index D i,p is obtained at this time, where:[[]]

[0070]

[0071] According to Equations (19) and (20), the similarity index of all typical curves in the target domain with respect to the p-th source domain is obtained, and the weighted sum is calculated to obtain the similarity index D p of the target domain with respect to the p-th source domain, as shown in the formula:[[]]

[0072]

[0073] where ω i is the weight of the i-th typical pattern, and the weight value is the proportion of the number of load curves under the i-th typical pattern.[[]]

[0074] Select the q-th source domain in the source domain as the source domain to be migrated. The q-th source domain satisfies the formula:[[]]

[0075] q = argmax D p

[0076] s.t q ∈ [1, P] (22)[[]]

[0077] The q-th source domain is the source domain with the largest similarity index among the P source domains, indicating that the building nature of this source domain is the most similar to the target domain. The selection formula is as shown in Equation (22). Transferring the knowledge of this source domain to the target domain can improve the performance of the reinforcement learning model.[[]]

[0078] The transfer process selects transfer reinforcement learning based on model parameters as the transfer model. The deep Q-network algorithm uses a neural network to estimate the value function Q(s, a). In transfer learning based on model parameters, the neural network parameters will be trained in the source domain. Then, some of the hidden layer parameters are frozen, and the remaining neural network parameters are fine-tuned in the target domain. In the present invention, the deep Q-network has four layers, and the source domain network parameters are w S =[w S1 , w S2 , w s3 , w S4 and b S =[b S1 , b S2 , b S3 , b S4 . During transfer, the number of layers N of the underlying feature extractor is taken as 2, the first and second layer networks are frozen, and the third and fourth layer network parameters are trained in the target domain. Then the target domain network parameters are w T =[w S1 , w S2 , w T3 , w T4 and b T =[b S1 , b S2 , b T3 , b T4 . The transfer relationship of the target domain network can be as shown in the formula:

[0079]

[0080] where X represents the input state, Y represents the output of each layer, g(wX + b) represents the activation function, and Q represents the output Q value. After the model parameter transfer, the first two layers of the neural network retain the knowledge obtained from the source domain training, reduce the number of parameters to be trained, and reduce the time and difficulty of online training of the DQN method. At the same time, the fine-tuning training of the last two layer parameters can enable the Q network to learn the knowledge of the target domain. BRIEF DESCRIPTION OF THE DRAWINGS

[0081] Figure 1 is a flowchart of the adaptive regulation method provided by the present invention

[0082] Figure 2 is a flowchart of a method for judging the completeness of load prediction information provided by the present invention

[0083] Figure 3 is a schematic diagram of the regulation method based on model prediction adopted by the present invention

[0084] Figure 4 is a schematic diagram of the process of a transfer imitation learning algorithm provided by the present invention DETAILED DESCRIPTION OF THE INVENTION

[0085] To make the technical solutions and advantages of the present invention clearer and more understandable, the following further elaborates on the present invention in combination with specific embodiments. The example is an office building with 3 floors above ground, and the main usage time of the building occupants is from 8:00 to 20:00. The terminal of the building air conditioning system adopts the variable air volume system form (VAV). On the energy system side, two screw air-cooled chiller units are used as the cold source, and each unit is equipped with a water pump. Usually, one unit is turned on and the other is in standby, and the number of water pumps turned on corresponds to the units. A thermal energy storage tank is provided to store the chilled water produced during the low electricity price period at night and release the cooling capacity during the high electricity price period to complete the load transfer.

[0086] The load prediction model described in S1 adopts the Extreme Gradient Boosting (XGBoost) algorithm. XGBoost is a machine learning algorithm improved based on the Gradient Boosting Decision Tree (GBDT) algorithm and is widely used in the prediction field. The specific principle and construction process of XGBoost can refer to existing literature. The model inputs include external and internal disturbance parameters such as outdoor dry-bulb temperature, indoor dry-bulb temperature, solar radiation intensity, indoor socket electricity consumption, and historical similar-day load. The coefficient of variation of root mean square error CV-RMSE index is used to evaluate the prediction effect. The data of the previous 14 days before the operation date of the case building is selected as the training set to train the prediction model, and the current date data is used to predict the cooling load. In daily prediction, with a time step of 30 minutes, the building cooling load at the next moment is predicted.

[0087] The air conditioning equipment model described in S2 constructs a grey box model according to existing literature. The model of the air-cooled chiller unit selects the DOE-2 model, and this model uses the nominal operating parameters and performance curves of the equipment to determine the operating performance of the equipment under various conditions. In the model, the nominal operating parameters include nominal cooling capacity and nominal electric power. The performance curves include the CapFTemp function, the EIRTemp function, and the EIRFPLR function, and the least squares method is used to fit the performance curves according to historical operation data.

[0088] The thermal energy storage tank model is an energy conservation model. The energy at the t + 1 moment is the energy of the water tank at the t moment plus the energy stored or minus the energy taken out at the t moment, and minus the energy loss of heat exchange with the surrounding environment. Its calculation formulas are shown in Formulas (24) and (25).

[0089] R(t + 1) = R(t) + Δt·Q TES (t) + φ·(T amD -T TEs )

[0090] Q TEs (t) ≥ 0 (24)

[0091] R(t + 1) = R(t) + Δt·Q TEs (t) + φ·(T amb -T TES )

[0092] Q TEs (t) ≤ 0 (25)

[0093] Among them, the above formula represents the energy storage process of the water tank at night, and the following formula represents the energy release process of the water tank during the day. R(t + 1) and R(t) are the heat storage amounts of the water tank at time t + 1 and time t, respectively, in kJ; Q TES (t) is the energy storage or release power of the water tank at time t, in kW; Δt is the energy storage and release control interval, in h; φ is the heat loss coefficient, set to 5 W / K; T amb and T TES are the ambient temperature and the average temperature of the water tank, respectively, in °C

[0094] The optimization problem described in S2 is constructed as follows: The air conditioning system operates in two time periods: from 6:00 to 21:00 during the day and from 22:00 to 6:00 the next day at night. In the daytime mode, the indoor air conditioning terminal is turned on, and the temperature set point is optimized within the time interval. The chiller is turned on to supply cooling to the indoor alone or in combination with the energy storage water tank; in the night mode, the indoor air conditioning terminal is turned off, and the indoor temperature follows the natural change. The chiller stores cold in the water tank, and the cold storage process is not optimized. When the average temperature in the water tank reaches 6 °C during cold storage, the energy storage process stops. At the end of the night mode, the initial conditions for daytime optimization can be determined, as shown in the following formula

[0095]

[0096] in，0 and T initial are the indoor temperature and the natural room temperature at the start of each day, in °C; T TES,0 is the water tank temperature at the initial moment, in °C

[0097] The scheduling control of the building air conditioning system includes the control of the indoor operating parameters of the building and the control of the operating parameters of the air conditioning system equipment during the daytime operation. Among them, the control of the indoor operating parameters includes the reset of the indoor temperature set point T in，set ; the control of the air conditioning equipment operation includes the optimization of the energy release amount Q TES，t of the energy storage water tank and the optimization of the outlet water temperature set point T chw，S of the chiller. The optimization objectives are the following three formulas

[0098]

[0099]

[0100] ​

[0101] Equation (27) represents the optimization objective for indoor comfort, equation (28) is the optimization objective for operating cost, and equation (29) is the optimization objective for operating energy consumption. Among them, ω comfort，t is the temperature calculation coefficient, which is related to the working hours of the personnel in the building and is calculated as follows; The function related to indoor comfort is calculated as follows.

[0102]

[0103]

[0104] and are the upper and lower temperature limits that do not cause discomfort, and are set to 24.1 °C and 27.4 °C respectively according to the thermal comfort requirements of ASHRAE Standard 55-2017. T in is the indoor temperature, in °C.

[0105] λ t and λ night are the time-of-use electricity price during the day and the low electricity price at night respectively, in yuan / kWh; Q TES，t is the power generation capacity during the day at time t, in kW; ω hp，t is the energy consumption calculation coefficient, which is related to the start-up time of the unit during the day and is calculated as follows; P hp，t is the energy consumption of the unit at time t, which is calculated through the aforementioned DOE-2 unit model.

[0106]

[0107] During the optimization process, to ensure the safety of operation and the rationality of the strategy, the following indoor temperature constraints are set:

[0108]

[0109] In the formula, and are the upper and lower limits of the indoor temperature respectively. According to the thermal comfort requirements in GB / T 33658-2017, the upper and lower limits of the indoor temperature are set to 28.9 °C and 22.5 °C respectively

[0110] The following energy-related constraints are set:

[0111] Q TES，t +Q hp，t =Q CL，t (34)

[0112] Q TES，t ≥0 (35)

[0113] ∑Q TES,t·Δt ≤ R mmax (36)

[0114] Q TEs，t ·Δt ≤ R remian，t (37)

[0115] 0.3·Q avi ≤ Q hp，t ≤ Q avi (38)

[0116] Equation (34) represents the balance between cooling supply and demand, that is, the chiller and the energy storage tank should meet the predicted indoor cooling demand. In the equation, Q hp，t is the cooling capacity of the chiller at time t, in kW, and Q CL，t is the predicted cooling load at time t, in kW; Equations (35) to (38) represent the constraints on the energy of the energy storage tank. Among them, R max represents the maximum capacity of the water tank, in kwh, and R remian,t represents the remaining energy of the water tank at time t, in kWh; Equation (35) means that the energy release power during the day is not less than zero; Equation (36) means that the total energy release during the day needs to be less than the stored energy; Equation (37) means that the energy release at time t needs to be no greater than the remaining energy of the water tank; Equation (38) represents the limitation of the chiller's cooling capacity. To ensure the efficiency and normal operation of the chiller, if the chiller is turned on, the cooling capacity range should be 30% to 100% of the available cooling capacity under the current working conditions.

[0117] The above optimization problem is solved by the multi-objective particle swarm optimization algorithm. By setting the number of particles in the population, the spatial search ability of the algorithm is controlled to avoid the algorithm being limited to the local optimal solution. The value within the first time step t in the solution result is the operation control parameter of the air-conditioning system, and t is 30 min.

[0118] The reinforcement learning Markov decision process described in S3 is constructed as follows: Three agents are used to separately schedule and control the operation parameters of the indoor temperature controller, the energy storage tank, and the chiller. Agent A estimates the optimal temperature setpoint T in，set at the current moment through the indoor and outdoor operation state parameters. Agent B learns the changing trend of the building's cooling load from the indoor and outdoor operation state parameters and the temperature set value of Agent A, and combines the operation parameters of the air-conditioning system and the electricity price to schedule the energy release of the energy storage tank at the current moment. Agent C controls the outlet water temperature of the chiller with the goal of reducing the system energy consumption.

[0119] Parameter settings are performed for Agents A, B, and C respectively.

[0120] The state variables of Agent A are as follows:

[0121]

[0122] Among them, Tin，t is the indoor temperature at time t, in °C; T out，t , T out，t-2 , T out，t-4 are the outdoor dry-bulb temperatures at time t, t - 2, and t - 4 respectively, in °C; I t is the solar radiation intensity at time t, in W / m 2 ; RH t is the outdoor relative humidity, in %; E plug,t is the electric power consumption of indoor sockets, in kW; t is the current time; λ t is the electricity price at the current time, in CNY / kWh.

[0123] For agent A, the output action is to reset the indoor temperature setpoint. According to the thermal comfort range, the set temperature reset range is [23°C, 28.5°C]. Considering the limitations of the indoor temperature control controller, the temperature is adjusted in steps of 0.5°C. Therefore, the action space of agent A is set as follows:

[0124]

[0125] The scheduling control objective of agent A is to reduce the operating cost of the air-conditioning system and ensure that the indoor temperature is within the comfort range by adjusting the indoor temperature in the scenario of time-of-use electricity price. The reward function of agent A is set as follows:

[0126]

[0127] In the formula, Q AHU,t is the load of the AHU unit, in kW; EIR ref is the efficiency of the chiller under nominal conditions; ω comfort，t and are the temperature calculation coefficient and the function related to indoor comfort respectively, calculated according to formulas (30) and (31); and are the weight coefficients; is the penalty term coefficient for violating comfort, as shown in formula (42); penalty represents the penalty term and takes a relatively large value.

[0128]

[0129] Agent B optimizes the energy release of the energy storage tank and needs to perceive the building cooling load and its change trend. The state variables should include indoor and outdoor parameters. At the same time, one of the purposes of the combined energy supply mode of the chiller and the tank is to save the operating cost, and the state variables should further include the time-of-use electricity price and time parameters. Finally, to perceive the state of the energy storage tank, the state variables should include some operating parameters of the air-conditioning system. The state variables of agent B are shown as follows.

[0130]

[0131] Among them, T in，set，t is the indoor temperature set point at time t, in °C; T TES，t is the average temperature of the energy storage water tank at time t, in °C.

[0132] For agent B, the output action is to schedule the energy release of the water tank at time t. The energy release is controlled according to the load percentage of the air handling unit (AHU) at time t. The relative proportion range of the energy release of the water tank to the AHU load is [0, 100%], and the adjustment step is 10%. The action space of agent B is set as follows:

[0133]

[0134] The scheduling purpose of agent B is to use the energy storage water tank to transfer the electricity load during high electricity price periods to the low electricity price periods at night, reducing the operating cost of the air conditioning system. At the same time, it is necessary to ensure that the indoor cooling load demand is met, that is, the indoor temperature meets the comfort range. The reward function of agent B is set as follows:

[0135]

[0136] Among them, and are weight coefficients; is the penalty term coefficient for agent B violating the constraints, as shown in Equation (46)

[0137] is the energy consumption of the unit without optimizing the outlet water temperature, calculated as shown in Equation (47).

[0138]

[0139]

[0140] In the formula, Q avi,t is the available cooling capacity under outdoor conditions at time t, in kW; Q hp,t is the load of the air-cooled chiller at time t, in kW; CapFTemp ref and EIRFTemp ref are the function values of CapFTemp and EIRFTemp at the evaporator and condenser temperatures under standard conditions; EIRFPLR t is the EIRFPLR value at time t, related to the chiller load.

[0141] Agent C controls the setpoint of the chilled water temperature of the air-cooled chiller. It needs to perceive the building and chiller load requirements, and the state variables should include indoor and outdoor parameters and the set value of the water discharge energy of the water tank. To meet the requirements of safe operation, the state variables should include operating parameters such as the supply and return water temperatures of the system. Since the goal is to reduce the energy consumption of the air-conditioning system, the operating time and time-of-use electricity price are not included in the state variables. The state variables of Agent C are as follows:

[0142]

[0143] In the formula, T chw,S,t and T chw,R,t are the supply and return water temperatures of the system at time t, in °C; M sys,t is the system flow rate at time t, in m 3 / h.

[0144] For Agent C, the output action is the setpoint of the chilled water temperature at time t. To meet the refrigeration and dehumidification requirements and refer to the equipment performance, the set range of the chilled water temperature is [5 °C, 10 °C], and the adjustment step is 0.5 °C. The action space of Agent C is set as follows:

[0145]

[0146] The adjustment purpose of Agent C is to optimize the energy consumption of the chiller by changing the setpoint of the chilled water temperature based on the current indoor temperature setting and the set value of the water discharge energy of the water tank. The reward function of Agent C is set as follows:

[0147]

[0148] In the formula, P hp，t is the measured power of the chiller at time t, in kW; is the penalty term coefficient for Agent C violating the constraints, calculated as shown in the formula.

[0149]

[0150] The imitation learning described in S3 introduces the loss function related to the expert action into the gradient calculation of the agent network. The loss function of the batch training samples is as shown in Equation (11), and the update process of the agent network parameters θ π is as shown in Equation (12).

[0151] Specifically, for the reinforcement learning scheduling control problem of the air-conditioning system in this example, the imitation learning loss function is introduced into Agent A and Agent B, as shown in Equations (52) and (53).

[0152]

[0153]

[0154] where, f A (·) and f B (·) are respectively the demonstration learning loss functions of agent A and agent B; and are respectively the maximum action value of agent A, the minimum action value of agent B, and the maximum action value of agent B; λ min and λ max are respectively the minimum and maximum values of the time-of-use electricity price. Equation (52) indicates that when in the high electricity price period, the air conditioning system needs to unload to reduce the operating cost and meet the demand response. In this period, the air conditioning set temperature should be a higher value. Equation (53) indicates that when in the low electricity price period, the discharge energy of the energy storage tank should be reduced or the energy storage tank should not be used; when in the high electricity price stage, the energy storage tank should be opened to complete the load transfer.

[0155] In this example, 3 source domain building training off-policy reinforcement learning models with different characteristics are established in advance. Among them, the source domain buildings are established based on the DOE building model and built using the EnergyPlus simulation platform. The source domain buildings are also 3-story office buildings with the same air conditioning system components, but in different climate zones and with different changes in the in-room rate of internal personnel. First, the historical summer cooling load data of the three source domains and the target domain in July and August are statistically obtained, and the load curves of each clustering center are obtained through the K-means algorithm as the typical mode curves. Through the domain similarity test method described in S3, the differences between the cold load curves of the source domain and the target domain are quantified, and the source domain most similar to the target domain is selected for the transfer of the reinforcement learning model.

[0156] In summary, two control methods for the building air conditioning system are constructed, and then according to the data completeness judgment method, the two control methods are switched. On the day before the control, the calculation method described in S1 is used to select the outdoor dry bulb temperature, solar radiation intensity, and outdoor environmental humidity of the weather forecast on the prediction day as the key external disturbance parameters, and compare them with the training set data (the data of the previous 14 days), calculate the JS divergence, and obtain the average value of the calculation results of the three external disturbance JS divergences to obtain the external disturbance difference index; select the training set load curve, perform wavelet decomposition and reconstruction to obtain the high-frequency signal, and calculate the coefficient of variation to obtain the internal disturbance uncertainty index. Select the outdoor dry bulb temperature, date on the prediction day, and the in-room rate of personnel on the previous day (assuming they are all working days) as the in-room rate of personnel on the prediction day for the three variables, calculate the cosine distance between the prediction day and the historical similar day, and obtain the historical similar day difference index. Combine the three types of indexes into a vector and cluster with the historical index data. If the clustering result is high-completeness data, use the model predictive control described in S2 for control, otherwise use the transfer reinforcement learning method described in S3 for control.

[0157] In this embodiment, July 31st to August 29th is selected as the test operation date, and the operation data from July 1st to July 30th is used as the initial training set of the prediction model to initially train the Xgboost load prediction model among them. At the same time, as the system runs, the prediction model is updated incrementally. The transfer imitation learning method described in S3 is used as the reinforcement learning method. The adaptive switching strategy described in S4 is applied, and EnergyPlus is used as the simulation platform for simulation within the operation date to evaluate the operation effect. The results show that compared with the original benchmark strategy, the proposed control strategy can reduce the operation cost by 33.6% and the operation energy consumption by 21.2% while meeting the comfort level. Compared with the single control method without switching, the maximum savings in operation cost is 4.7%, and the maximum reduction in operation energy consumption is 2.8%.

[0158] Furthermore, to verify the robustness of the proposed control method, simulations in scenarios of input uncertainty are carried out. Random errors are introduced into the input values of outdoor dry-bulb temperature and relative humidity to simulate the uncertainty brought by inaccurate weather forecasts, and the random errors follow a normal distribution. The results show that compared with the original benchmark strategy, the proposed control strategy can still reduce the operation cost by 24.8% and the operation energy consumption by 18.7% while meeting the comfort level. It is proved that the proposed control method has good robustness. When facing uncertainties in the input, compared with the single control method without switching, the maximum savings in operation cost is 6.7%, and the maximum reduction in operation energy consumption is 5.0%.

[0159] It shows that the proposed control method has a more significant performance improvement compared with the single control method when facing data uncertainties.

Claims

1. An adaptive control method for an air conditioning system based on information matching, characterized in that, Including the following steps: S1 Information Completeness Judgment: First, establish a training set - prediction set data completeness judgment index for the external disturbance, internal disturbance, and historical similar day factors of building load prediction: including quantifying the difference in the external disturbance distribution between the prediction day and the training set through JS divergence; reconstructing the load curve through wavelet decomposition to obtain the high-frequency signal reflecting the internal disturbance in the training set data, and calculating the coefficient of variation value to reflect the signal fluctuation and the uncertainty of the internal disturbance; calculating the cosine distance of the normalized feature vectors of the external and internal disturbances between the prediction day and the historical day. The historical day with the minimum cosine distance is the most similar to the prediction day, and this value is used as the historical similar day difference index. Combine the three types of indexes to form an information completeness index vector, and cluster the data set according to the information completeness index through spectral clustering, and divide the load prediction data set into low and high completeness. S2 Constructing a Model Predictive Control Method: 1) According to the high-completeness data set, establish a data-driven load prediction model to predict the building cooling load within a time window of length T. 2) According to the prediction results and the constructed air-conditioning system model, optimize the control variables for the next T time intervals to minimize the objective function and satisfy the constraint conditions; 3) Use the optimized results of the control variables within the first time window as the scheduling control signal and apply it to the air-conditioning system. 4) Update the operating data according to the actual air-conditioning system, and repeat 1) to 4) to complete the rolling prediction optimization. S3 Constructing a Deep Reinforcement Learning Control Method: The present invention proposes a transfer imitation learning control method based on deep reinforcement learning: 1) First, construct the state space, action space, and reward function of the Markov decision process according to the air-conditioning system, and construct the algorithm framework and training process of the deep Q network. 2) Offline train the value function Q neural network in the pre-constructed source domain simulation building air-conditioning system. At the same time, introduce imitation learning during the training process to increase the training and parameter adjustment efficiency of the reinforcement learning model: introduce the expert strategy into the gradient calculation of the agent network to provide a reference for the agent's actions under specific working conditions. 3) Conduct a domain similarity test between the target building air-conditioning system and the source domain building air-conditioning system. According to the test results, select the offline-trained deep reinforcement learning network of the source domain most similar to the target domain and transfer it to the target domain. When transferring, freeze the parameters of the first N layers of the deep Q network, that is, the parameters of the underlying feature extractor, and further online train the remaining value function network parameters in the target domain air-conditioning system and provide control actions. S4 Adaptive Regulation Method Switching: One day before the day to be controlled, combined with historical data, pre-judge the reliability of the load prediction on that day according to the data completeness judgment method described in S1 and select the regulation method. For the high-completeness prediction data, select the model predictive control method established in S2. For the low-completeness prediction data, select the transfer imitation learning control method established in S3 to achieve the adaptive recognition of different working conditions and the flexible switching of the regulation methods.

2. The adaptive regulation method of the air conditioning system based on information matching according to claim 1, wherein, The method for judging the information completeness of the S1 data completeness judgment index is as follows: Quantify the difference in the external disturbance distribution between the prediction day and the training set through JS divergence; reconstruct the load curve through wavelet decomposition to obtain the high-frequency signal reflecting the internal disturbance in the training set data, and calculate the coefficient of variation value to reflect the signal fluctuation and the uncertainty of the internal disturbance; calculate the cosine distance of the normalized feature vectors of the internal and external disturbances between the prediction day and the historical day. The historical day with the minimum cosine distance is the most similar to the prediction day, and this value is used as the difference index for the historical similar day. Combine the three types of indexes to form an information completeness index vector, and cluster the data set according to the information completeness index through spectral clustering, and divide the load prediction data set into low and high completeness levels.

3. The adaptive regulation method of the air-conditioning system based on information matching according to claim 1, characterized in that In S3, during the reinforcement learning training process, introduce the expert policy into the gradient of the agent network to improve the problem of sparse rewards in the reinforcement learning running environment and improve the training efficiency. The loss function of the batch training samples introduced is as follows: where the function f(π(s i |θ π ), a i ) describes the gap between the agent policy and the expert policy, N D is the number of samples taken from the experience pool, π(s i |θ π ) is the agent policy of the i-th sample, a i is the expert policy of the i-th sample, and is the agent action as a reference under specific operating conditions. Then the agent network parameter θ π is updated as shown in the equation: Where α is the learning rate, λ1 and λ2 are weight coefficients, J is the loss function of the deep Q-network algorithm, and L lfd is the loss function of imitation learning. Therefore, in the above formula, in addition to the reinforcement learning loss function being optimized, the loss function of imitation learning is also optimized, that is, the difference between the agent and the expert control actions is reduced, which enables the expert policy to be added to the agent network update. Specifically, for the problem of reinforcement learning scheduling control of the air-conditioning system in the embodiment of the present invention, the imitation learning loss function is introduced into the agent A for controlling the set value of the indoor air-conditioning temperature and the agent B for controlling the cooling capacity released by the energy storage water tank, as follows: The agent action is as follows: where, f A (·) and f B (·) are the imitation learning loss functions of agent A and agent B respectively, and the action is the room temperature set point, °C; the action represents the ratio of the energy released from the water tank to the AHU load. and are the maximum action value of agent A, the minimum action value of agent B, and the maximum action value of agent B respectively; λ min and λ max are the minimum and maximum values of the time-of-use electricity price respectively. According to updating the agent network parameter θ means that when it is in the high electricity price period, the air conditioning system needs to unload to reduce the operating cost and meet the demand response. During this period, the air conditioning set temperature should be a higher value. According to updating the agent network parameter θ means that when it is in the low electricity price period, the cooling capacity released from the energy storage water tank should be reduced or the energy storage water tank should not be used; when it is in the high electricity price stage, the energy storage water tank should be opened to complete the load transfer.

4. The adaptive control method of the air-conditioning system based on information matching according to claim 1, wherein In the transfer learning in S3, to ensure the similarity and matching of the source domain knowledge and the target domain in the transfer learning, the steps of the domain similarity test method include: First, obtain the typical load curve sets of the source domain and the target domain through a clustering algorithm, further use cross-spectrum analysis to judge the shape similarity of the typical load curves of the source domain and the target domain, and finally, according to the similarity judgment, select the source domain most similar to the target domain and transfer the deep reinforcement learning network. The method for establishing the similarity judgment index of the load curve according to cross-spectrum analysis is to obtain the coherence spectrum and the phase spectrum by solving the cross-spectrum density function of two load sequences, further perform normalization processing on the two and calculate the difference as the similarity index. The closer the result is to 1, the more similar the load curves are, and the more similar the source domain building and the target domain are.

5. The adaptive control method for an air conditioning system based on information matching according to claim 1, wherein The steps for switching the adaptive regulation method in S4 are as follows: One day before the control day, combined with historical data, pre-judge the reliability of the load prediction on the current day according to the data completeness judgment method described in S1 and select the regulation method. For the high-completeness prediction data, select the model prediction control method established in S2. For the low-completeness prediction data, select the transfer imitation learning control method established in S3 to achieve the flexible switching of the adaptive identification and regulation methods for different working conditions.

Images