A cold source control method and air conditioning system
By sampling the state parameters of the air conditioning system in real time and using a target model based on deep reinforcement learning for cold source control, the problem of time-consuming, labor-intensive, and difficult-to-switch manual control is solved, achieving efficient and accurate cold source control.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- QINGDAO HISENSE TRANS TECH
- Filing Date
- 2025-12-29
- Publication Date
- 2026-06-23
AI Technical Summary
Manually controlling the switching of cold sources in a dual-cold-source air conditioning system is time-consuming, labor-intensive, and difficult to achieve precisely.
By sampling the state parameters of the air conditioning system in real time, the target model is used to predict the actions and obtain the predicted cold source execution parameters. The cold source is then controlled in combination with the current cold source execution parameters. The target model includes a policy network and an evaluation network. The cold source switching strategy is optimized through deep reinforcement learning.
It achieves efficient and precise cold source control of the air conditioning system, improving the precision of temperature regulation and the stability of system operation.
Smart Images
Figure CN121430165B_ABST
Abstract
Description
Technical Field
[0001] This application relates to the field of air conditioning control technology, and in particular to a cold source control method and an air conditioning system. Background Technology
[0002] With the development of technology and the increasing demand for temperature regulation, ordinary single-cooling air conditioners are no longer sufficient to meet the temperature regulation needs of some scenarios, requiring the use of dual-cooling-source air conditioning systems. Dual-cooling-source air conditioning systems have two independent cooling source systems, such as an air-cooled system and a water-cooled system, which enable more precise temperature regulation.
[0003] In related technologies, the cooling source control of dual-source air conditioning systems relies on manual intervention. For example, when the load is low, the system is manually switched to an air-cooled system, and when the load is high, it is manually switched to a water-cooled system.
[0004] However, manual control of the cold source is not only time-consuming and labor-intensive, but also difficult to achieve precise control. Summary of the Invention
[0005] This application provides a cold source control method and an air conditioning system for precise and efficient cold source control.
[0006] In a first aspect, embodiments of this application provide a cold source control method applied to an air conditioning system, the method comprising:
[0007] The current status parameters of the air conditioning system are sampled in real time; wherein, the status parameters include environmental parameters, operating parameters, and performance parameters.
[0008] Based on the current state parameters, action prediction is performed to obtain the corresponding predicted cold source execution parameters;
[0009] Based on the predicted cold source execution parameters and combined with the current cold source execution parameters of the air conditioning system, the air conditioning system is subjected to cold source control; wherein, the predicted cold source execution parameters and the current cold source execution parameters each include execution information corresponding to multiple operating items.
[0010] In some optional implementations, action prediction is performed based on the current state parameters to obtain the corresponding predicted cold source execution parameters, including:
[0011] The current state parameters are input into the policy network of the target model, and the policy network is used to predict the actions based on the current state parameters to obtain the predicted cold source execution parameters; wherein, the target model includes the policy network and the evaluation network;
[0012] The target model is obtained through the following methods:
[0013] The target model is trained iteratively multiple times based on the sample dataset; wherein each sample dataset includes a first state parameter, a first cold source execution parameter, a second state parameter, and a reward value for the second state parameter; the second state parameter is the state parameter reached after executing the first cold source execution parameter on the first state parameter.
[0014] In some optional implementations, each iteration includes:
[0015] For any selected sample data, the first state parameter and the first cold source execution parameter of the sample data are input into the evaluation network to obtain the corresponding first evaluation value;
[0016] The evaluation network is tuned based on the first evaluation value and the target evaluation value; wherein the target evaluation value is determined based on the reward value in the sample data.
[0017] The first state parameters of the sample data are input into the strategy network to obtain the second cold source execution parameters;
[0018] The first state parameters and the second cold source execution parameters of the sample data are input into the evaluation network to obtain the corresponding second evaluation value, and the strategy network is adjusted based on the second evaluation value.
[0019] Some optional implementations also include:
[0020] Obtain the reward value corresponding to each of the multiple state parameters sampled within the first target time period, and compare the obtained multiple reward values with the reward threshold respectively;
[0021] When the number of reward values less than the reward threshold exceeds a preset number, the target model is updated.
[0022] In some alternative implementations, the corresponding reward value is determined for any given state parameter in the following manner:
[0023] The weighted sum of the multiple rewards corresponding to the state parameters is used to obtain the corresponding reward value.
[0024] The environmental parameters include electricity consumption information; the operating parameters include multiple refrigeration parameters, unit operating time, and number of unit start-ups and shutdowns; the performance parameters include total cooling source power and energy efficiency ratio; and the multiple awards include some or all of the following:
[0025] An energy efficiency bonus is obtained by comparing the energy efficiency coefficient in the state parameters with the target energy efficiency coefficient.
[0026] The cost reward is obtained based on the electricity consumption information and total power of the cold source in the aforementioned status parameters;
[0027] A stability bonus is obtained based on the difference between the state parameter and at least one cooling parameter of the previous state parameter; the at least one cooling parameter includes some or all of the cooling capacity, chilled water supply temperature, and chilled water supply pressure.
[0028] The health reward is obtained based on the unit's operating time and the number of unit start-ups and shutdowns in the status parameters.
[0029] In some optional implementations, based on the predicted cold source execution parameters and combined with the current cold source execution parameters of the air conditioning system, cold source control of the air conditioning system is performed, including:
[0030] When there is no first operating item, the air conditioning system shall continue to operate according to the current cold source execution parameters;
[0031] When a first operating item is available, the air conditioning system is adjusted based on the predicted cold source execution parameters;
[0032] Wherein, the first running item is the running item among the plurality of running items where the first execution information and the second execution information satisfy the corresponding preset switching conditions, the first execution information corresponds to the predicted cold source execution parameters, and the second execution information corresponds to the current cold source execution parameters.
[0033] In some optional implementations, the air conditioning system is adjusted based on the predicted cold source execution parameters, including:
[0034] Based on the predicted cold source execution parameters during the second target time period, the target cold source execution parameters are obtained;
[0035] The air conditioning system is switched to operate with the parameters set by the target cold source.
[0036] In some optional implementations, the target cold source execution parameters are obtained based on the predicted cold source execution parameters during the second target time period, including:
[0037] Based on the predicted cold source execution parameters within the second target time period, and combined with the value range corresponding to the second operation item among the multiple operation items, the target cold source execution parameters are obtained.
[0038] In some optional implementations, when the first operation item includes a cold source type operation item, switching the air conditioning system to operate with the target cold source execution parameters includes:
[0039] After the first cold source system is turned on for the target duration, the second cold source system is turned off; wherein, the first cold source corresponds to the target cold source execution parameters, and the second cold source corresponds to the current cold source execution parameters.
[0040] Secondly, embodiments of this application provide a cold source control device applied to an air conditioning system, the device comprising:
[0041] The sampling module is used to sample the current status parameters of the air conditioning system in real time; wherein, the status parameters include environmental parameters, operating parameters, and performance parameters;
[0042] The action prediction module is used to predict actions based on the current state parameters and obtain the corresponding predicted cold source execution parameters.
[0043] The control module is used to control the air conditioning system based on the predicted cold source execution parameters and the current cold source execution parameters of the air conditioning system; wherein the predicted cold source execution parameters and the current cold source execution parameters each include execution information corresponding to multiple operating items.
[0044] Thirdly, embodiments of this application provide an air conditioning system, including at least one controller and at least one memory; wherein the memory stores a computer program, and when the program is executed by the controller, the controller performs the cold source control method as described in any of the first aspects above.
[0045] Fourthly, embodiments of this application provide a computer-readable storage medium storing a computer program executable by a processor, which, when run on the processor, causes the processor to perform any of the cold source control methods described in the first aspect above.
[0046] In this embodiment, the current state parameters of the air conditioning system are sampled in real time. These state parameters include environmental parameters, operating parameters, and performance parameters of the air conditioning system from multiple dimensions. Since the state parameters describe the state of the air conditioning system from multiple dimensions, action prediction based on the state parameters can obtain predicted cold source execution parameters suitable for the current state of the air conditioning system. Furthermore, based on the predicted cold source execution parameters and combined with the current cold source execution parameters of the air conditioning system, efficient and accurate cold source control of the air conditioning system can be achieved. Attached Figure Description
[0047] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.
[0048] Figure 1 An air conditioning system architecture diagram provided for an embodiment of this application;
[0049] Figure 2A schematic flowchart illustrating the first cold source control method provided in this application embodiment;
[0050] Figure 3 A schematic flowchart illustrating the second cold source control method provided in this application embodiment;
[0051] Figure 4 A flowchart illustrating the target model training method provided in this application embodiment;
[0052] Figure 5 A flowchart illustrating the third cold source control method provided in this application embodiment;
[0053] Figure 6 This is a schematic diagram of the structure of a cold source control device provided in an embodiment of this application;
[0054] Figure 7 This is a schematic diagram of the structure of an air conditioning system provided in an embodiment of this application. Detailed Implementation
[0055] To make the objectives, technical solutions, and advantages of this application clearer, the application will be further described in detail below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments in this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.
[0056] The terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Therefore, a feature defined as "first" or "second" may explicitly or implicitly include one or more of that feature. In the description of this application, unless otherwise stated, "a plurality of" means two or more.
[0057] In the description of this application, it should be noted that, unless otherwise expressly specified and limited, the term "connection" should be interpreted broadly. For example, it can refer to a direct connection, an indirect connection through an intermediate medium, or a connection within two devices. Those skilled in the art can understand the specific meaning of the above term in this application based on the specific circumstances.
[0058] With the development of technology and the increasing demand for temperature regulation, ordinary single-cooling air conditioners are no longer sufficient to meet the temperature regulation needs of some scenarios, requiring the use of dual-cooling-source air conditioning systems. Dual-cooling-source air conditioning systems have two independent cooling source systems, such as an air-cooled system and a water-cooled system, which enable more precise temperature regulation.
[0059] In related technologies, the cooling source control of dual-source air conditioning systems relies on manual intervention. For example, when the load is low, the system is manually switched to an air-cooled system, and when the load is high, it is manually switched to a water-cooled system.
[0060] However, manual control of the cold source is not only time-consuming and labor-intensive, but also difficult to achieve precise control.
[0061] Therefore, this application proposes a cold source control method and an air conditioning system for precise and efficient cold source control.
[0062] See Figure 1 As shown, this embodiment provides an air conditioning system, including a cold source system 1, a cold source system 2, and a controller.
[0063] Among them, cold source system 1 and cold source system 2 are two independent cold source systems. Taking cold source system 1 as an air-cooled system as an example, cold source system 1 includes multiple air-cooled heat pumps; taking cold source system 2 as a water-cooled system as an example, cold source system 2 includes multiple chiller units.
[0064] The above Figure 1 This is merely an illustrative example of an air conditioning system. In practice, an air conditioning system may also include other components such as terminal devices, which will not be described in detail here.
[0065] The technical solution of this application and how it solves the above-mentioned technical problems will be described in detail below with reference to the accompanying drawings and specific embodiments. The following specific embodiments can be combined with each other, and the same or similar concepts or processes may not be described again in some embodiments.
[0066] Figure 2 This is a flowchart illustrating the first cold source control method provided in this application embodiment, which can be applied to the aforementioned controller, such as... Figure 2 As shown, it includes the following steps:
[0067] Step S201: Sample the current status parameters of the air conditioning system in real time; wherein, the status parameters include environmental parameters, operating parameters and performance parameters.
[0068] During implementation, the current status parameters of the air conditioning system are sampled in real time. These status parameters include environmental parameters, operating parameters, and performance parameters of the air conditioning system from multiple dimensions. Therefore, the status parameters describe the current state of the air conditioning system from multiple dimensions.
[0069] This application does not specifically limit the environmental parameters, operating parameters, and performance parameters. For example, environmental parameters may include any parameters related to the external and internal environments of the air conditioning system, operating parameters may include any parameters generated by the air conditioning system during operation, and performance parameters may include any parameters that can describe the performance of the air conditioning system.
[0070] In some alternative implementations, environmental parameters include weather information (such as outdoor temperature and outdoor humidity), cooling load information (such as actual cooling load and predicted cooling load), and electricity consumption information (such as electricity price).
[0071] The operating parameters include the operating status (such as start-up, shutdown, load, etc.) of each unit (including units and terminal equipment of multiple cooling source systems), multiple refrigeration parameters (such as chilled water supply temperature, chilled water supply flow rate, chilled water return temperature, chilled water return flow rate, cooling water supply temperature, cooling water supply flow rate, cooling water return temperature, cooling water return flow rate, and cooling capacity), unit operating time, and the number of unit start-ups and shutdowns (total number of start-ups and shutdowns).
[0072] The performance parameters include total power of the cooling source, total power of the terminal, equivalent number of terminal units, coefficient of performance (COP), comfort index (user-triggered), and equipment fault information.
[0073] Some of the parameters mentioned above are obtained directly from data collection, while others are obtained through calculation or prediction. For example, the actual cooling load, total terminal power, equivalent number of terminals, and energy efficiency ratio are obtained through calculation, while the predicted cooling load is obtained through prediction.
[0074] For example, actual cooling load = preset coefficient Chilled water return flow rate The temperature difference between chilled water supply and return water;
[0075] The total power at the terminal is the sum of the actual power of all terminal devices;
[0076] Equivalent number of terminal units = Total terminal power / Base power, Base power = Sum of rated power of all terminal devices / Total number of terminal devices;
[0077] Energy efficiency coefficient = cooling capacity / total power of cold source.
[0078] In some alternative implementations, the predicted cooling load described above can be obtained through a load prediction model.
[0079] For example, the data sequence is input into the load forecasting model to obtain the predicted cold load at the target time.
[0080] The data sequence includes multiple data sets corresponding to different historical moments. These data sets include some or all of the outdoor temperature and humidity, solar radiation intensity, actual cooling load, and terminal operation data.
[0081] The aforementioned load forecasting model can employ a Long Short-Term Memory (LSTM) model based on an encoding and decoding architecture. By using an attention mechanism to capture long- and short-term dependencies in the data sequence, it can dynamically enhance the characteristics of historical moments and data groups with significant influence, thereby improving forecast accuracy.
[0082] The training process of the load forecasting model is illustrated below:
[0083] Obtain first sample data selected from the dataset for training, and second sample data for validation (the ratio of the two can be 8:2). Both the first and second sample data include the sample data sequence and the target cold load.
[0084] The load forecasting model is iteratively trained based on the first sample data, and then validated based on the second sample data sequence. Each iteration includes:
[0085] Input the sample data sequence of the selected first sample data into the load prediction model to obtain the predicted cooling load;
[0086] Determine the prediction error between the predicted cooling load and the target cooling load;
[0087] The load forecasting model is tuned using the mean squared error (MSE) of the prediction errors of all selected first sample data as the loss function. MSE amplifies the penalty for larger errors through the squared term, forcing the model to prioritize addressing significant prediction biases. This is suitable for forecasting tasks that aim to capture extreme values or are sensitive to large errors, providing a clear and differentiable optimization objective for model training.
[0088] During iteration, if the loss function does not decrease significantly over several consecutive rounds, the learning rate can be adjusted to promote stable convergence of the model toward a deeper minimum.
[0089] In practice, Adam (an optimizer) can be used to efficiently optimize the loss function. Adam is an advanced optimization algorithm that combines the ideas of momentum and adaptive learning rate. This mechanism makes Adam suitable for training models with sparse gradients or noise problems, such as LSTM.
[0090] The above-described methods for obtaining predicted cooling load are merely illustrative. In practice, predicted cooling load may not be used, or other methods may be employed to obtain predicted cooling load. This application does not impose any specific limitations on these methods.
[0091] Step S202: Based on the current state parameters, perform action prediction to obtain the corresponding predicted cold source execution parameters.
[0092] Since state parameters describe the state of the air conditioning system from multiple dimensions, combining state parameters to predict actions can yield predicted cold source execution parameters suitable for the current state of the air conditioning system.
[0093] Step S203: Based on the predicted cold source execution parameters and combined with the current cold source execution parameters of the air conditioning system, perform cold source control on the air conditioning system; wherein, the predicted cold source execution parameters and the current cold source execution parameters each include execution information corresponding to multiple operating items.
[0094] The above scheme samples the current state parameters of the air conditioning system in real time. These state parameters include environmental parameters, operating parameters, and performance parameters of the air conditioning system from multiple dimensions. Since the state parameters describe the state of the air conditioning system from multiple dimensions, action prediction based on the state parameters can obtain predicted cold source execution parameters suitable for the current state of the air conditioning system. Furthermore, based on the predicted cold source execution parameters and combined with the current cold source execution parameters of the air conditioning system, efficient and accurate cold source control of the air conditioning system can be achieved.
[0095] Figure 3 A flowchart illustrating the second cold source control method provided in this application embodiment is shown below. Figure 3 As shown, it includes the following steps:
[0096] Step S301: Sample the current status parameters of the air conditioning system in real time; wherein, the status parameters include environmental parameters, operating parameters and performance parameters.
[0097] The specific implementation of step S301 can be found in the above embodiments, and will not be repeated here.
[0098] Step S302: Input the current state parameters into the policy network of the target model, and use the policy network to predict the actions of the current state parameters to obtain the predicted cold source execution parameters; wherein, the target model includes the policy network and the evaluation network.
[0099] The target model is obtained through the following methods:
[0100] The target model is trained iteratively multiple times based on the sample dataset; wherein each sample dataset includes a first state parameter, a first cold source execution parameter, a second state parameter, and a reward value for the second state parameter; the second state parameter is the state parameter reached after executing the first cold source execution parameter on the first state parameter.
[0101] In implementation, a target model comprising a policy network (Actor) and an evaluation network (Critic) is trained based on a sample dataset. Since the sample data includes a first state parameter s, a first cold source execution parameter a, a second state parameter (next state parameter s'), and a reward value r, i.e., [s, a, r, s'], the Critic, combined with the sample data, can learn the score (evaluation value) achievable by executing a cold source execution parameter for a given state parameter. This helps the Actor learn the correlation between the state parameter and the cold source execution parameter, thereby enabling the Actor to accurately predict actions.
[0102] The embodiments of this application do not limit the specific algorithm used in the target model, and may use the DDPG algorithm (a policy gradient algorithm), SARSA (an online policy reinforcement learning algorithm), etc.
[0103] For example, the cold source control is modeled as a Markov Decision Process (MDP), and the optimal control strategy is learned through Deep Reinforcement Learning (DRL). In this model, the target model acts as the agent, state parameters represent the states, cold source execution parameters represent the actions, and reward values represent the rewards. Through self-learning, the system finds suitable cold source execution parameters for the air conditioning system.
[0104] Step S303: Based on the predicted cold source execution parameters and combined with the current cold source execution parameters of the air conditioning system, perform cold source control on the air conditioning system; wherein, the predicted cold source execution parameters and the current cold source execution parameters each include execution information corresponding to multiple operating items.
[0105] The specific implementation of step S303 can be found in the above embodiments, and will not be repeated here.
[0106] In some alternative implementations, the corresponding reward value for any state parameter can be determined in the following ways:
[0107] The weighted sum of the multiple rewards corresponding to the state parameters is used to obtain the corresponding reward value.
[0108] In implementation, multiple rewards are assigned their own weights, and a balance point for multiple rewards is learned through the DRL algorithm to obtain reasonable weights. The weighted sum of the multiple rewards corresponding to the state parameters is then performed to obtain the corresponding reward value, achieving global optimization across multiple objectives.
[0109] Multiple awards include some or all of the following:
[0110] 1. An energy efficiency bonus is obtained by comparing the energy efficiency coefficient in the state parameters with the target energy efficiency coefficient.
[0111] The coefficient of performance (COP) is calculated as cooling capacity divided by the total power of the cooling source. Therefore, a higher COP indicates a higher efficiency in converting electrical energy into cooling capacity. In practice, energy efficiency bonuses are determined based on the COP, enabling the air conditioning system to operate efficiently while meeting load demands.
[0112] For example, when the energy efficiency coefficient in state parameter i is less than the target energy efficiency coefficient, the energy efficiency bonus COP of state parameter i is... i =exp{- [(COP i - COP o ) 2 / (2 σ1 2 When the energy efficiency coefficient in state parameter i is greater than or equal to the target energy efficiency coefficient, the energy efficiency bonus COP of state parameter i is increased. i =exp{- [(COP i - COP o ) 2 / (2 σ2 2 )]};wherein, COP i COP is the energy efficiency coefficient for state parameter i. o σ1 is the target energy efficiency coefficient, σ2 is the first coefficient, and σ2 is the second coefficient.
[0113] In some alternative implementations, σ1 > σ2, indicating that when the coefficient of performance (COP) is less than the target COP, the energy efficiency reward decreases slowly (at this point, it's just low efficiency), while when it's greater than the target COP, the energy efficiency reward decreases rapidly. When the COP is less than the optimal COP, it's just low efficiency; when the COP is greater than the optimal COP, it may have entered the unstable region, and therefore, the energy efficiency reward decreases even faster.
[0114] 2. Cost incentives obtained based on the electricity consumption information and total power of the cold source in the status parameters; the electricity consumption information includes electricity price.
[0115] For example, the energy efficiency reward Rcost for state parameter i i =Renergy+Rdemand,Renergy=- Price i Ptotal i γ Rdemand = -λ max(0, Ptotal i - Ptarget)²
[0116] Among them, Price i Let Ptotal be the electricity price for state parameter i. iLet be the total power of the cold source with state parameter i, Ptarget be the power threshold, Renergy be the bonus of the base cost, Rdemand be the power penalty (which is a non-positive number), γ be the third coefficient, and λ be the fourth coefficient.
[0117] In Renergy, γ can be a fixed value or a dynamically changing value. For example, when the electricity price is higher than the preset price, γ>1, meaning that at high electricity prices, the same total cooling power will be penalized more and rewarded less. When the electricity price is less than or equal to the preset price, γ<1, meaning that at low electricity prices, the same total cooling power will be penalized less and rewarded more. Furthermore, setting corresponding γ values for different electricity price ranges allows for more precise adjustments.
[0118] In Rdemand, when Ptotal i When Ptarget is less than or equal to Ptarget, Rdemand is 0, and there is no power penalty; when Ptotal ... i When Ptarget is greater than Ptotal, Ptotal i The bigger the punishment, the greater the penalty.
[0119] 3. A stability bonus is obtained based on the difference between the state parameter and at least one cooling parameter of the previous state parameter; the at least one cooling parameter includes some or all of the cooling capacity, chilled water supply temperature, and chilled water supply pressure.
[0120] In practice, smaller fluctuations in cooling capacity indicate a more stable supply load, and smaller fluctuations in water supply indicate a more stable water supply status. Therefore, a stability bonus can be obtained by combining the differences between the current state parameters and the previous state parameters, namely cooling capacity, water supply temperature, and water supply pressure.
[0121] For example, the energy efficiency reward Rstability of state parameter i i =β1 W1+β2 W2+β3 W3; where β1 is the fifth coefficient, β2 is the sixth coefficient, β3 is the seventh coefficient, W1 is the difference in cooling capacity between state parameter i and state parameter i-1, W2 is the difference in chilled water supply temperature between state parameter i and state parameter i-1, and W3 is the difference in chilled water supply pressure between state parameter i and state parameter i-1.
[0122] 4. The health reward is obtained based on the unit running time and the number of unit start-ups and shutdowns in the status parameters.
[0123] In practice, balancing the operating time of the units, ensuring that the same units do not operate simultaneously, can extend the lifespan of the entire air conditioning system. Conversely, overusing a particular unit can lead to premature aging, while frequent start-stop cycles are a major factor contributing to unit wear and tear. Therefore, health rewards can be awarded based on unit operating time and the number of start-stop cycles.
[0124] For example, the health reward Rhealth for state parameter i i =δ1 σhours+δ2 Nstart, δ1 is the eighth coefficient, δ1 is the ninth coefficient, σhours is the standard deviation of the unit's operating time, and Nstart is the number of times the unit starts and stops.
[0125] Figure 4 This is a flowchart illustrating the target model training method provided in the embodiments of this application, as shown below. Figure 4 As shown, it includes the following steps:
[0126] Step S401: Train the target model iteratively multiple times based on the sample dataset; wherein each iteration includes:
[0127] Step S4011: For any selected sample data, input the first state parameter and the first cold source execution parameter of the sample data into the evaluation network to obtain the corresponding first evaluation value.
[0128] In practice, the evaluation network is used to score the combination of state parameters and cold source execution parameters. When training the evaluation network, it is necessary to determine the first evaluation value (current Q) corresponding to the first state parameter and the first cold source execution parameter through the evaluation network.
[0129] Step S4012: Based on the first evaluation value and the target evaluation value, the parameters of the evaluation network are tuned; wherein, the target evaluation value is determined based on the reward value in the sample data.
[0130] In order for the evaluation network to accurately score the combination of state parameters and cold source execution parameters, the evaluation network needs to be tuned based on the first evaluation value and the target evaluation value (target Q) so that the current Q output by the evaluation network gets closer and closer to the corresponding target Q.
[0131] For example, the evaluation network is tuned based on the difference between the first evaluation value and the target evaluation value.
[0132] The objective Q mentioned above is the sum of the reward value in the sample data and the long-term cumulative reward.
[0133] Step S4013: Input the first state parameters of the sample data into the strategy network to obtain the second cold source execution parameters.
[0134] In implementation, the policy network is used to generate cold source execution parameters based on the state parameters. When training the policy network, it is necessary to obtain the second cold source execution parameters corresponding to the first state parameters through the policy network, and the evaluation network scores the combination of the first state parameters and the second cold source execution parameters.
[0135] Step S4014: Input the first state parameters and the second cold source execution parameters of the sample data into the evaluation network to obtain the corresponding second evaluation value, and adjust the parameters of the strategy network based on the second evaluation value.
[0136] As described above, when training the policy network, it is necessary to obtain the second cold source execution parameters corresponding to the first state parameters through the policy network, and the evaluation network scores the combination of the first state parameters and the second cold source execution parameters to obtain the second evaluation value.
[0137] This second evaluation value reflects whether the second cold source execution parameters predicted by the strategy network are suitable for the first state parameters. Therefore, the strategy network can be tuned based on the second evaluation value, so that the combination of the input state parameters and the output cold source execution parameters of the strategy network yields increasingly higher scores. In this way, during use, the strategy network can accurately output the predicted cold source execution parameters corresponding to the current state parameters.
[0138] During implementation, the target model is obtained by reaching the preset number of iterations or achieving convergence and stability during the iterative process.
[0139] In some alternative implementations, based on any embodiment, the following steps may also be performed:
[0140] Obtain the reward value corresponding to each of the multiple state parameters sampled within the first target time period, and compare the obtained multiple reward values with the reward threshold respectively;
[0141] When the number of reward values less than the reward threshold exceeds a preset number, the target model is updated.
[0142] In implementation, the control effect is evaluated based on the reward values corresponding to the sampled state parameters during the cold source control process. For example, if the number of reward values less than the reward threshold exceeds a preset number, it indicates that most of the predicted cold source execution parameters output by the target model are inaccurate, resulting in lower rewards for the adjusted state parameters. Therefore, the target model is updated. If the number of reward values less than the reward threshold does not exceed a preset number, it indicates that most of the predicted cold source execution parameters output by the target model are relatively accurate. Therefore, the adjusted state parameter rewards are high, and no update of the target model is necessary.
[0143] The update process of the target model can be referred to the training process of the target model, and will not be repeated here.
[0144] In some optional implementations, after obtaining the updated target model, the updated target model and the original target model are validated based on the sample dataset. When the performance of the updated target model is better than that of the original target model, the original target model is then replaced with the updated target model.
[0145] Figure 5 A flowchart illustrating the third cold source control method provided in this application embodiment is shown below. Figure 5 As shown, it includes the following steps:
[0146] Step S501: Sample the current status parameters of the air conditioning system in real time; wherein, the status parameters include environmental parameters, operating parameters and performance parameters.
[0147] Step S502: Based on the current state parameters, perform action prediction to obtain the corresponding predicted cold source execution parameters.
[0148] The specific implementation of steps S501 to S502 can be referred to the above embodiments, and will not be repeated here.
[0149] Step S503: Determine whether there is a first running item among the plurality of running items.
[0150] Wherein, the first running item is the running item among the plurality of running items where the first execution information and the second execution information satisfy the corresponding preset switching conditions, the first execution information corresponds to the predicted cold source execution parameters, and the second execution information corresponds to the current cold source execution parameters.
[0151] In this embodiment of the application, the first execution information and the second execution information in each first running item are compared to determine whether the corresponding preset switching conditions are met.
[0152] In practice, as long as there is an operating item that meets the corresponding preset switching conditions, it is determined as the first operating item, triggering the switching of cold source execution parameters (adjusting the air conditioning system based on the predicted cold source execution parameters), i.e., executing step S505; if there is no operating item that meets the corresponding preset switching conditions (no first operating item), the air conditioning system is kept running according to the current cold source execution parameters.
[0153] Different running items have different preset switching conditions. The following are some specific examples to illustrate this:
[0154] For cold source type operation items, as long as the first execution information is different from the second execution information, that is, the predicted cold source execution parameters are different from the cold source system in the current cold source execution parameters, the preset switching conditions are determined to be met.
[0155] For non-cold source type operation items (such as unit start-up quantity operation item, chilled water supply temperature operation item, and chilled water return temperature operation item, etc.), when the deviation between the first execution information and the second execution information is greater than the preset deviation, it is determined that the preset switching condition is met.
[0156] Step S504: Keep the air conditioning system running according to the current cold source execution parameters.
[0157] When there is no first operating item, it indicates that the predicted cold source execution parameters are highly consistent with the current cold source execution parameters. Therefore, the air conditioning system should be kept running according to the current cold source execution parameters.
[0158] Step S505: Adjust the air conditioning system based on the predicted cold source execution parameters.
[0159] When there is a first operating item, it indicates that the predicted cold source execution parameters are not very consistent with the current cold source execution parameters. Therefore, the air conditioning system is adjusted based on the predicted cold source execution parameters.
[0160] In some optional implementations, step S505 above can be implemented in, but is not limited to, the following ways:
[0161] Based on the predicted cold source execution parameters during the second target time period, the target cold source execution parameters are obtained;
[0162] The air conditioning system is switched to operate with the parameters set by the target cold source.
[0163] This application embodiment obtains target cold source execution parameters that match the state of the air conditioning system during the second target time period by statistically analyzing the predicted cold source execution parameters within that period. Instead of controlling the cold source based on a single predicted cold source execution parameter, this approach achieves more stable cold source control, and the target cold source execution parameters better reflect the state of the air conditioning system compared to a single predicted cold source execution parameter. Therefore, switching the air conditioning system to operate using the target cold source execution parameters results in more precise and stable cold source control.
[0164] In some alternative implementations, the target cold source execution parameters can be obtained in, but are not limited to, the following ways:
[0165] Based on the predicted cold source execution parameters within the second target time period, and combined with the value range corresponding to the second operation item among the multiple operation items, the target cold source execution parameters are obtained.
[0166] In practice, by combining the value range of the second operating item among multiple operating items, the execution information of the second operating item in the target cold source execution parameters is limited to the corresponding value range, making the air conditioning system operate more safely.
[0167] For each cold source type operation item, select the majority of cold source types from these multiple predicted cold source execution parameters;
[0168] For non-cold source type operation items, calculate the average value from these multiple predicted cold source execution parameters. If the average value is within the corresponding value range, select the average value; otherwise, select the upper or lower limit of the value range.
[0169] The embodiments of this application do not specifically limit the second operating item, which can be an operating item whose corresponding execution information is a specific value, such as the above-mentioned unit start-up quantity operating item, chilled water supply temperature operating item, and chilled water return temperature operating item, etc., which are non-cold source type operating items.
[0170] In some optional implementations, when the first operation item includes a cold source type operation item, switching the air conditioning system to operate with the target cold source execution parameters can be achieved in, but is not limited to, the following ways:
[0171] After the first cold source system is turned on for the target duration, the second cold source system is turned off; wherein, the first cold source corresponds to the target cold source execution parameters, and the second cold source corresponds to the current cold source execution parameters.
[0172] During implementation, when the execution parameters of the target cold source differ from the execution information corresponding to the cold source type operation item of the current cold source execution parameters, a cold source system switch is required. Since the cold source system needs to preheat for a period of time after being turned on to achieve a good cooling effect, a smooth switch of the cold source system is achieved by turning off the previously running second cold source system after the first cold source system has been turned on for the target duration, thus ensuring the cooling effect.
[0173] The following is a specific example to illustrate this:
[0174] The target cooling source execution parameters are as follows: the first cooling source system is an air-cooled system, with 3 units started, a chilled water supply temperature of 7.2 degrees Celsius, and a chilled water return temperature of 12.5 degrees Celsius; the second cooling source system in the current cooling source execution parameters is a water-cooled system. After the air-cooled system has been running for 30 minutes according to the target cooling source execution parameters, the water-cooled system will be shut down.
[0175] During implementation, when switching of the cold source system is involved, it is necessary to coordinate with the start-up and shutdown constraints and supporting operational constraints of the cold source system, such as the start-up and shutdown constraints specific to the chiller unit, and the supporting operational constraints of the chilled pump, cooling pump, and cooling tower.
[0176] In some optional implementations, if an abnormal situation occurs after the air conditioning system switches to the target cold source execution parameters, it can revert to the previous cold source execution parameters to ensure that the air conditioning system can operate normally.
[0177] like Figure 6 As shown, this application embodiment provides a cold source control device 600, applied to an air conditioning system, the device comprising:
[0178] The sampling module 601 is used to sample the current status parameters of the air conditioning system in real time; wherein, the status parameters include environmental parameters, operating parameters and performance parameters;
[0179] Action prediction module 602 is used to predict actions based on the current state parameters and obtain corresponding predicted cold source execution parameters;
[0180] The control module 603 is used to control the air conditioning system based on the predicted cold source execution parameters and the current cold source execution parameters of the air conditioning system; wherein the predicted cold source execution parameters and the current cold source execution parameters each include execution information corresponding to multiple operating items.
[0181] In some optional implementations, the action prediction module 602 is specifically used for:
[0182] The current state parameters are input into the policy network of the target model, and the policy network is used to predict the actions based on the current state parameters to obtain the predicted cold source execution parameters; wherein, the target model includes the policy network and the evaluation network;
[0183] It also includes a training module 604 for obtaining the target model in the following manner:
[0184] The target model is trained iteratively multiple times based on the sample dataset; wherein each sample dataset includes a first state parameter, a first cold source execution parameter, a second state parameter, and a reward value for the second state parameter; the second state parameter is the state parameter reached after executing the first cold source execution parameter on the first state parameter.
[0185] In some optional implementations, each iteration includes:
[0186] For any selected sample data, the first state parameter and the first cold source execution parameter of the sample data are input into the evaluation network to obtain the corresponding first evaluation value;
[0187] The evaluation network is tuned based on the first evaluation value and the target evaluation value; wherein the target evaluation value is determined based on the reward value in the sample data.
[0188] The first state parameters of the sample data are input into the strategy network to obtain the second cold source execution parameters;
[0189] The first state parameters and the second cold source execution parameters of the sample data are input into the evaluation network to obtain the corresponding second evaluation value, and the strategy network is adjusted based on the second evaluation value.
[0190] In some alternative implementations, the training module 604 is further configured to:
[0191] Obtain the reward value corresponding to each of the multiple state parameters sampled within the first target time period, and compare the obtained multiple reward values with the reward threshold respectively;
[0192] When the number of reward values less than the reward threshold exceeds a preset number, the target model is updated.
[0193] In some alternative implementations, the corresponding reward value is determined for any given state parameter in the following manner:
[0194] The weighted sum of the multiple rewards corresponding to the state parameters is used to obtain the corresponding reward value.
[0195] The environmental parameters include electricity consumption information; the operating parameters include multiple refrigeration parameters, unit operating time, and number of unit start-ups and shutdowns; the performance parameters include total cooling source power and energy efficiency ratio; and the multiple awards include some or all of the following:
[0196] An energy efficiency bonus is obtained by comparing the energy efficiency coefficient in the state parameters with the target energy efficiency coefficient.
[0197] The cost reward is obtained based on the electricity consumption information and total power of the cold source in the aforementioned status parameters;
[0198] A stability bonus is obtained based on the difference between the state parameter and at least one cooling parameter of the previous state parameter; the at least one cooling parameter includes some or all of the cooling capacity, chilled water supply temperature, and chilled water supply pressure.
[0199] The health reward is obtained based on the unit's operating time and the number of unit start-ups and shutdowns in the status parameters.
[0200] In some optional implementations, the control module 603 is specifically used for:
[0201] When there is no first operating item, the air conditioning system shall continue to operate according to the current cold source execution parameters;
[0202] When a first operating item is available, the air conditioning system is adjusted based on the predicted cold source execution parameters;
[0203] Wherein, the first running item is the running item among the plurality of running items where the first execution information and the second execution information satisfy the corresponding preset switching conditions, the first execution information corresponds to the predicted cold source execution parameters, and the second execution information corresponds to the current cold source execution parameters.
[0204] In some optional implementations, the control module 603 is specifically used for:
[0205] Based on the predicted cold source execution parameters during the second target time period, the target cold source execution parameters are obtained;
[0206] The air conditioning system is switched to operate with the parameters set by the target cold source.
[0207] In some optional implementations, the control module 603 is specifically used for:
[0208] Based on the predicted cold source execution parameters within the second target time period, and combined with the value range corresponding to the second operation item among the multiple operation items, the target cold source execution parameters are obtained.
[0209] In some optional implementations, when the first operating item includes a cold source type operating item, the control module 603 is specifically used for:
[0210] After the first cold source system is turned on for the target duration, the second cold source system is turned off; wherein, the first cold source corresponds to the target cold source execution parameters, and the second cold source corresponds to the current cold source execution parameters.
[0211] Since this device is the same as the device in the method of this application embodiment, and the principle of the device in solving the problem is similar to that of the method, the implementation of the device can be referred to the implementation of the method, and the repeated parts will not be described again.
[0212] Based on the same technical concept, this application also provides an air conditioning system 700, such as... Figure 7 As shown, it includes at least one controller 701 and a memory 702 connected to at least one controller. In this embodiment, the specific connection medium between the controller 701 and the memory 702 is not limited. Figure 7 Taking the connection between the controller 701 and the memory 702 via a bus 703 as an example. The bus can be divided into address bus, data bus, control bus, etc. For ease of illustration, Figure 7 The bus is represented by a single thick line, but this does not mean that there is only one bus or one type of bus.
[0213] The controller 701 is the control center of the air conditioning system. It can connect to various parts of the air conditioning system using various interfaces and lines. It performs data processing by running or executing instructions stored in the memory 702 and calling data stored in the memory 702. Optionally, the controller 701 may include one or more processing units. The controller 701 may integrate an application processor and a modem processor. The application processor mainly handles the operating system, user interface, and applications, while the modem processor mainly handles issuing instructions. It is understood that the modem processor may not be integrated into the controller 701. In some embodiments, the controller 701 and the memory 702 can be implemented on the same chip; in some embodiments, they can also be implemented on separate chips.
[0214] The controller 701 can be a general-purpose processor, such as a CPU, digital signal processor, application-specific integrated circuit (ASIC), field-programmable gate array or other programmable logic device, discrete gate or transistor logic device, or discrete hardware component, capable of implementing or executing the methods, steps, and logic block diagrams disclosed in the embodiments of this application. The general-purpose processor can be a microprocessor or any conventional processor. The steps of the methods disclosed in the embodiments of the cold source control method can be directly manifested as being executed by a hardware processor, or executed by a combination of hardware and software modules within the processor.
[0215] Memory 702, as a non-volatile computer-readable storage medium, can be used to store non-volatile software programs, non-volatile computer-executable programs, and modules. Memory 702 may include at least one type of storage medium, such as flash memory, hard disk, multimedia card, card-type memory, random access memory (RAM), static random access memory (SRAM), programmable read-only memory (PROM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), magnetic memory, magnetic disk, optical disk, etc. Memory 702 can be any other medium capable of carrying or storing desired program code in the form of instructions or data structures that can be accessed by a computer, but is not limited thereto. In the embodiments of this application, memory 702 can also be a circuit or any other device capable of implementing storage functions for storing program instructions and / or data.
[0216] In this embodiment, the memory 702 stores a computer program, which, when executed by the controller 701, causes the controller 701 to perform the following:
[0217] The current status parameters of the air conditioning system are sampled in real time; wherein, the status parameters include environmental parameters, operating parameters, and performance parameters.
[0218] Based on the current state parameters, action prediction is performed to obtain the corresponding predicted cold source execution parameters;
[0219] Based on the predicted cold source execution parameters and combined with the current cold source execution parameters of the air conditioning system, the air conditioning system is subjected to cold source control; wherein, the predicted cold source execution parameters and the current cold source execution parameters each include execution information corresponding to multiple operating items.
[0220] In some optional implementations, controller 701 specifically performs:
[0221] The current state parameters are input into the policy network of the target model, and the policy network is used to predict the actions based on the current state parameters to obtain the predicted cold source execution parameters; wherein, the target model includes the policy network and the evaluation network;
[0222] The target model is obtained through the following methods:
[0223] The target model is trained iteratively multiple times based on the sample dataset; wherein each sample dataset includes a first state parameter, a first cold source execution parameter, a second state parameter, and a reward value for the second state parameter; the second state parameter is the state parameter reached after executing the first cold source execution parameter on the first state parameter.
[0224] In some optional implementations, each iteration includes:
[0225] For any selected sample data, the first state parameter and the first cold source execution parameter of the sample data are input into the evaluation network to obtain the corresponding first evaluation value;
[0226] The evaluation network is tuned based on the first evaluation value and the target evaluation value; wherein the target evaluation value is determined based on the reward value in the sample data.
[0227] The first state parameters of the sample data are input into the strategy network to obtain the second cold source execution parameters;
[0228] The first state parameters and the second cold source execution parameters of the sample data are input into the evaluation network to obtain the corresponding second evaluation value, and the strategy network is adjusted based on the second evaluation value.
[0229] In some optional implementations, controller 701 also performs:
[0230] Obtain the reward value corresponding to each of the multiple state parameters sampled within the first target time period, and compare the obtained multiple reward values with the reward threshold respectively;
[0231] When the number of reward values less than the reward threshold exceeds a preset number, the target model is updated.
[0232] In some alternative implementations, the corresponding reward value is determined for any given state parameter in the following manner:
[0233] The weighted sum of the multiple rewards corresponding to the state parameters is used to obtain the corresponding reward value.
[0234] The environmental parameters include electricity consumption information; the operating parameters include multiple refrigeration parameters, unit operating time, and number of unit start-ups and shutdowns; the performance parameters include total cooling source power and energy efficiency ratio; and the multiple awards include some or all of the following:
[0235] An energy efficiency bonus is obtained by comparing the energy efficiency coefficient in the state parameters with the target energy efficiency coefficient.
[0236] The cost reward is obtained based on the electricity consumption information and total power of the cold source in the aforementioned status parameters;
[0237] A stability bonus is obtained based on the difference between the state parameter and at least one cooling parameter of the previous state parameter; the at least one cooling parameter includes some or all of the cooling capacity, chilled water supply temperature, and chilled water supply pressure.
[0238] The health reward is obtained based on the unit's operating time and the number of unit start-ups and shutdowns in the status parameters.
[0239] In some optional implementations, controller 701 specifically performs:
[0240] When there is no first operating item, the air conditioning system shall continue to operate according to the current cold source execution parameters;
[0241] When a first operating item is available, the air conditioning system is adjusted based on the predicted cold source execution parameters;
[0242] Wherein, the first running item is the running item among the plurality of running items where the first execution information and the second execution information satisfy the corresponding preset switching conditions, the first execution information corresponds to the predicted cold source execution parameters, and the second execution information corresponds to the current cold source execution parameters.
[0243] In some optional implementations, controller 701 specifically performs:
[0244] Based on the predicted cold source execution parameters during the second target time period, the target cold source execution parameters are obtained;
[0245] The air conditioning system is switched to operate with the parameters set by the target cold source.
[0246] In some optional implementations, controller 701 specifically performs:
[0247] Based on the predicted cold source execution parameters within the second target time period, and combined with the value range corresponding to the second operation item among the multiple operation items, the target cold source execution parameters are obtained.
[0248] In some optional implementations, when the first operating item includes a cold source type operating item, the controller 701 specifically executes:
[0249] After the first cold source system is turned on for the target duration, the second cold source system is turned off; wherein, the first cold source corresponds to the target cold source execution parameters, and the second cold source corresponds to the current cold source execution parameters.
[0250] Based on the same technical concept, embodiments of this application also provide a computer-readable storage medium storing a computer program executable by a processor, which, when run on the processor, causes the processor to perform the steps of the above-described cold source control method.
[0251] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product embodied on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.
[0252] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to this application. It should be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.
[0253] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.
[0254] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.
[0255] Although preferred embodiments of this application have been described, those skilled in the art, upon learning the basic inventive concept, can make other changes and modifications to these embodiments. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments as well as all changes and modifications falling within the scope of this application.
[0256] Obviously, those skilled in the art can make various modifications and variations to this application without departing from the spirit and scope of this application. Therefore, if such modifications and variations fall within the scope of the claims of this application and their equivalents, this application also intends to include such modifications and variations.
Claims
1. A cold source control method, characterized in that, Applied to air conditioning systems, the method includes: The current status parameters of the air conditioning system are sampled in real time; wherein, the status parameters include environmental parameters, operating parameters, and performance parameters. Based on the current state parameters, action prediction is performed to obtain the corresponding predicted cold source execution parameters; this includes: inputting the current state parameters into the policy network of the target model, and performing action prediction on the current state parameters through the policy network to obtain the predicted cold source execution parameters; wherein, the target model includes the policy network and an evaluation network; the target model is obtained by iteratively training the target model based on a sample dataset; wherein, each sample dataset includes a first state parameter, a first cold source execution parameter, a second state parameter, and a reward value for the second state parameter; the second state parameter is the state parameter reached after executing the first cold source execution parameter on the first state parameter; for any state parameter, the following method is used... Determine the corresponding reward value: A weighted sum of multiple rewards corresponding to the state parameters is performed to obtain the corresponding reward value. The environmental parameters include electricity consumption information; the operating parameters include multiple cooling parameters, unit operating time, and number of unit start-ups and shutdowns; the performance parameters include total cooling source power and energy efficiency coefficient (EEC). The multiple rewards include: an energy efficiency reward obtained by comparing the EEC in the state parameters with a target EEC; a cost reward obtained based on the electricity consumption information and total cooling source power in the state parameters; a stability reward obtained based on the difference between the state parameter and at least one cooling parameter from the previous state parameter; the at least one cooling parameter includes some or all of the cooling capacity, chilled water supply temperature, and chilled water supply pressure; and a health reward obtained based on the unit operating time and number of unit start-ups and shutdowns in the state parameters. Based on the predicted cold source execution parameters and combined with the current cold source execution parameters of the air conditioning system, the air conditioning system is subjected to cold source control; wherein, the predicted cold source execution parameters and the current cold source execution parameters each include execution information corresponding to multiple operating items.
2. The method as described in claim 1, characterized in that, Each iteration includes: For any selected sample data, the first state parameter and the first cold source execution parameter of the sample data are input into the evaluation network to obtain the corresponding first evaluation value; The evaluation network is tuned based on the first evaluation value and the target evaluation value; wherein the target evaluation value is determined based on the reward value in the sample data. The first state parameters of the sample data are input into the strategy network to obtain the second cold source execution parameters; The first state parameters and the second cold source execution parameters of the sample data are input into the evaluation network to obtain the corresponding second evaluation value, and the strategy network is adjusted based on the second evaluation value.
3. The method as described in claim 1, characterized in that, Also includes: Obtain the reward value corresponding to each of the multiple state parameters sampled within the first target time period, and compare the obtained multiple reward values with the reward threshold respectively; When the number of reward values less than the reward threshold exceeds a preset number, the target model is updated.
4. The method as described in claim 1, characterized in that, Based on the predicted cold source execution parameters and combined with the current cold source execution parameters of the air conditioning system, cold source control of the air conditioning system is performed, including: When there is no first operating item, the air conditioning system shall continue to operate according to the current cold source execution parameters; When a first operating item is available, the air conditioning system is adjusted based on the predicted cold source execution parameters; Wherein, the first running item is the running item among the plurality of running items where the first execution information and the second execution information satisfy the corresponding preset switching conditions, the first execution information corresponds to the predicted cold source execution parameters, and the second execution information corresponds to the current cold source execution parameters.
5. The method as described in claim 4, characterized in that, Adjusting the air conditioning system based on the predicted cold source execution parameters includes: Based on the predicted cold source execution parameters during the second target time period, the target cold source execution parameters are obtained; The air conditioning system is switched to operate with the parameters set by the target cold source.
6. The method as described in claim 5, characterized in that, Based on the predicted cold source execution parameters during the second target time period, the target cold source execution parameters are obtained, including: Based on the predicted cold source execution parameters within the second target time period, and combined with the value range corresponding to the second operation item among the multiple operation items, the target cold source execution parameters are obtained.
7. The method as described in claim 5, characterized in that, When the first operation item includes a cold source type operation item, the air conditioning system is switched to operate with the target cold source execution parameters, including: After the first cold source system is turned on for the target duration, the second cold source system is turned off; wherein, the first cold source corresponds to the target cold source execution parameters, and the second cold source corresponds to the current cold source execution parameters.
8. An air conditioning system, characterized in that, It includes at least one controller and at least one memory; wherein the memory stores a computer program that, when executed by the controller, causes the controller to perform the method as described in any one of claims 1 to 7.