Self-adaptive regulation and control method, device, equipment, medium and product of central air-conditioning system

Abstract

The invention discloses a self-adaptive regulation and control method, device and equipment for a central air-conditioning system, a medium and a product, and relates to the technical field of central air-conditioning system regulation and control. The method comprises the steps of firstly initializing a Rule function used for calculating a reward value of the air conditioner energy-saving rate, a Sigmoid function used for calculating a reward value of the air conditioner comfort degree, an experience playback pool, a Q network and a target network, then on one hand, selecting a to-be-executed action possibly having the maximum Q value based on the Q network and a greedy strategy, and issuing and executing the to-be-executed action; on the other hand, a comprehensive reward value obtained after action execution is calculated based on a Rule function and a Sigmoid function, the comprehensive reward value is arranged into experience sample data to be stored in an experience playback pool, and finally a certain amount of experience sample data is randomly selected from the experience playback pool to update a target network and a Q network. And a closed-loop control scheme capable of ensuring that the central air-conditioning system always operates in the optimal mode is formed, so that an ideal choice for solving the intelligent control problem of the central air-conditioning system can be formed.

Description

Technical Field

[0001] The present invention belongs to the technical field of central air-conditioning system control, and specifically relates to a central air-conditioning system adaptive control method, device, equipment, medium and product. Background Art

[0002] In today's rapidly developing technological era, central air-conditioning systems are an important part of building environmental control, and their intelligence and automation levels directly affect the energy efficiency and environmental comfort of buildings. However, traditional central air-conditioning systems usually adopt rule-based control strategies based on operating experience, which makes traditional central air-conditioning system control solutions often have problems such as slow response speed, high energy consumption, and inability to adapt to complex and changeable system operations, resulting in poor control effects. In addition, these solutions usually lack real-time feedback mechanisms, making it impossible to adjust operating parameters in time according to changes in system parameters.

[0003] With the advancement of artificial intelligence and machine learning technologies, more advanced solutions are being sought to overcome the above challenges. Deep learning technology and reinforcement learning technology have emerged. They can learn the optimal control strategy through interaction with the environment and each has its own advantages. Therefore, how to combine the advantages of deep learning technology and reinforcement learning technology to provide a new central air-conditioning system control solution that can not only process high-dimensional data but also learn the optimal control strategy from complex environmental interactions, so as to achieve multi-objective optimization and have real-time feedback capabilities at the same time, quickly adapt to environmental changes, and thus become an ideal choice for solving the problem of intelligent control of central air-conditioning systems. This is a topic that technicians in this field urgently need to study. Summary of the invention

[0004] The purpose of the present invention is to provide a central air-conditioning system adaptive control method, device, computer equipment, computer-readable storage medium and computer program product to solve the problems of slow response speed, high energy consumption, inability to adapt to complex and changeable system operation and lack of real-time feedback mechanism in existing central air-conditioning system control schemes.

[0005] In order to achieve the above object, the present invention adopts the following technical solutions:

[0006] In a first aspect, a method for adaptively controlling a central air conditioning system is provided, comprising:

[0007] Read the pre-configured action range, action rule constraints, network parameters, control end conditions, experience pool sample size N, minimum update experience pool sample size, training sample size M, action interval time, greed coefficient and target network update rate K, wherein the action range includes all control actions of all control devices in the central air-conditioning system, the network parameters include learning rate, training round cycle and / or number of hidden layers, N represents a positive integer, M represents a positive integer and is less than N, and K represents a positive integer;

[0008] Initialize a Rule function for calculating a reward value of an air-conditioning energy saving rate, a Sigmoid function for calculating a reward value of an air-conditioning comfort level, an experience replay pool with a capacity of N, and a counter with an initial count value of zero, wherein the air-conditioning energy saving rate refers to the energy saving rate of the central air-conditioning system per unit time, and the air-conditioning comfort level refers to the comfort level generated by the central air-conditioning system;

[0009] Initializing a Q network and a target network according to the network parameters, wherein the Q network and the target network are respectively used to calculate the Q values ​​of each action taken within the action range in the corresponding state after inputting the state value of the multidimensional state parameter of the central air-conditioning system, the parameters of the target network are the same as those of the Q network, and the multidimensional state parameter includes the index parameters of each operating device in the central air-conditioning system;

[0010] Obtaining the current state value of the multidimensional state parameter of the central air-conditioning system;

[0011] According to the current state value of the multi-dimensional state parameter of the central air-conditioning system, determining whether the control end condition is met;

[0012] If it is determined that the control end condition is not met, the current state value of the multidimensional state parameter of the central air-conditioning system is input into the Q network, and the Q value of each action taken in the current state is output;

[0013] Applying the greedy strategy and the greedy coefficient, selecting an action corresponding to the maximum Q value from the action range or randomly selecting an action as the action to be executed;

[0014] Determining whether the action to be performed complies with the action rule constraint;

[0015] If it is determined that the action to be performed complies with the action rule constraint, a control instruction for performing the action to be performed is issued to the control device in the central air-conditioning system, so that the control instruction is executed by the control device to complete the execution of the action to be performed;

[0016] After waiting for the action interval time, the next state value of the multidimensional state parameter of the central air-conditioning system is obtained, and a comprehensive reward value is calculated according to the next state value of the multidimensional state parameter of the central air-conditioning system, the Rule function and the Sigmoid function, wherein the comprehensive reward value refers to a reward value obtained by combining the reward value of the air-conditioning energy saving rate and the reward value of the air-conditioning comfort;

[0017] The four-tuple {s t ,a t ,r t ,s t+1} is stored as an experience sample data in the experience playback pool, wherein st represents the current state value of the multidimensional state parameter of the central air-conditioning system, a t Indicates the action to be performed, r t represents the comprehensive reward value, s t+1 Indicates the next state value of the multidimensional state parameter of the central air-conditioning system;

[0018] Determine whether the current sample size of the experience replay pool is greater than or equal to the minimum update experience pool sample size;

[0019] If it is determined that the current sample size is greater than or equal to the minimum updated experience pool sample size, then M experience sample data are randomly selected from the experience replay pool;

[0020] Apply the M empirical sample data to calculate the mean square error loss of the target network, and back-propagate the mean square error loss to update the parameters of the target network with gradients;

[0021] The count value of the counter is incremented by 1, and then it is determined whether the remainder obtained by dividing the count value by K is zero. If so, the parameters of the Q network are updated to the current parameters of the target network, and then the process returns to the step of obtaining the current state value of the multidimensional state parameters of the central air-conditioning system. Otherwise, the process directly returns to the step of obtaining the current state value of the multidimensional state parameters of the central air-conditioning system.

[0022] Based on the above invention content, a new scheme is provided that combines deep learning technology with reinforcement learning technology to perform adaptive control of a central air-conditioning system, that is, firstly initialize a Rule function for calculating a reward value of an air-conditioning energy saving rate, a Sigmoid function for calculating a reward value of an air-conditioning comfort level, an experience replay pool, a Q network, and a target network, and then, on the one hand, select an action to be executed that may have the maximum Q value based on the Q network and a greedy strategy and issue it for execution, and on the other hand, calculate the comprehensive reward value obtained after the action is executed based on the Rule function and the Sigmoid function, and organize it into an experience sample data and store it in the experience replay pool, and finally, randomly extract a certain amount of experience sample data from the experience replay pool to update the target network and the Q network, so as to form a closed-loop control scheme that can ensure that the central air-conditioning system always runs in the most optimized mode, so that it can process high-dimensional data, learn the optimal control strategy from complex environmental interactions, realize multi-objective optimization, and at the same time have real-time feedback capabilities, quickly adapt to environmental changes, and thus become an ideal choice for solving the problem of intelligent control of central air-conditioning systems, and facilitate practical application and promotion.

[0023] In a possible design, initializing the experience replay pool with a capacity of N includes:

[0024] Use a double-ended queue to build a sample data replay buffer with a sample capacity of N as an experience replay pool with a capacity of N.

[0025] In a possible design, after determining whether the regulation end condition is met, the method further includes:

[0026] If it is determined that the control end condition is met, then after waiting for the action interval time, the process returns to execute the step of obtaining the current state value of the multi-dimensional state parameter of the central air-conditioning system.

[0027] In a possible design, a greedy strategy and the greedy coefficient are applied to select an action corresponding to the maximum Q value from the action range or randomly select an action as the action to be executed, including:

[0028] A pure decimal is randomly generated, and it is determined whether the pure decimal is greater than 1-ε. If so, an action corresponding to the maximum Q value is selected from the action range as the action to be executed, otherwise an action is randomly selected from the action range as the action to be executed, where ε represents the greedy coefficient.

[0029] In a possible design, a comprehensive reward value is calculated according to the next state value of the multidimensional state parameter of the central air-conditioning system, the Rule function and the Sigmoid function, including:

[0030] According to the next state value of the multidimensional state parameter of the central air-conditioning system, a first reward value r1 corresponding to the air-conditioning energy saving rate is calculated by the Rule function, and a second reward value r2 corresponding to the air-conditioning comfort is calculated by the Sigmoid function;

[0031] According to the first reward value r1 and the second reward value r2, the comprehensive reward value r is calculated according to the following formula: t :

[0032] r t =r1×η1+r2×η2

[0033] In the formula, η1 represents a first preset weight coefficient corresponding to the air-conditioning energy saving rate, η2 represents a second preset weight coefficient corresponding to the air-conditioning comfort level, and η1+η2=1.

[0034] In one possible design, using the M empirical sample data to calculate the mean square error loss of the target network includes:

[0035] For each of the M empirical sample data, the next state value of the multidimensional state parameter of the central air-conditioning system is extracted from the corresponding sample data, and the next state value is input into the target network, and the Q value of each action taken in the next state is output, and then the corresponding target Q value is calculated according to the maximum Q value in the next state and the comprehensive reward value extracted from the corresponding sample data, combined with the control end condition and the preset discount factor, and the action to be executed is also extracted from the corresponding sample data, and the Q value corresponding to the action to be executed is used as the corresponding current Q value;

[0036] The difference between the current Q value and the target Q value of each empirical sample data is squared, and all the obtained square results are calculated by averaging to obtain the mean square error loss of the target network.

[0037] In a second aspect, an adaptive control device for a central air conditioning system is provided, comprising a configuration data reading unit, a first initialization unit, a second initialization unit, a current state acquisition unit, a first judgment unit, a Q value calculation unit, an action selection unit, a second judgment unit, an instruction issuing unit, a reward value calculation unit, a sample storage unit, a third judgment unit, a sample extraction unit, a network parameter updating unit, and a Q network updating unit;

[0038] The configuration data reading unit is used to read the pre-configured action range, action rule constraints, network parameters, control end conditions, experience pool sample size N, minimum update experience pool sample size, training sample size M, action interval time, greed coefficient and target network update rate K, wherein the action range includes all control actions of all control devices in the central air-conditioning system, the network parameters include learning rate, training round cycle and / or number of hidden layers, N represents a positive integer, M represents a positive integer and is less than N, and K represents a positive integer;

[0039] The first initialization unit is used to initialize a Rule function for calculating a reward value of an air-conditioning energy saving rate, a Sigmoid function for calculating a reward value of an air-conditioning comfort level, an experience replay pool with a capacity of N, and a counter with a count value initially set to zero, wherein the air-conditioning energy saving rate refers to the energy saving rate of the central air-conditioning system per unit time, and the air-conditioning comfort level refers to the comfort level generated by the central air-conditioning system;

[0040] The second initialization unit is used to initialize the Q network and the target network according to the network parameters, wherein the Q network and the target network are respectively used to calculate the Q values ​​of each action taken within the action range in the corresponding state after inputting the state value of the multidimensional state parameter of the central air-conditioning system, the parameters of the target network are the same as those of the Q network, and the multidimensional state parameter includes the index parameters of each operating device in the central air-conditioning system;

[0041] The current state acquisition unit is used to acquire the current state value of the multi-dimensional state parameter of the central air-conditioning system;

[0042] The first judgment unit is used to judge whether the control end condition is met according to the current state value of the multi-dimensional state parameter of the central air-conditioning system;

[0043] The Q value calculation unit is used to input the current state value of the multidimensional state parameter of the central air-conditioning system into the Q network if it is determined that the control end condition is not met, and output the Q value of each action taken in the current state;

[0044] The action selection unit is used to apply the greedy strategy and the greedy coefficient to select an action corresponding to the maximum Q value from the action range or randomly select an action as the action to be executed;

[0045] The second judgment unit is used to judge whether the action to be executed complies with the action rule constraint;

[0046] The instruction issuing unit is used to issue a control instruction for executing the action to be executed to the control device in the central air-conditioning system if it is determined that the action to be executed meets the action rule constraint, so that the control device executes the instruction to complete the execution of the action to be executed;

[0047] The reward value calculation unit is used to obtain the next state value of the multidimensional state parameter of the central air-conditioning system after waiting for the action interval time, and calculate the comprehensive reward value according to the next state value of the multidimensional state parameter of the central air-conditioning system, the Rule function and the Sigmoid function, wherein the comprehensive reward value refers to the reward value obtained by combining the reward value of the air-conditioning energy saving rate and the reward value of the air-conditioning comfort;

[0048] The sample storage unit is used to store the four-tuple {s t ,a t ,r t ,s t+1} is stored as an experience sample data in the experience replay pool, where s t represents the current state value of the multidimensional state parameter of the central air conditioning system, a t Indicates the action to be performed, r t represents the comprehensive reward value, s t+1 Indicates the next state value of the multidimensional state parameter of the central air-conditioning system;

[0049] The third judgment unit is used to judge whether the current sample size of the experience replay pool is greater than or equal to the minimum update experience pool sample size;

[0050] The sample extraction unit is used to randomly extract M experience sample data from the experience playback pool if it is determined that the current sample size is greater than or equal to the minimum update experience pool sample size;

[0051] The network parameter updating unit is used to calculate the mean square error loss of the target network by applying the M empirical sample data, and to back-propagate the mean square error loss to update the parameters of the target network by gradient;

[0052] The Q network updating unit is used to increase the count value of the counter by 1, and then determine whether the remainder obtained after dividing the count value by K is zero. If so, the parameters of the Q network are updated to the current parameters of the target network, and then the current state acquisition unit is started; otherwise, the current state acquisition unit is directly started.

[0053] In a third aspect, the present invention provides a computer device comprising a memory, a processor and a transceiver which are communicatively connected in sequence, wherein the memory is used to store a computer program, the transceiver is used to send and receive messages, and the processor is used to read the computer program to execute the adaptive control method of the central air-conditioning system as described in the first aspect or any possible design of the first aspect.

[0054] In a fourth aspect, the present invention provides a computer-readable storage medium having instructions stored thereon. When the instructions are executed on a computer, the adaptive control method of a central air-conditioning system as described in the first aspect or any possible design of the first aspect is executed.

[0055] In a fifth aspect, the present invention provides a computer program product, including a computer program or instructions, which, when executed by a computer, implements the adaptive control method of a central air-conditioning system as described in the first aspect or any possible design of the first aspect.

[0056] Beneficial effects of the above scheme:

[0057] (1) The present invention creatively provides a new scheme for adaptive control of central air-conditioning system by combining deep learning technology with reinforcement learning technology, namely, firstly initializing the Rule function for calculating the reward value of air-conditioning energy saving rate, the Sigmoid function for calculating the reward value of air-conditioning comfort, the experience replay pool, the Q network and the target network, then selecting the action to be executed with the maximum Q value based on the Q network and the greedy strategy and issuing it for execution, and calculating the comprehensive reward value obtained after the action is executed based on the Rule function and the Sigmoid function, and organizing it into an experience sample data and storing it in the experience replay pool, and finally randomly extracting a certain amount of experience sample data from the experience replay pool to update the target network and the Q network, so as to form a closed-loop control scheme that can ensure that the central air-conditioning system always runs in the optimal mode, so as to process high-dimensional data, learn the optimal control strategy from complex environmental interactions, realize multi-objective optimization, and have real-time feedback capability at the same time, quickly adapt to environmental changes, and thus become an ideal choice for solving the problem of intelligent control of central air-conditioning system, which is convenient for practical application and promotion;

[0058] (2) It can improve the intelligence level of central air-conditioning system operation and achieve more precise and efficient operation control, thereby optimizing energy efficiency, reducing operating costs, and improving user comfort experience;

[0059] (3) Improved energy efficiency: Using advanced intelligent control algorithms, the cloud platform can finely manage the operation of central air conditioners, accurately predict and respond to energy demand, and thus significantly reduce energy waste;

[0060] (4) Enhance indoor environmental comfort: By real-time monitoring of indoor and outdoor environmental parameters, the cloud platform can sensitively capture tiny environmental changes and dynamically adjust the operating parameters of the central air conditioner;

[0061] (5) Automatic adaptive control: The central air conditioning system adopts the latest adaptive control technology, which can automatically adjust the operation strategy according to real-time environmental data;

[0062] (6) Improve the reliability and stability of the platform: Through intelligent control, the system can maintain stable operation under various environmental conditions, reducing the risk of failure caused by human operating errors or equipment aging;

[0063] (7) Extending equipment life: Intelligent control algorithms can adjust the operating status of equipment according to actual load demand to avoid excessive operation, thereby extending the service life of the equipment and reducing maintenance costs. BRIEF DESCRIPTION OF THE DRAWINGS

[0064] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0065] Figure 1 A schematic flow chart of the adaptive control method for a central air-conditioning system provided in an embodiment of the present application.

[0066] Figure 2 A schematic diagram of the network structure of the central air-conditioning system provided in an embodiment of the present application.

[0067] Figure 3 This is an example diagram of the mapping relationship between the Rule function and the energy saving rate provided in the embodiment of the present application.

[0068] Figure 4 This is an example diagram of the mapping relationship between the Sigmoid function and the indoor temperature provided in an embodiment of the present application.

[0069] Figure 5 A schematic diagram of the structure of a computer device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0070] In order to more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the present invention will be briefly introduced below in combination with the drawings and the description of the embodiments or the prior art. Obviously, the following description of the structures of the drawings is only some embodiments of the present invention. For ordinary technicians in this field, other embodiments can be obtained based on these embodiments without creative work. It should be noted that the description of these embodiments is used to help understand the present invention, but does not constitute a limitation of the present invention.

[0071] It should be understood that although the terms first and second, etc. may be used herein to describe various objects, these objects should not be limited by these terms. These terms are only used to distinguish one object from another object. For example, a first object can be referred to as a second object, and similarly, a second object can be referred to as a first object without departing from the scope of the exemplary embodiments of the present invention.

[0072] It should be understood that the term "and / or" that may appear in this document is merely a description of the association relationship between associated objects, indicating that there may be three relationships. For example, A and / or B can indicate three situations: A exists alone, B exists alone, or A and B exist at the same time. For another example, A, B and / or C can indicate the existence of any one of A, B and C or any combination of them. The term " / and" that may appear in this document describes another type of association object relationship, indicating that there may be two relationships. For example, A / and B can indicate two situations: A exists alone or A and B exist at the same time. In addition, the character " / " that may appear in this document generally indicates that the previous and next associated objects are in an "or" relationship.

[0073] Example

[0074] like Figures 1 to 4 As shown, the adaptive control method of the central air conditioning system provided in the first aspect of this embodiment can be executed by, but is not limited to, a computer device with certain computing resources, such as a cloud server, a personal computer (PC, a multi-purpose computer with a size, price and performance suitable for personal use; desktops, laptops, small laptops, tablets and ultrabooks are all personal computers), smart phones, personal digital assistants (PDA) or wearable devices. Figure 1 As shown, the central air-conditioning system adaptive control method may include but is not limited to the following steps S1 to S15.

[0075] S1. Read the pre-configured action range, action rule constraints, network parameters, control end conditions, experience pool sample size N, minimum update experience pool sample size, training sample size M, action interval time, greed coefficient and target network update rate K, wherein the action range includes but is not limited to all control actions of all control devices in the central air-conditioning system, the network parameters include but are not limited to learning rate, training round cycle and / or number of hidden layers, etc., N represents a positive integer, M represents a positive integer and is less than N, and K represents a positive integer.

[0076] In step S1, the specific structure of the central air conditioning system is as follows: Figure 2As shown, it includes but is not limited to terminal devices such as refrigeration hosts, cooling water pumps and / or cooling towers (which may include control devices such as air-cooled temperature regulators, and may also include sensor devices such as temperature and humidity sensors, etc.), LoRa smart terminals that connect different terminal devices through RS485 bus wired communication, LoRa gateways that wirelessly connect the LoRa (Long Range Radio) smart terminals, and cloud (i.e., cloud servers) that communicate with the LoRa gateways, etc., so as to achieve the following HVAC edge-cloud collaboration: In the field of HVAC, through the collection of IoT devices, a large amount of data from the central air-conditioning system can be collected and analyzed to achieve precise control; the edge node (i.e., the LoRa smart terminal) is mainly responsible for the collection of field / terminal data, and performs preliminary processing and analysis of the data according to rules or data models, and then uploads the processing results and related data to the cloud; the cloud provides storage, analysis and value mining of massive data, and sends instructions to the LoRa gateway for device control to achieve rapid control of related equipment; ultimately, the relevant data flows in an orderly manner between the edge and the cloud to form a complete data flow path. The action range can specifically use different action values ​​to uniquely represent different actions; for example, the following three action values ​​can be pre-configured for the air-cooled temperature regulator: -1, 0 and 1 (that is, the action range is expressed as [-1,0,1]), where -1 represents a temperature lowering action, 0 represents a temperature non-adjustment action, and 1 represents a temperature raising action. The action rule constraint is used to ensure that the final executed control action operates within a safe boundary to avoid safety risk events after the action; for example, for the air-cooled temperature regulator, the temperature after control can be pre-configured to belong to the temperature safety range [20,50] degrees Celsius as the corresponding action rule constraint. The network parameters are used to initially construct the subsequent Q network and the target network, where the Q network is a prior art concept in reinforcement learning technology and will not be described in detail here. The control end condition is used as a criterion for determining whether the central air-conditioning system needs to be controlled. Specifically, it may include multiple groups of conditional expressions configured based on the multi-dimensional state parameters of the central air-conditioning system, wherein each group of conditional expressions may be specifically composed of conditional types (such as "and" and "or", etc.), comparison conditions (such as "greater than", "less than or equal to", "equal to", "greater than or equal to" and "greater than", etc.) and comparison types (such as "compared with the dynamic value of the device" and "compared with a constant", etc.), and when all or any of the conditional expressions are satisfied, it may be determined that the control end condition is satisfied so as to end this control, otherwise, start this control. The experience pool sample size N is used to initialize the subsequent experience playback pool, which needs to be much larger than the minimum update experience pool sample size.In addition, the aforementioned action scope, action rule constraints, network parameters, control end conditions, experience pool sample size N, minimum update experience pool sample size, training sample size M, action interval time, greediness coefficient and target network update rate K can be specifically configured by experienced personnel on the cloud.

[0077] S2. Initialize a Rule function for calculating the reward value of the air-conditioning energy saving rate, a Sigmoid function for calculating the reward value of the air-conditioning comfort level, an experience replay pool with a capacity of N, and a counter with an initial count value of zero, wherein the air-conditioning energy saving rate refers to the energy saving rate of the central air-conditioning system per unit time, and the air-conditioning comfort level refers to the comfort level generated by the central air-conditioning system.

[0078] In step S2, the Rule function is used to map the energy saving rate of the central air conditioning system within a unit time (for example, 1 minute or 5 minutes) to the interval [0, 1]. The specific mapping relationship can be as follows: Figure 3 Specifically, the energy saving rate γ of the central air-conditioning system per unit time can be calculated according to the following formula:

[0079]

[0080] In the formula, E2 represents the power consumption per unit time of the central air conditioning system after the last operation, and E1 represents the power consumption per unit time of the central air conditioning system after the current operation. In addition, the above two power consumptions can be obtained by the electric meter equipment in the central air conditioning system.

[0081] In step S2, the Sigmoid function is used to map the comfort measurement variable such as indoor temperature generated by the central air conditioning system to the interval [0,1]. The specific mapping relationship can be as follows: Figure 4 As shown: Specifically, the shape of the Sigmoid function can be obtained by selecting three representative temperature points for fitting, that is, the function Medium parameters: a, b and c (these parameters can adjust the shape of the function to make it better fit the actual data and more accurately reflect the relationship between indoor temperature and comfort).

[0082] In the step S2, preferably, initializing the experience replay pool with a capacity of N includes but is not limited to: using a double-ended queue to construct a sample data replay buffer with a sample capacity of N as the experience replay pool with a capacity of N. The double-ended queue refers to a queue that runs at both ends for queue entry and queue exit operations, and the logical structure of its elements is still a linear structure; the double-ended queue is used because it can realize automatic capacity limitation, so that the random.sample function can be used to achieve random sampling without directly accessing elements randomly from the queue.

[0083] S3. Initialize the Q network and the target network according to the network parameters, wherein the Q network and the target network are respectively used to calculate the Q values ​​of each action taken within the action range under the corresponding state after inputting the state value of the multidimensional state parameters of the central air-conditioning system, the parameters of the target network are the same as those of the Q network, and the multidimensional state parameters include but are not limited to the indicator parameters of each operating equipment in the central air-conditioning system.

[0084] In step S3, the specific initialization process of the Q network can be obtained by referring to the conventional derivation of existing reinforcement learning technology, which will not be repeated here. The index parameters specifically include but are not limited to outdoor dry-bulb humidity (which corresponds to heat pump equipment), outdoor wet-bulb temperature (which corresponds to water machine equipment), chilled water supply temperature / supply-return temperature difference (which corresponds to characteristic main pipe equipment), chilled water flow, chilled pump frequency, number of chilled pumps and equipment operation time segmentation (such as working hours and off-duty hours, etc.), etc. They can be obtained by conventional screening and configuration by experienced people according to actual scenarios. In addition, if there is a Model model file obtained and saved through the previous training, the Model model file can be directly read to initialize the Q network and the target network (at this time, the Q network and the target network will no longer be initialized according to the network parameters).

[0085] S4. Obtain the current state value of the multi-dimensional state parameter of the central air-conditioning system.

[0086] In the step S4, the current state value of the multi-dimensional state parameter of the central air-conditioning system may be obtained specifically but not limited to preliminary processing and analysis by an edge node.

[0087] S5. According to the current state value of the multi-dimensional state parameter of the central air-conditioning system, determine whether the control end condition is met.

[0088] S6. If it is determined that the control termination condition is not met, the current state value of the multidimensional state parameter of the central air-conditioning system is input into the Q network, and the Q value of each action taken in the current state is output.

[0089] After step S5, specifically, the method further includes but is not limited to: if it is determined that the control end condition is met, then after waiting for the action interval time, return to execute the step for obtaining the current state value of the multidimensional state parameter of the central air-conditioning system (i.e., step S4).

[0090] S7. Apply the greedy strategy and the greedy coefficient to select an action corresponding to the maximum Q value from the action range or randomly select an action as the action to be executed.

[0091] In the step S7, specifically, the greedy strategy and the greedy coefficient are applied to select an action corresponding to the maximum Q value from the action range or randomly select an action as the action to be executed, including but not limited to: randomly generating a pure decimal, and determining whether the pure decimal is greater than 1-ε, if so, selecting an action corresponding to the maximum Q value from the action range as the action to be executed, otherwise randomly selecting an action from the action range as the action to be executed, wherein ε represents the greedy coefficient.

[0092] S8. Determine whether the action to be executed complies with the action rule constraints.

[0093] In step S8, for example, the action to be executed is a temperature lowering action for the air-cooled temperature regulator. If the temperature after the temperature lowering action is executed is lower than the lower limit value of the temperature safety range interval [20,50] degrees Celsius (i.e., 20 degrees Celsius), it is determined that the action rule constraints are not met. If the temperature after the temperature lowering action is executed still belongs to the temperature safety range interval [20,50] degrees Celsius, it is determined that the action rule constraints are met.

[0094] S9. If it is determined that the action to be executed complies with the action rule constraint, a control instruction for executing the action to be executed is sent to the control device in the central air-conditioning system, so that the control instruction is executed by the control device to complete the execution of the action to be executed.

[0095] In step S9, since the action to be executed may correspond to the maximum Q value, it can represent the best control strategy in the current state, so as to generate the corresponding control instruction. In addition, the issuance and execution process of the control instruction can be conventionally derived based on existing communication technology and control technology, which will not be repeated here.

[0096] S10. After waiting for the action interval time, obtain the next state value of the multidimensional state parameter of the central air-conditioning system, and calculate the comprehensive reward value according to the next state value of the multidimensional state parameter of the central air-conditioning system, the Rule function and the Sigmoid function, wherein the comprehensive reward value refers to the reward value obtained by combining the reward value of the air-conditioning energy saving rate and the reward value of the air-conditioning comfort.

[0097] In the step S10, the action interval is used to ensure that after the action to be executed is executed, each device in the central air-conditioning system is in a stable state, thereby ensuring that the next state value of the multidimensional state parameter of the central air-conditioning system is stable and reliable. Specifically, according to the next state value of the multidimensional state parameter of the central air-conditioning system, the Rule function and the Sigmoid function, a comprehensive reward value is calculated, including but not limited to the following steps: first, according to the next state value of the multidimensional state parameter of the central air-conditioning system, the Rule function is used to calculate the first reward value r1 corresponding to the air-conditioning energy saving rate (for example, according to the power consumption per unit time in the next state value of the multidimensional state parameter of the central air-conditioning system, etc.), and the Sigmoid function is used to calculate the second reward value r2 corresponding to the air-conditioning comfort (for example, according to the indoor temperature in the next state value of the multidimensional state parameter of the central air-conditioning system, etc.); then, according to the first reward value r1 and the second reward value r2, the comprehensive reward value r is calculated according to the following formula t :

[0098] r t =r1×η1+r2×η2

[0099] In the formula, η1 represents the first preset weight coefficient corresponding to the air conditioning energy saving rate, η2 represents the second preset weight coefficient corresponding to the air conditioning comfort, and η1+η2=1. In addition, the next state value of the multidimensional state parameter of the central air conditioning system can also be obtained by preliminary processing and analysis by the edge node, but not limited to, and after step S8, if it is determined that the action to be executed does not meet the action rule constraint, the action to be executed is not executed, and after waiting for the action interval time, the next state value of the multidimensional state parameter of the central air conditioning system is obtained, and the comprehensive reward value is directly equal to zero.

[0100] S11. The four-tuple {s t ,a t ,r t ,s t+1} is stored as an experience sample data in the experience playback pool, wherein st represents the current state value of the multidimensional state parameter of the central air-conditioning system, a t Indicates the action to be performed, r t represents the comprehensive reward value, s t+1 Represents the next state value of the multidimensional state parameter of the central air-conditioning system.

[0101] In the step S11, the greedy coefficient may also be routinely and dynamically updated according to a pre-configured minimum greedy coefficient and a greedy coefficient attenuation factor.

[0102] S12. Determine whether the current sample size of the experience replay pool is greater than or equal to the minimum update experience pool sample size.

[0103] S13. If it is determined that the current sample size is greater than or equal to the minimum updated experience pool sample size, M experience sample data are randomly selected from the experience replay pool.

[0104] S14. Apply the M empirical sample data to calculate the mean square error loss of the target network, and back-propagate the mean square error loss to update the parameters of the target network with gradients.

[0105] In the step S14, the specific process of back-propagating the mean square error loss to update the parameters of the target network by gradient can be conventionally derived based on the existing network parameter update technology, and will not be repeated here. Specifically, the mean square error loss of the target network is calculated using the M empirical sample data, including but not limited to the following steps S141 to S142.

[0106] S141. For each of the M empirical sample data, the next state value of the multidimensional state parameter of the central air-conditioning system is extracted from the corresponding sample data, and the next state value is input into the target network, and the Q value of each action taken in the next state is output, and then the corresponding target Q value is calculated based on the maximum Q value in the next state and the comprehensive reward value extracted from the corresponding sample data, combined with the control end condition and the preset discount factor, and the action to be executed is also extracted from the corresponding sample data, and the Q value corresponding to the action to be executed is used as the corresponding current Q value.

[0107] In the step S141, the specific calculation process of the target Q value can be derived by referring to the conventional prior art.

[0108] S142. Square the difference between the current Q value and the target Q value of each empirical sample data, and calculate all the obtained square results by averaging to obtain the mean square error loss of the target network.

[0109] S15. Increment the count value of the counter by 1, and then determine whether the remainder obtained by dividing the count value by K is zero. If so, update the parameters of the Q network to the current parameters of the target network, and then return to execute the step for obtaining the current state value of the multidimensional state parameter of the central air-conditioning system (i.e., execute the step S4; of course, the current state value of the multidimensional state parameter of the central air-conditioning system can also be updated to the next state value of the multidimensional state parameter of the central air-conditioning system, and then execute step S5); otherwise, directly return to execute the step for obtaining the current state value of the multidimensional state parameter of the central air-conditioning system (i.e., execute the step S4; of course, the current state value of the multidimensional state parameter of the central air-conditioning system can also be updated to the next state value of the multidimensional state parameter of the central air-conditioning system, and then execute step S5).

[0110] After step S12, if it is determined that the current sample size is greater than or equal to the minimum updated experience pool sample size, then the process directly returns to the step of obtaining the current state value of the multidimensional state parameter of the central air-conditioning system (i.e., executing step S4; of course, the current state value of the multidimensional state parameter of the central air-conditioning system can also be updated to the next state value of the multidimensional state parameter of the central air-conditioning system, and then executing step S5).

[0111] Therefore, based on the adaptive control method of the central air-conditioning system described in the aforementioned steps S1 to S15, a new scheme for adaptive control of the central air-conditioning system by combining deep learning technology with reinforcement learning technology is provided, that is, the Rule function for calculating the reward value of the air-conditioning energy saving rate, the Sigmoid function for calculating the reward value of the air-conditioning comfort, the experience replay pool, the Q network and the target network are first initialized, and then on the one hand, the to-be-executed action with the maximum Q value is selected based on the Q network and the greedy strategy and issued for execution, and on the other hand, the comprehensive reward value obtained after the action is executed is calculated based on the Rule function and the Sigmoid function, and is organized into an experience sample data and stored in the experience replay pool, and finally a certain amount of experience sample data is randomly extracted from the experience replay pool to update the target network and the Q network, forming a closed-loop control scheme that can ensure that the central air-conditioning system always operates in the optimal mode, so that it can not only process high-dimensional data, but also learn the optimal control strategy from complex environmental interactions, realize multi-objective optimization, and at the same time have real-time feedback capabilities, quickly adapt to environmental changes, and thus become an ideal choice for solving the intelligent control problem of the central air-conditioning system, which is convenient for practical application and promotion.

[0112] The second aspect of the present embodiment provides a virtual device for implementing the adaptive control method of the central air-conditioning system described in the first aspect, including a configuration data reading unit, a first initialization unit, a second initialization unit, a current state acquisition unit, a first judgment unit, a Q value calculation unit, an action selection unit, a second judgment unit, an instruction issuing unit, a reward value calculation unit, a sample storage unit, a third judgment unit, a sample extraction unit, a network parameter updating unit and a Q network updating unit;

[0113] The configuration data reading unit is used to read the pre-configured action range, action rule constraints, network parameters, control end conditions, experience pool sample size N, minimum update experience pool sample size, training sample size M, action interval time, greed coefficient and target network update rate K, wherein the action range includes all control actions of all control devices in the central air-conditioning system, the network parameters include learning rate, training round cycle and / or number of hidden layers, N represents a positive integer, M represents a positive integer and is less than N, and K represents a positive integer;

[0114] The first initialization unit is used to initialize a Rule function for calculating a reward value of an air-conditioning energy saving rate, a Sigmoid function for calculating a reward value of an air-conditioning comfort level, an experience replay pool with a capacity of N, and a counter with a count value initially set to zero, wherein the air-conditioning energy saving rate refers to the energy saving rate of the central air-conditioning system per unit time, and the air-conditioning comfort level refers to the comfort level generated by the central air-conditioning system;

[0115] The second initialization unit is used to initialize the Q network and the target network according to the network parameters, wherein the Q network and the target network are respectively used to calculate the Q values ​​of each action taken within the action range in the corresponding state after inputting the state value of the multidimensional state parameter of the central air-conditioning system, the parameters of the target network are the same as those of the Q network, and the multidimensional state parameter includes the index parameters of each operating device in the central air-conditioning system;

[0116] The current state acquisition unit is used to acquire the current state value of the multi-dimensional state parameter of the central air-conditioning system;

[0117] The first judgment unit is used to judge whether the control end condition is met according to the current state value of the multi-dimensional state parameter of the central air-conditioning system;

[0118] The Q value calculation unit is used to input the current state value of the multidimensional state parameter of the central air-conditioning system into the Q network if it is determined that the control end condition is not met, and output the Q value of each action taken in the current state;

[0119] The action selection unit is used to apply the greedy strategy and the greedy coefficient to select an action corresponding to the maximum Q value from the action range or randomly select an action as the action to be executed;

[0120] The second judgment unit is used to judge whether the action to be executed complies with the action rule constraint;

[0121] The instruction issuing unit is used to issue a control instruction for executing the action to be executed to the control device in the central air-conditioning system if it is determined that the action to be executed meets the action rule constraint, so that the control device executes the instruction to complete the execution of the action to be executed;

[0122] The reward value calculation unit is used to obtain the next state value of the multidimensional state parameter of the central air-conditioning system after waiting for the action interval time, and calculate the comprehensive reward value according to the next state value of the multidimensional state parameter of the central air-conditioning system, the Rule function and the Sigmoid function, wherein the comprehensive reward value refers to the reward value obtained by combining the reward value of the air-conditioning energy saving rate and the reward value of the air-conditioning comfort;

[0123] The sample storage unit is used to store the four-tuple {s t ,a t ,r t ,s t+1} is stored as an experience sample data in the experience replay pool, where s trepresents the current state value of the multidimensional state parameter of the central air conditioning system, a t Indicates the action to be performed, r t represents the comprehensive reward value, s t+1 Indicates the next state value of the multidimensional state parameter of the central air-conditioning system;

[0124] The third judgment unit is used to judge whether the current sample size of the experience replay pool is greater than or equal to the minimum update experience pool sample size;

[0125] The sample extraction unit is used to randomly extract M experience sample data from the experience playback pool if it is determined that the current sample size is greater than or equal to the minimum update experience pool sample size;

[0126] The network parameter updating unit is used to calculate the mean square error loss of the target network by applying the M empirical sample data, and to back-propagate the mean square error loss to update the parameters of the target network by gradient;

[0127] The Q network updating unit is used to increase the count value of the counter by 1, and then determine whether the remainder obtained after dividing the count value by K is zero. If so, the parameters of the Q network are updated to the current parameters of the target network, and then the current state acquisition unit is started; otherwise, the current state acquisition unit is directly started.

[0128] The working process, working details and technical effects of the aforementioned device provided in the second aspect of this embodiment can be referred to the adaptive control method of the central air-conditioning system described in the first aspect, and will not be repeated here.

[0129] like Figure 5As shown, the third aspect of this embodiment provides a computer device for executing the adaptive control method of the central air-conditioning system as described in the first aspect, including a memory, a processor and a transceiver connected in sequence, wherein the memory is used to store a computer program, the transceiver is used to send and receive messages, and the processor is used to read the computer program to execute the adaptive control method of the central air-conditioning system as described in the first aspect. For example, the memory may include, but is not limited to, a random access memory (Random-Access Memory, RAM), a read-only memory (Read-Only Memory, ROM), a flash memory (Flash Memory), a first-in-first-out memory (First Input First Output, FIFO) and / or a first-in-last-out memory (First Input Last Output, FILO), etc.; the processor may include, but is not limited to, a microprocessor of the STM32F105 series. In addition, the computer device may also include, but is not limited to, a power module, a display screen and other necessary components.

[0130] The working process, working details and technical effects of the aforementioned computer equipment provided in the third aspect of this embodiment can be found in the adaptive control method of the central air-conditioning system described in the first aspect, and will not be described in detail here.

[0131] The fourth aspect of this embodiment provides a computer-readable storage medium storing instructions including the method for adaptive control of a central air-conditioning system as described in the first aspect, that is, the computer-readable storage medium stores instructions, and when the instructions are executed on a computer, the method for adaptive control of a central air-conditioning system as described in the first aspect is executed. The computer-readable storage medium refers to a carrier for storing data, which may include but is not limited to computer-readable storage media such as floppy disks, optical disks, hard disks, flash memories, USB flash drives, and / or memory sticks, and the computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices.

[0132] The working process, working details and technical effects of the aforementioned computer-readable storage medium provided in the fourth aspect of this embodiment can be referred to the adaptive control method of the central air-conditioning system described in the first aspect, and will not be repeated here.

[0133] In a fifth aspect of this embodiment, a computer program product is provided, including a computer program or an instruction, wherein when the computer program or the instruction is executed by a computer, the method for adaptively controlling a central air conditioning system as described in the first aspect is implemented. The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device.

[0134] Finally, it should be noted that the above description is only a preferred embodiment of the present invention and is not intended to limit the protection scope of the present invention. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. A method for adaptively controlling a central air conditioning system, characterized in that: include: Read the pre-configured action range, action rule constraints, network parameters, control end conditions, experience pool sample size N, minimum update experience pool sample size, training sample size M, action interval time, greed coefficient and target network update rate K, wherein the action range includes all control actions of all control devices in the central air-conditioning system, the network parameters include learning rate, training round cycle and / or number of hidden layers, N represents a positive integer, M represents a positive integer and is less than N, and K represents a positive integer; Initialize a Rule function for calculating a reward value of an air-conditioning energy saving rate, a Sigmoid function for calculating a reward value of an air-conditioning comfort level, an experience replay pool with a capacity of N, and a counter with an initial count value of zero, wherein the air-conditioning energy saving rate refers to the energy saving rate of the central air-conditioning system per unit time, and the air-conditioning comfort level refers to the comfort level generated by the central air-conditioning system; Initializing a Q network and a target network according to the network parameters, wherein the Q network and the target network are respectively used to calculate the Q values ​​of each action taken within the action range in the corresponding state after inputting the state value of the multidimensional state parameter of the central air-conditioning system, the parameters of the target network are the same as those of the Q network, and the multidimensional state parameter includes the index parameters of each operating device in the central air-conditioning system; Obtaining the current state value of the multidimensional state parameter of the central air-conditioning system; According to the current state value of the multi-dimensional state parameter of the central air-conditioning system, determining whether the control end condition is met; If it is determined that the control end condition is not met, the current state value of the multidimensional state parameter of the central air-conditioning system is input into the Q network, and the Q value of each action taken in the current state is output; Applying the greedy strategy and the greedy coefficient, selecting an action corresponding to the maximum Q value from the action range or randomly selecting an action as the action to be executed; Determining whether the action to be performed complies with the action rule constraint; If it is determined that the action to be performed complies with the action rule constraint, a control instruction for performing the action to be performed is issued to the control device in the central air-conditioning system, so that the control instruction is executed by the control device to complete the execution of the action to be performed; After waiting for the action interval time, the next state value of the multidimensional state parameter of the central air-conditioning system is obtained, and a comprehensive reward value is calculated according to the next state value of the multidimensional state parameter of the central air-conditioning system, the Rule function and the Sigmoid function, wherein the comprehensive reward value refers to a reward value obtained by combining the reward value of the air-conditioning energy saving rate and the reward value of the air-conditioning comfort; The four-tuple {s t ,a t ,r t ,s t+1 } is stored as an experience sample data in the experience replay pool, where s t represents the current state value of the multidimensional state parameter of the central air conditioning system, a t Indicates the action to be performed, r t represents the comprehensive reward value, s t+1 Indicates the next state value of the multidimensional state parameter of the central air-conditioning system; Determine whether the current sample size of the experience replay pool is greater than or equal to the minimum update experience pool sample size; If it is determined that the current sample size is greater than or equal to the minimum updated experience pool sample size, then M experience sample data are randomly selected from the experience replay pool; Apply the M empirical sample data to calculate the mean square error loss of the target network, and back-propagate the mean square error loss to update the parameters of the target network with gradients; The count value of the counter is incremented by 1, and then it is determined whether the remainder obtained by dividing the count value by K is zero. If so, the parameters of the Q network are updated to the current parameters of the target network, and then the process returns to the step of obtaining the current state value of the multidimensional state parameters of the central air-conditioning system. Otherwise, the process directly returns to the step of obtaining the current state value of the multidimensional state parameters of the central air-conditioning system.

2. The adaptive control method of central air conditioning system according to claim 1, characterized in that: Initialize the experience replay pool with a capacity of N, including: Use a double-ended queue to build a sample data replay buffer with a sample capacity of N as an experience replay pool with a capacity of N.

3. The adaptive control method of central air conditioning system according to claim 1, characterized in that: After determining whether the control end condition is met, the method further includes: If it is determined that the control end condition is met, then after waiting for the action interval time, the process returns to execute the step of obtaining the current state value of the multi-dimensional state parameter of the central air-conditioning system.

4. The adaptive control method for a central air conditioning system according to claim 1, characterized in that: Applying the greedy strategy and the greedy coefficient, selecting an action corresponding to the maximum Q value from the action range or randomly selecting an action as the action to be executed, including: A pure decimal is randomly generated, and it is determined whether the pure decimal is greater than 1-ε. If so, an action corresponding to the maximum Q value is selected from the action range as the action to be executed, otherwise an action is randomly selected from the action range as the action to be executed, where ε represents the greedy coefficient.

5. The adaptive control method of central air conditioning system according to claim 1, characterized in that: The comprehensive reward value is calculated according to the next state value of the multidimensional state parameter of the central air-conditioning system, the Rule function and the Sigmoid function, including: According to the next state value of the multidimensional state parameter of the central air-conditioning system, a first reward value r1 corresponding to the air-conditioning energy saving rate is calculated by the Rule function, and a second reward value r2 corresponding to the air-conditioning comfort is calculated by the Sigmoid function; According to the first reward value r1 and the second reward value r2, the comprehensive reward value r is calculated according to the following formula: t : r t =r1×η1+r2×η2 In the formula, η1 represents a first preset weight coefficient corresponding to the air-conditioning energy saving rate, η2 represents a second preset weight coefficient corresponding to the air-conditioning comfort level, and η1+η2=1.

6. The adaptive control method of central air conditioning system according to claim 1, characterized in that: Applying the M empirical sample data to calculate the mean square error loss of the target network includes: For each of the M empirical sample data, the next state value of the multidimensional state parameter of the central air-conditioning system is extracted from the corresponding sample data, and the next state value is input into the target network, and the Q value of each action taken in the next state is output, and then the corresponding target Q value is calculated according to the maximum Q value in the next state and the comprehensive reward value extracted from the corresponding sample data, combined with the control end condition and the preset discount factor, and the action to be executed is also extracted from the corresponding sample data, and the Q value corresponding to the action to be executed is used as the corresponding current Q value; The difference between the current Q value and the target Q value of each empirical sample data is squared, and all the obtained square results are calculated by averaging to obtain the mean square error loss of the target network.

7. An adaptive control device for a central air conditioning system, characterized in that: It includes a configuration data reading unit, a first initialization unit, a second initialization unit, a current state acquisition unit, a first judgment unit, a Q value calculation unit, an action selection unit, a second judgment unit, an instruction issuing unit, a reward value calculation unit, a sample storage unit, a third judgment unit, a sample extraction unit, a network parameter updating unit and a Q network updating unit; The configuration data reading unit is used to read the pre-configured action range, action rule constraints, network parameters, control end conditions, experience pool sample size N, minimum update experience pool sample size, training sample size M, action interval time, greed coefficient and target network update rate K, wherein the action range includes all control actions of all control devices in the central air-conditioning system, the network parameters include learning rate, training round cycle and / or number of hidden layers, N represents a positive integer, M represents a positive integer and is less than N, and K represents a positive integer; The first initialization unit is used to initialize a Rule function for calculating a reward value of an air-conditioning energy saving rate, a Sigmoid function for calculating a reward value of an air-conditioning comfort level, an experience replay pool with a capacity of N, and a counter with a count value initially set to zero, wherein the air-conditioning energy saving rate refers to the energy saving rate of the central air-conditioning system per unit time, and the air-conditioning comfort level refers to the comfort level generated by the central air-conditioning system; The second initialization unit is used to initialize the Q network and the target network according to the network parameters, wherein the Q network and the target network are respectively used to calculate the Q values ​​of each action taken within the action range in the corresponding state after inputting the state value of the multidimensional state parameter of the central air-conditioning system, the parameters of the target network are the same as those of the Q network, and the multidimensional state parameter includes the index parameters of each operating device in the central air-conditioning system; The current state acquisition unit is used to acquire the current state value of the multi-dimensional state parameter of the central air-conditioning system; The first judgment unit is used to judge whether the control end condition is met according to the current state value of the multi-dimensional state parameter of the central air-conditioning system; The Q value calculation unit is used to input the current state value of the multidimensional state parameter of the central air-conditioning system into the Q network if it is determined that the control end condition is not met, and output the Q value of each action taken in the current state; The action selection unit is used to apply the greedy strategy and the greedy coefficient to select an action corresponding to the maximum Q value from the action range or randomly select an action as the action to be executed; The second judgment unit is used to judge whether the action to be executed complies with the action rule constraint; The instruction issuing unit is used to issue a control instruction for executing the action to be executed to the control device in the central air-conditioning system if it is determined that the action to be executed meets the action rule constraint, so that the control device executes the instruction to complete the execution of the action to be executed; The reward value calculation unit is used to obtain the next state value of the multidimensional state parameter of the central air-conditioning system after waiting for the action interval time, and calculate the comprehensive reward value according to the next state value of the multidimensional state parameter of the central air-conditioning system, the Rule function and the Sigmoid function, wherein the comprehensive reward value refers to the reward value obtained by combining the reward value of the air-conditioning energy saving rate and the reward value of the air-conditioning comfort; The sample storage unit is used to store the four-tuple {s t ,a t ,r t ,s t+1 } is stored as an experience sample data in the experience replay pool, where s t represents the current state value of the multidimensional state parameter of the central air conditioning system, a t Indicates the action to be performed, r t represents the comprehensive reward value, s t+1 Indicates the next state value of the multidimensional state parameter of the central air-conditioning system; The third judgment unit is used to judge whether the current sample size of the experience replay pool is greater than or equal to the minimum update experience pool sample size; The sample extraction unit is used to randomly extract M experience sample data from the experience playback pool if it is determined that the current sample size is greater than or equal to the minimum update experience pool sample size; The network parameter updating unit is used to calculate the mean square error loss of the target network by applying the M empirical sample data, and to back-propagate the mean square error loss to update the parameters of the target network by gradient; The Q network updating unit is used to increase the count value of the counter by 1, and then determine whether the remainder obtained after dividing the count value by K is zero. If so, the parameters of the Q network are updated to the current parameters of the target network, and then the current state acquisition unit is started; otherwise, the current state acquisition unit is directly started.

8. A computer device, characterized in that: It includes a memory, a processor and a transceiver which are communicatively connected in sequence, wherein the memory is used to store a computer program, the transceiver is used to send and receive messages, and the processor is used to read the computer program to execute the adaptive control method of the central air-conditioning system as described in any one of claims 1 to 6.

9. A computer-readable storage medium, characterized in that The computer-readable storage medium stores instructions, and when the instructions are executed on the computer, the adaptive control method of the central air-conditioning system as described in any one of claims 1 to 6 is executed.

10. A computer program product comprising a computer program or instructions, characterized in that When the computer program or the instruction is executed by a computer, the adaptive control method for a central air-conditioning system according to any one of claims 1 to 6 is implemented.

Images