Central air conditioning control model training method, central air conditioning control method and device

Abstract

The present application discloses a training method for a central air conditioning control model, a central air conditioning control method and a device, and relates to the technical field of central air conditioning. The method of the present application comprises: establishing a neural network model according to the business rules adopted in the actual environment, and training the neural network model to obtain a simulation environment model; establishing a reinforcement learning model according to a preset algorithm; and training the reinforcement learning model based on the simulation environment model until a preset training stop condition is reached to obtain a central air conditioning control model.

Description

Technical Field

[0001] The present application relates to the technical field of central air conditioning, and in particular to a training method for a central air conditioning control model, and a central air conditioning control method and device. Background Art

[0002] Central air conditioning is the infrastructure of large public buildings to meet people's demand for indoor temperature. At the same time, the energy consumption of central air conditioning has received more and more attention. Taking data centers as an example, according to the "Green Data Center White Paper (2019)", the electricity consumption of global data centers will account for 3%-4% of global electricity consumption in 2020, of which a considerable part of the electricity is used for cooling and heating of central air conditioning. Therefore, how to reduce the energy consumption of central air conditioning while maintaining indoor temperature is an urgent problem to be solved. Summary of the invention

[0003] In view of this, the present application provides a training method for a central air-conditioning control model, a central air-conditioning control method and a device, the main purpose of which is to reduce the energy consumption of the central air-conditioning while maintaining the indoor temperature.

[0004] In order to solve the above problems, this application mainly provides the following technical solutions:

[0005] In a first aspect, the present application provides a training method for a central air conditioning control model, the method being applied to a target terminal device, the method comprising:

[0006] Establishing a neural network model according to the business rules adopted in the actual environment, and training the neural network model to obtain a simulation environment model;

[0007] Build a reinforcement learning model based on a preset algorithm;

[0008] The reinforcement learning model is trained based on the simulation environment model until a preset training stop condition is reached to obtain a central air conditioning control model.

[0009] Optionally, the reinforcement learning model is trained based on the simulation environment model until a preset training stop condition is reached to obtain a central air conditioning control model, including:

[0010] Step S1: using the simulation environment model to generate a first training state vector, wherein the first training state vector is generated by the simulation environment model according to the state of the simulation environment model at a first moment, and the first training state vector includes: a switch state of an uncontrolled device, a target external parameter, an operating parameter of an uncontrolled device, and a target internal parameter, and the first moment is the current moment;

[0011] Step S2: inputting the first training state vector into the reinforcement learning model so that the reinforcement learning model outputs a training action vector, wherein the training action vector includes: a heat pump outlet water temperature setting value, a circulating water pump frequency setting value, and an AHU water valve opening setting value;

[0012] Step S3: inputting the training motion vector into the simulation environment model so that the simulation environment model controls the simulation central air conditioner according to the training motion vector;

[0013] Step S4: acquiring a plurality of indoor temperature setting values, a plurality of indoor temperature actual values, a plurality of system total powers and a preset weight coefficient from the simulation environment model according to a preset duration and a preset rule, and substituting the plurality of indoor temperature setting values, the plurality of indoor temperature actual values, the plurality of system total powers and the preset weight coefficient into a preset reward function formula to calculate a training reward value;

[0014] Step S5: using the simulation environment model to generate a second training state vector, and storing the first training state vector, the training action vector, the training reward value and the second training state vector as training samples in a training sample set, wherein the second training state vector is generated by the simulation environment model according to the state of the simulation environment model at a second moment, and the second training state vector includes: a switch state of an uncontrolled device, target external parameters, operating parameters of an uncontrolled device and target internal parameters, and the second moment is separated from the first moment by the preset time length;

[0015] Step S6: randomly obtaining N training samples from the training sample set, and using the N training samples to optimize and adjust the model parameters in the reinforcement learning model, where N is a positive integer;

[0016] Step S7: Determine whether the preset training stop condition is reached; if reached, confirm the reinforcement learning model as the central air-conditioning control model; if not reached, return to step S1.

[0017] In a second aspect, the present application provides a central air conditioning control method for controlling a target central air conditioning based on a central air conditioning control model obtained by the training method described in the first aspect.

[0018] Optionally, when controlling the target central air conditioner, a first actual state vector is generated according to the state of the actual environment at a first moment, wherein the first actual state vector includes: a switch state of an uncontrolled device, a target external parameter, an operating parameter of an uncontrolled device, and a target internal parameter, and the first moment is the current moment;

[0019] Inputting the first actual state vector into the central air conditioning control model so that the central air conditioning control model outputs an actual action vector, wherein the actual action vector includes: a heat pump outlet water temperature setting value, a circulating water pump frequency setting value, and an AHU water valve opening setting value;

[0020] The target central air conditioner is controlled using the actual motion vector.

[0021] Optionally, after using the actual motion vector to control the target central air conditioner, the method further includes:

[0022] Acquire the working mode, indoor temperature setting value and indoor temperature actual value corresponding to the target central air conditioner from the actual environment;

[0023] When the working mode is cooling mode, if the actual indoor temperature value is greater than the indoor temperature setting value, the AHU water valve opening, the circulating water pump frequency and the heat pump outlet water temperature are adjusted in sequence so that the adjusted actual indoor temperature value is less than or equal to the indoor temperature setting value;

[0024] When the working mode is the heating mode, if the actual value of the indoor temperature is less than the set value of the indoor temperature, the AHU water valve opening, the circulating water pump frequency and the heat pump outlet water temperature are adjusted in sequence so that the adjusted actual value of the indoor temperature is greater than or equal to the set value of the indoor temperature.

[0025] Optionally, after using the actual motion vector to control the target central air conditioner, the method further includes:

[0026] Acquire multiple indoor temperature setting values, multiple indoor temperature actual values, multiple system total powers, and preset weight coefficients from the actual environment according to a preset duration and a preset rule, and substitute the multiple indoor temperature setting values, multiple indoor temperature actual values, multiple system total powers, and the preset weight coefficients into a preset reward function formula to calculate an actual reward value;

[0027] Generate a second actual state vector according to the state of the actual environment at the second moment, wherein the second actual state vector includes: the switch state of the uncontrolled device, the target external parameter, the operating parameter of the uncontrolled device, and the target internal parameter, and the second moment is separated from the first moment by the preset time length;

[0028] storing the first actual state vector, the actual action vector, the actual reward value, and the second actual state vector as training samples in a training sample set;

[0029] M training samples are randomly obtained from the training sample set, and the M training samples are used to optimize and adjust the model parameters in the central air-conditioning control model, where M is a positive integer.

[0030] In a third aspect, the present application provides a training device for a central air conditioning control model, the device comprising:

[0031] A first establishing unit is used to establish a neural network model according to the business rules adopted in the actual environment;

[0032] A first training unit, used for training the neural network model to obtain a simulation environment model;

[0033] A second establishing unit is used to establish a reinforcement learning model according to a preset algorithm;

[0034] The second training unit is used to train the reinforcement learning model based on the simulation environment model until a preset training stop condition is reached to obtain a central air conditioning control model.

[0035] Optionally, the second training unit includes:

[0036] A first generating module is configured to generate a first training state vector using the simulation environment model, wherein the first training state vector is generated by the simulation environment model according to the state of the simulation environment model at a first moment, and the first training state vector includes: a switch state of an uncontrolled device, a target external parameter, an operating parameter of an uncontrolled device, and a target internal parameter, and the first moment is a current moment;

[0037] A first input module, used to input the first training state vector into the reinforcement learning model, so that the reinforcement learning model outputs a training action vector, wherein the training action vector includes: a heat pump outlet water temperature setting value, a circulating water pump frequency setting value, and an AHU water valve opening setting value;

[0038] A second input module, used for inputting the training motion vector into the simulation environment model, so that the simulation environment model controls the simulation central air conditioner according to the training motion vector;

[0039] an acquisition module, used for acquiring a plurality of indoor temperature setting values, a plurality of indoor temperature actual values, a plurality of system total powers and a preset weight coefficient from the simulation environment model according to a preset duration and a preset rule, and substituting the plurality of indoor temperature setting values, the plurality of indoor temperature actual values, the plurality of system total powers and the preset weight coefficient into a preset reward function formula to calculate a training reward value;

[0040] A second generating module is used to generate a second training state vector using the simulation environment model, and store the first training state vector, the training action vector, the training reward value and the second training state vector as training samples in a training sample set, wherein the second training state vector is generated by the simulation environment model according to the state of the simulation environment model at a second moment, and the second training state vector includes: a switch state of an uncontrolled device, a target external parameter, an operating parameter of an uncontrolled device and a target internal parameter, and the second moment is separated from the first moment by the preset time length;

[0041] An adjustment module, used to randomly obtain N training samples from the training sample set, and use the N training samples to optimize and adjust the model parameters in the reinforcement learning model, where N is a positive integer;

[0042] A judgment module is used to judge whether the preset training stop condition is reached; when the preset training stop condition is reached, the reinforcement learning model is confirmed as the central air-conditioning control model; when the preset training stop condition is not reached, the first generation module, the first input module, the second input module, the acquisition module, the second generation module and the adjustment module are controlled to train the reinforcement learning model.

[0043] In a fourth aspect, the present application provides a target central air-conditioning control system, which is used to control the target central air-conditioning based on the central air-conditioning control model obtained by the training method described in the first aspect.

[0044] Optionally, the target central air-conditioning control system includes:

[0045] A first generating unit, configured to generate a first actual state vector according to a state of an actual environment at a first moment, wherein the first actual state vector includes: a switch state of an uncontrolled device, a target external parameter, an operating parameter of an uncontrolled device, and a target internal parameter, and the first moment is a current moment;

[0046] a first input unit, used to input the first actual state vector into the central air conditioning control model, so that the central air conditioning control model outputs an actual action vector, wherein the actual action vector includes: a heat pump outlet water temperature setting value, a circulating water pump frequency setting value, and an AHU water valve opening setting value;

[0047] A control unit is used to control a target central air conditioner using the actual motion vector.

[0048] Optionally, the target central air-conditioning control system further includes:

[0049] A first acquisition unit is used to acquire the working mode, indoor temperature setting value and indoor temperature actual value corresponding to the target central air conditioner from the actual environment;

[0050] A first adjustment unit is used for, when the working mode is a cooling mode, if the actual value of the indoor temperature is greater than the set value of the indoor temperature, adjusting the AHU water valve opening, the circulating water pump frequency and the heat pump outlet water temperature in sequence, so that the adjusted actual value of the indoor temperature is less than or equal to the set value of the indoor temperature;

[0051] The first adjustment unit is also used to adjust the AHU water valve opening, the circulating water pump frequency and the heat pump outlet water temperature in sequence when the working mode is the heating mode, if the actual value of the indoor temperature is less than the indoor temperature setting value, so that the adjusted actual value of the indoor temperature is greater than or equal to the indoor temperature setting value.

[0052] Optionally, the target central air-conditioning control system further includes:

[0053] A second acquisition unit is used to acquire a plurality of indoor temperature setting values, a plurality of indoor temperature actual values, a plurality of system total powers and a preset weight coefficient from the actual environment according to a preset time length and a preset rule, and substitute the plurality of indoor temperature setting values, the plurality of indoor temperature actual values, the plurality of system total powers and the preset weight coefficient into a preset reward function formula to calculate an actual reward value;

[0054] A second generating unit is configured to generate a second actual state vector according to the state of the actual environment at a second moment, wherein the second actual state vector includes: a switch state of an uncontrolled device, a target external parameter, an operating parameter of an uncontrolled device, and a target internal parameter, and the second moment is separated from the first moment by the preset time length;

[0055] a storage unit, configured to store the first actual state vector, the actual action vector, the actual reward value, and the second actual state vector as training samples in a training sample set;

[0056] The second adjustment unit is used to randomly obtain M training samples from the training sample set, and use the M training samples to optimize and adjust the model parameters in the central air-conditioning control model, where M is a positive integer.

[0057] In order to achieve the above-mentioned purpose, the fifth aspect of the present application provides a storage medium, which includes a stored program, wherein when the program is running, the device where the storage medium is located is controlled to execute the training method of the central air-conditioning control model described in the first aspect above; or execute the central air-conditioning control method described in the second aspect above.

[0058] In order to achieve the above-mentioned purpose, the sixth aspect of the present application provides an electronic device, which includes at least one processor, and at least one memory and a bus connected to the processor; wherein the processor and the memory communicate with each other through the bus; the processor is used to call the program instructions in the memory to execute the training method of the central air-conditioning control model described in the first aspect above; or execute the central air-conditioning control method described in the second aspect above.

[0059] By means of the above technical solution, the technical solution provided by this application has at least the following advantages:

[0060] The present application provides a training method for a central air conditioning control model, a central air conditioning control method and a device. The present application can establish a neural network model according to the business rules adopted by the target terminal device in the actual environment, and train the neural network model to obtain a simulation environment model, and then establish a reinforcement learning model according to a preset algorithm, and train the reinforcement learning model based on the simulation environment model until the preset training stop condition is reached, thereby obtaining a central air conditioning control model. After the central air conditioning control model is migrated to the target central air conditioning control system, the target central air conditioning control system can control the target central air conditioning based on the central air conditioning control model. Since the training process of the simulation environment model is simple, training the reinforcement learning model based on the simulation environment model can enable the reinforcement learning model to quickly reach the training stop condition, so the central air conditioning control model can be quickly trained; and the central air conditioning control system can achieve precise control of the central air conditioning based on the central air conditioning control model, thereby maintaining the indoor temperature while reducing the energy consumption of the central air conditioning.

[0061] The above description is only an overview of the technical solution of the present application. In order to more clearly understand the technical means of the present application, it can be implemented in accordance with the contents of the specification. In order to make the above and other purposes, features and advantages of the present application more obvious and easy to understand, the specific implementation methods of the present application are listed below. BRIEF DESCRIPTION OF THE DRAWINGS

[0062] Various other advantages and benefits will become apparent to those of ordinary skill in the art by reading the detailed description of the preferred embodiments below. The accompanying drawings are only for the purpose of illustrating the preferred embodiments and are not to be considered as limiting the present application. Also, the same reference symbols are used throughout the accompanying drawings to represent the same components. In the accompanying drawings:

[0063] Figure 1 A flow chart of a training method for a first central air conditioning control model provided in an embodiment of the present application is shown;

[0064] Figure 2A flow chart of a training method for a second central air conditioning control model provided in an embodiment of the present application is shown;

[0065] Figure 3 A flow chart of a third central air conditioning control method provided in an embodiment of the present application is shown;

[0066] Figure 4 A flow chart of a fourth central air conditioning control method provided in an embodiment of the present application is shown;

[0067] Figure 5 A block diagram showing the composition of a first central air conditioning control model training device provided in an embodiment of the present application is shown;

[0068] Figure 6 A block diagram showing the composition of a second central air conditioning control model training device provided in an embodiment of the present application is shown;

[0069] Figure 7 The following is a block diagram showing a first target central air-conditioning control system provided by an embodiment of the present application;

[0070] Figure 8 A block diagram of a second target central air-conditioning control system provided in an embodiment of the present application is shown;

[0071] Fig. 9 A structural block diagram of a device provided in an embodiment of the present application is shown. DETAILED DESCRIPTION

[0072] The exemplary embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. Although the exemplary embodiments of the present disclosure are shown in the accompanying drawings, it should be understood that the present disclosure can be implemented in various forms and should not be limited by the embodiments set forth herein. On the contrary, these embodiments are provided to enable a more thorough understanding of the present disclosure and to fully convey the scope of the present disclosure to those skilled in the art.

[0073] It should be noted that, unless otherwise specified, the technical terms or scientific terms used in this application should have the common meanings understood by technicians in the field to which this application belongs.

[0074] In a first aspect, an embodiment of the present application provides a training method for a central air conditioning control model, which is applied to a target terminal device, such as Figure 1 As shown, the method includes:

[0075] 101. Establish a neural network model based on the business rules adopted in the actual environment, and train the neural network model to obtain a simulation environment model.

[0076] The target terminal device may be, but is not limited to, a computer, a server, etc. The business rules adopted in the actual environment may include, but are not limited to, the following: (1) the lower the cooling temperature is, the higher the power of the central air conditioner is; (2) the higher the cooling temperature is, the lower the power of the central air conditioner is; (3) the higher the heating temperature is, the higher the power of the central air conditioner is; (4) the lower the heating temperature is, the lower the power of the central air conditioner is; (5) the higher the frequency of the circulating water pump is, the higher the flow rate of the circulating water is; (4) the lower the frequency of the circulating water pump is, the lower the flow rate of the circulating water is, etc.; the established neural network model may be any type of neural network model, and the embodiments of the present application do not specifically limit this.

[0077] In an embodiment of the present application, the target terminal device first needs to establish a neural network model according to the business rules adopted in the actual environment, and train the neural network model to obtain a simulation environment model, wherein the simulation environment model obtained by training has the same business rules as the actual environment.

[0078] Specifically, in this step, the neural network model can be trained in the following manner to obtain a simulation environment model: generate a training sample set and a test sample set according to preset training data, use the training samples in the training sample set to train the neural network model, and use the test samples in the test sample set to test the neural network model; when the accuracy of the neural network model reaches the preset accuracy requirement, the neural network model is determined as a simulation environment model; other existing neural network model training methods can also be used to train the neural network model, and the embodiments of the present application do not specifically limit this.

[0079] 102. Establish a reinforcement learning model according to a preset algorithm, and train the reinforcement learning model based on a simulation environment model until a preset training stop condition is reached to obtain a central air conditioning control model.

[0080] Among them, the preset algorithm is specifically: TD3 algorithm; the preset training stop condition may include but is not limited to: the loss function of the reinforcement learning model converges, when the indoor temperature setting value remains unchanged, the values ​​of each control parameter contained in the action vector output by the reinforcement learning model tend to be stable, that is, the value of each control parameter floats within a specified range or remains unchanged.

[0081] In an embodiment of the present application, after the target terminal device obtains the simulation environment model through training, it can establish a reinforcement learning model according to a preset algorithm (i.e., the TD3 algorithm), and train the reinforcement learning model based on the simulation environment model until the preset training stop condition is reached, thereby obtaining a central air-conditioning control model. After migrating the central air-conditioning control model to the target central air-conditioning control system, the target central air-conditioning control system can control the target central air-conditioning based on the central air-conditioning control model.

[0082] The embodiment of the present application provides a training method for a central air-conditioning control model. The embodiment of the present application can establish a neural network model according to the business rules adopted by the target terminal device in the actual environment, and train the neural network model to obtain the simulation environment model, and then establish a reinforcement learning model according to a preset algorithm, and train the reinforcement learning model based on the simulation environment model until the preset training stop condition is reached, thereby obtaining the central air-conditioning control model. After the central air-conditioning control model is migrated to the target central air-conditioning control system, the target central air-conditioning control system can control the target central air-conditioning based on the central air-conditioning control model. Since the training process of the simulation environment model is simple, training the reinforcement learning model based on the simulation environment model can enable the reinforcement learning model to quickly reach the training stop condition, so the central air-conditioning control model can be quickly trained; and the central air-conditioning control system can realize accurate control of the central air-conditioning based on the central air-conditioning control model, thereby reducing the energy consumption of the central air-conditioning while maintaining the indoor temperature.

[0083] In a second aspect, the present application provides a training method for a central air conditioning control model, which is applied to a target terminal device, such as Figure 2 As shown, the method includes:

[0084] 201. A neural network model is established according to the business rules adopted in the actual environment, and the neural network model is trained to obtain a simulation environment model.

[0085] Among them, regarding step 201, establishing a neural network model according to the business rules adopted in the actual environment, and training the neural network model to obtain a simulation environment model, reference can be made to Figure 1 The description of the corresponding parts will not be repeated here in the embodiments of the present application.

[0086] 202. Establish a reinforcement learning model based on the preset algorithm.

[0087] Regarding step 202, establishing a reinforcement learning model according to a preset algorithm, please refer to Figure 1 The description of the corresponding parts will not be repeated here in the embodiments of the present application.

[0088] 203. The reinforcement learning model is trained based on the simulation environment model until a preset training stop condition is reached to obtain a central air conditioning control model.

[0089] In the embodiment of the present application, after the target terminal device establishes the reinforcement learning model according to the preset algorithm, it can train the reinforcement learning model based on the simulation environment model until the preset training stop condition is reached, thereby obtaining the central air conditioning control model. The following will describe in detail how the target terminal device trains the reinforcement learning model based on the simulation environment model until the preset training stop condition is reached to obtain the central air conditioning control model.

[0090] (1) Generate a first training state vector using a simulation environment model.

[0091] The first training state vector is generated by the simulation environment model according to the state of the simulation environment model at a first moment; the first moment is the current moment.

[0092] Among them, the first training state vector includes: the switch state of the uncontrolled device, the target external parameters, the operating parameters of the uncontrolled device and the target internal parameters; among them, the uncontrolled device is the device that cannot be controlled by the central air-conditioning control system, that is, the device that needs to be manually controlled to open and close, which may include but is not limited to: some air outlet switches, circulating water pumps, etc.; the target external parameters are some parameters outside the central air-conditioning control system, which may be but are not limited to: outdoor temperature, outdoor humidity, outdoor carbon dioxide concentration, etc.; the uncontrolled device is the device that can be controlled by the central air-conditioning control system but is not controlled; the target internal parameters are the internal parameters in the central air-conditioning control system that are affected by the control parameters, and the control parameters are multiple parameters contained in the training action vector output by the reinforcement learning model.

[0093] (2) Inputting the first training state vector into the reinforcement learning model so that the reinforcement learning model outputs a training action vector.

[0094] The training action vectors include: the set value of the heat pump outlet water temperature, the set value of the circulating water pump frequency, and the set value of the AHU water valve opening.

[0095] (3) The training motion vector is input into the simulation environment model so that the simulation environment model controls the simulated central air conditioner according to the training motion vector.

[0096] In an embodiment of the present application, after the target terminal device inputs the training action vector output by the reinforcement learning model into the simulation environment model, the simulation environment model will use the control parameters contained in the training action vector (i.e., the heat pump outlet water temperature setting value, the circulating water pump frequency setting value, and the AHU water valve opening setting value) to control the simulated central air conditioning.

[0097] (4) According to the preset time and preset rules, multiple indoor temperature setting values, multiple indoor temperature actual values, multiple system total powers and preset weight coefficients are obtained from the simulation environment model, and the multiple indoor temperature setting values, multiple indoor temperature actual values, multiple system total powers and preset weight coefficients are substituted into the preset reward function formula to calculate the training reward value.

[0098] The preset duration may be, but is not limited to, 1S, 10S, 50S, 100S, etc.; the preset rule may be, but is not limited to, obtaining data every X seconds, where X is a positive integer less than the value corresponding to the preset duration.

[0099] In an embodiment of the present application, within a preset time after the simulation environment model uses the control parameters contained in the action vector to control the simulated central air conditioner, the target terminal device obtains multiple indoor temperature setting values, multiple indoor temperature actual values, multiple system total powers and preset weight coefficients from the simulation environment model according to preset rules, and substitutes the multiple indoor temperature setting values, multiple indoor temperature actual values, multiple system total powers and preset weight coefficients into the preset reward function formula to calculate the training reward value, wherein the preset reward function formula is as follows:

[0100]

[0101] Among them, R is the training reward value, T set_t+i is the indoor temperature setting value obtained at a time interval of i seconds from the current time, T t+i is the actual indoor temperature value obtained at a time interval of i seconds from the current time, P t+i is the total system power obtained at a time interval of i seconds from the current time, λ is the preset weighting coefficient, n is the number of multiple indoor temperature setting values, and i is a natural number multiple of X.

[0102] (5) Generate a second training state vector using the simulation environment model, and store the first training state vector, the training action vector, the training reward value, and the second training state vector as training samples in a training sample set.

[0103] Among them, the second training state vector is generated by the simulation environment model according to the state of the simulation environment model at the second moment, and the second training state vector includes: the switch state of the uncontrolled device, the target external parameters, the operating parameters of the uncontrolled device and the target internal parameters; the second moment is separated from the first moment by a preset time length.

[0104] (6) Randomly obtain N training samples from the training sample set, and use the N training samples to optimize and adjust the model parameters in the reinforcement learning model.

[0105] Wherein, N is a positive integer.

[0106] Specifically, in this step, the target terminal device adopts the parameter adjustment mechanism corresponding to the TD3 algorithm and uses N training samples to optimize and adjust the model parameters in the reinforcement learning model.

[0107] It should be noted that in actual application, when the number of training samples stored in the training sample set is less than or equal to N, the target terminal device obtains all training samples stored in the training sample set, and uses these training samples to optimize and adjust the model parameters in the reinforcement learning model.

[0108] It should be noted that in actual application, the training sample set is an ordered set with a fixed capacity. When the number of training samples stored in the training sample set reaches a threshold and newly generated training samples need to be stored in the training sample set, the target terminal device needs to delete the training samples first stored in the training sample set, and then store the newly generated training samples in the training sample set.

[0109] (7) Determine whether the preset training stop condition is reached; if the preset training stop condition is reached, the trained reinforcement learning model is confirmed as the central air conditioning control model; if the preset training stop condition is not reached, the target terminal device needs to use the method recorded in the above steps (1)-(6) to train the reinforcement learning model again until the preset training stop condition is reached.

[0110] In a third aspect, an embodiment of the present application provides a central air conditioning control method, which is used to control a target central air conditioning based on a central air conditioning control model obtained by the training method described in the first aspect or the second aspect, such as Figure 3 As shown, the method includes:

[0111] 301. Generate a first actual state vector according to a state of an actual environment at a first moment.

[0112] Among them, the first moment is the current moment. Among them, the first actual state vector includes: the switch state of the uncontrolled device, the target external parameters, the operating parameters of the uncontrolled device and the target internal parameters; among them, the uncontrolled device is the device that the target central air-conditioning control system cannot control, that is, the device that needs to be manually controlled to open and close, which can include but is not limited to: some air outlet switches, circulating water pumps, etc.; the target external parameters are some parameters outside the target central air-conditioning control system, which can be but are not limited to: outdoor temperature, outdoor humidity, outdoor carbon dioxide concentration, etc.; the uncontrolled device is the device that the target central air-conditioning control system can control but does not control; the target internal parameters are the internal parameters in the target central air-conditioning control system that are affected by the control parameters, and the control parameters are multiple parameters contained in the actual action vector output by the central air-conditioning control model.

[0113] In an embodiment of the present application, after the central air-conditioning control model is migrated to the target central air-conditioning control system, the target central air-conditioning control system can control the target central air-conditioning based on the central air-conditioning control model; when the target central air-conditioning control system controls the target central air-conditioning based on the central air-conditioning control model, it first needs to generate a first actual state vector based on the state of the actual environment at the first moment (i.e., the current moment).

[0114] 302. Input the first actual state vector into the central air conditioning control model so that the central air conditioning control model outputs an actual action vector.

[0115] The actual action vectors include: the set value of the heat pump outlet water temperature, the set value of the circulating water pump frequency, and the set value of the AHU water valve opening.

[0116] In an embodiment of the present application, after the target central air-conditioning control system generates a first actual state vector based on the state of the actual environment at the first moment (i.e., the current moment), it is necessary to input the first actual state vector into the central air-conditioning control model; after the target central air-conditioning control system inputs the first actual state vector into the central air-conditioning control model, the central air-conditioning control model will output an actual action vector.

[0117] 303. Use the actual motion vector to control the target central air conditioner.

[0118] In an embodiment of the present application, after the central air conditioning control model outputs the actual action vector, the target central air conditioning control system can use the control parameters contained in the actual action vector (i.e., the heat pump outlet water temperature setting value, the circulating water pump frequency setting value and the AHU water valve opening setting value) to control the target central air conditioning.

[0119] In a fourth aspect, an embodiment of the present application provides a central air conditioning control method, which is used to control a target central air conditioning based on a central air conditioning control model obtained by the training method described in the first aspect or the second aspect, such as Figure 4 As shown, the method includes:

[0120] 401. Generate a first actual state vector according to a state of an actual environment at a first moment.

[0121] Regarding step 401, generating a first actual state vector according to the state of the actual environment at the first moment, reference may be made to Figure 1 The description of the corresponding parts will not be repeated here in the embodiments of the present application.

[0122] 402. Input the first actual state vector into the central air conditioning control model so that the central air conditioning control model outputs an actual action vector.

[0123] Regarding step 402, inputting the first actual state vector into the central air conditioning control model so that the central air conditioning control model outputs the actual action vector, reference may be made to Figure 1 The description of the corresponding parts will not be repeated here in the embodiments of the present application.

[0124] 403. Use the actual motion vector to control the target central air conditioner.

[0125] Regarding step 403, using the actual motion vector to control the target central air conditioner, please refer to Figure 1 The description of the corresponding parts will not be repeated here in the embodiments of the present application.

[0126] Furthermore, in an embodiment of the present application, in order to ensure that the actual value of the indoor temperature in the actual environment meets the indoor temperature setting value, after using the control parameters contained in the actual action vector to control the target central air-conditioning, the target central air-conditioning control system can obtain the working mode, indoor temperature setting value and indoor temperature actual value corresponding to the target central air-conditioning from the actual environment. When the working mode corresponding to the target central air-conditioning is the cooling mode, if the actual value of the indoor temperature in the actual environment is greater than the indoor temperature setting value, the AHU water valve opening, the circulating water pump frequency and the heat pump outlet water temperature are adjusted in sequence so that the adjusted actual value of the indoor temperature is less than or equal to the indoor temperature setting value; when the working mode corresponding to the target central air-conditioning is the heating mode, if the actual value of the indoor temperature in the actual environment is less than the indoor temperature setting value, the AHU water valve opening, the circulating water pump frequency and the heat pump outlet water temperature are adjusted in sequence so that the adjusted actual value of the indoor temperature is greater than or equal to the indoor temperature setting value. Specifically, in this step, in the process of sequentially adjusting the AHU water valve opening, the circulating water pump frequency, and the heat pump outlet water temperature so that the adjusted actual indoor temperature value is less than or equal to the indoor temperature setting value, the target central air conditioning control system may adjust the circulating water pump frequency when the actual indoor temperature value is still greater than the indoor temperature setting value after the AHU water valve opening is adjusted to the maximum; and adjust the heat pump outlet water temperature when the actual indoor temperature value is still greater than the indoor temperature setting value after the circulating water pump frequency is adjusted to the maximum. In the process of sequentially adjusting the AHU water valve opening, the circulating water pump frequency, and the heat pump outlet water temperature so that the adjusted actual indoor temperature value is greater than or equal to the indoor temperature setting value, the target central air conditioning control system may adjust the circulating water pump frequency when the actual indoor temperature value is still less than the indoor temperature setting value after the AHU water valve opening is adjusted to the maximum; and adjust the heat pump outlet water temperature when the actual indoor temperature value is still less than the indoor temperature setting value after the circulating water pump frequency is adjusted to the maximum, but is not limited thereto.

[0127] 404. Train the central air conditioning control model based on the actual environment.

[0128] In the embodiment of the present application, after controlling the target central air conditioner using the control parameters included in the actual action vector, the target central air conditioner control system can also train the central air conditioner control model based on the actual environment.

[0129] Specifically, the target central air conditioning control system can train the central air conditioning control model based on the actual environment in the following ways:

[0130] (1) According to a preset duration and a preset rule, a plurality of indoor temperature setting values, a plurality of indoor temperature actual values, a plurality of system total powers, and a preset weighting coefficient are obtained from an actual environment, and the plurality of indoor temperature setting values, the plurality of indoor temperature actual values, the plurality of system total powers, and the preset weighting coefficient are substituted into a preset reward function formula to calculate an actual reward value.

[0131] The preset duration may be, but is not limited to, 1S, 10S, 50S, 100S, etc.; the preset rule may be, but is not limited to, obtaining data every X seconds, where X is a positive integer less than the value corresponding to the preset duration.

[0132] In the embodiment of the present application, within the preset time after the control parameters included in the actual action vector are used to control the target central air conditioner, the target central air conditioner control system needs to obtain multiple indoor temperature setting values, multiple indoor temperature actual values, multiple system total powers and preset weight coefficients from the actual environment according to preset rules, and substitute the multiple indoor temperature setting values, multiple indoor temperature actual values, multiple system total powers and preset weight coefficients into the preset reward function formula to calculate the actual reward value, wherein the preset reward function formula is as follows:

[0133]

[0134] Among them, R is the actual reward value, T set_t+i is the indoor temperature setting value obtained at a time interval of i seconds from the current time, T t+i is the actual indoor temperature value obtained at a time interval of i seconds from the current time, P t+i is the total system power obtained at a time interval of i seconds from the current time, λ is the preset weighting coefficient, n is the number of multiple indoor temperature setting values, and i is a natural number multiple of X.

[0135] (2) Generate a second actual state vector according to the state of the actual environment at the second moment.

[0136] The second actual state vector includes: the switch state of the uncontrolled device, the target external parameters, the operating parameters of the uncontrolled device and the target internal parameters, and the second moment is separated from the first moment by the preset time length.

[0137] (3) The first actual state vector, the actual action vector, the actual reward value, and the second actual state vector are stored as training samples in a training sample set.

[0138] (4) Randomly obtain M training samples from the training sample set, and use the M training samples to optimize and adjust the model parameters in the central air-conditioning control model.

[0139] Wherein, M is a positive integer.

[0140] Specifically, in this step, the target central air-conditioning control system adopts the parameter adjustment mechanism corresponding to the TD3 algorithm and uses M training samples to optimize and adjust the model parameters in the central air-conditioning control model.

[0141] It should be noted that in actual application, when the number of training samples stored in the training sample set is less than or equal to N, the target central air-conditioning control system obtains all training samples stored in the training sample set, and uses these training samples to optimize and adjust the model parameters in the central air-conditioning control model.

[0142] It should be noted that in actual application, the training sample set is an ordered set with a fixed capacity. When the number of training samples stored in the training sample set reaches a threshold and newly generated training samples need to be stored in the training sample set, the target central air-conditioning control system needs to delete the training samples first stored in the training sample set, and then store the newly generated training samples in the training sample set.

[0143] In order to achieve the above-mentioned purpose, according to another aspect of the present application, an embodiment of the present application also provides a storage medium, wherein the storage medium includes a stored program, wherein when the program is running, the device where the storage medium is located is controlled to execute the training method of the central air-conditioning control model described in the first aspect or the training method of the central air-conditioning control model described in the second aspect.

[0144] In order to achieve the above-mentioned purpose, according to another aspect of the present application, an embodiment of the present application also provides a storage medium, which includes a stored program, wherein when the program is running, the device where the storage medium is located is controlled to execute the central air-conditioning control method described in the third aspect or the central air-conditioning control method described in the fourth aspect.

[0145] In order to achieve the above-mentioned purpose, according to another aspect of the present application, an embodiment of the present application also provides an electronic device, which includes at least one processor, and at least one memory and a bus connected to the processor; wherein the processor and the memory communicate with each other through the bus; the processor is used to call the program instructions in the memory to execute the training method of the central air-conditioning control model described in the first aspect or the training method of the central air-conditioning control model described in the second aspect.

[0146] In order to achieve the above-mentioned purpose, according to another aspect of the present application, an embodiment of the present application also provides an electronic device, which includes at least one processor, and at least one memory and a bus connected to the processor; wherein the processor and the memory communicate with each other through the bus; the processor is used to call the program instructions in the memory to execute the central air-conditioning control method described in the third aspect or the central air-conditioning control method described in the fourth aspect.

[0147] Furthermore, as a response to the above Figure 1 and Figure 2 In order to realize the method shown in the figure, another embodiment of the present application also provides a training device for a central air conditioning control model. This device embodiment corresponds to the aforementioned method embodiment. For ease of reading, this device embodiment will not repeat the details of the aforementioned method embodiment one by one, but it should be clear that the device in this embodiment can correspond to all the contents of the aforementioned method embodiment. This device is used to reduce the energy consumption of the central air conditioner while maintaining the indoor temperature. Specifically, Figure 5 As shown, the device comprises:

[0148] A first establishing unit 51 is used to establish a neural network model according to the business rules adopted in the actual environment;

[0149] A first training unit 52, used for training the neural network model to obtain a simulation environment model;

[0150] A second establishing unit 53, used to establish a reinforcement learning model according to a preset algorithm;

[0151] The second training unit 54 is used to train the reinforcement learning model based on the simulation environment model until a preset training stop condition is reached to obtain a central air conditioning control model.

[0152] Further, such as Figure 6 As shown, the second training unit 54 includes:

[0153] A first generating module 541 is configured to generate a first training state vector using the simulation environment model, wherein the first training state vector is generated by the simulation environment model according to the state of the simulation environment model at a first moment, and the first training state vector includes: a switch state of an uncontrolled device, a target external parameter, an operating parameter of an uncontrolled device, and a target internal parameter, and the first moment is the current moment;

[0154] A first input module 542 is used to input the first training state vector into the reinforcement learning model so that the reinforcement learning model outputs a training action vector, wherein the training action vector includes: a heat pump outlet water temperature setting value, a circulating water pump frequency setting value, and an AHU water valve opening setting value;

[0155] A second input module 543 is used to input the training motion vector into the simulation environment model so that the simulation environment model controls the simulation central air conditioner according to the training motion vector;

[0156] an acquisition module 544, configured to acquire a plurality of indoor temperature setting values, a plurality of indoor temperature actual values, a plurality of system total powers, and a preset weight coefficient from the simulation environment model according to a preset duration and a preset rule, and substitute the plurality of indoor temperature setting values, the plurality of indoor temperature actual values, the plurality of system total powers, and the preset weight coefficient into a preset reward function formula to calculate a training reward value;

[0157] A second generating module 545 is used to generate a second training state vector using the simulation environment model, and store the first training state vector, the training action vector, the training reward value and the second training state vector as training samples in a training sample set, wherein the second training state vector is generated by the simulation environment model according to the state of the simulation environment model at a second moment, and the second training state vector includes: a switch state of an uncontrolled device, a target external parameter, an operating parameter of an uncontrolled device and a target internal parameter, and the second moment is separated from the first moment by the preset time length;

[0158] An adjustment module 546 is used to randomly obtain N training samples from the training sample set, and use the N training samples to optimize and adjust the model parameters in the reinforcement learning model, where N is a positive integer;

[0159] The judgment module 547 is used to judge whether the preset training stop condition is reached; when the preset training stop condition is reached, the reinforcement learning model is confirmed as the central air-conditioning control model; when the preset training stop condition is not reached, the first generation module 541, the first input module 542, the second input module 543, the acquisition module 544, the second generation module 545 and the adjustment module 546 are controlled to train the reinforcement learning model.

[0160] Furthermore, as a response to the above Figure 3 and Figure 4 In order to realize the method shown, another embodiment of the present application further provides a target central air-conditioning control system, which is used to control the target central air-conditioning based on the central air-conditioning control model obtained by the training method described in the first aspect or the second aspect. This device embodiment corresponds to the aforementioned method embodiment. For ease of reading, this device embodiment will no longer repeat the details of the aforementioned method embodiment one by one, but it should be clear that the target central air-conditioning control system in this embodiment can correspond to all the contents of the aforementioned method embodiment. The target central air-conditioning control system is used to reduce the energy consumption of the central air-conditioning while maintaining the indoor temperature, as shown in the following example. Figure 7 As shown, the device comprises:

[0161] A first generating unit 61 is configured to generate a first actual state vector according to the state of the actual environment at a first moment, wherein the first actual state vector includes: a switch state of an uncontrolled device, a target external parameter, an operating parameter of an uncontrolled device, and a target internal parameter, and the first moment is a current moment;

[0162] A first input unit 62, used to input the first actual state vector into the central air conditioning control model, so that the central air conditioning control model outputs an actual action vector, the actual action vector includes: a heat pump outlet water temperature setting value, a circulating water pump frequency setting value, and an AHU water valve opening setting value;

[0163] The control unit 63 is used to control the target central air conditioner using the actual motion vector.

[0164] Further, such as Figure 8 As shown, the target central air-conditioning control system also includes:

[0165] A first acquisition unit 64 is used to acquire the working mode, indoor temperature setting value and indoor temperature actual value corresponding to the target central air conditioner from the actual environment;

[0166] A first adjustment unit 65 is used for, when the working mode is the cooling mode, if the actual value of the indoor temperature is greater than the set value of the indoor temperature, adjusting the AHU water valve opening, the circulating water pump frequency and the heat pump outlet water temperature in sequence, so that the adjusted actual value of the indoor temperature is less than or equal to the set value of the indoor temperature;

[0167] The first adjustment unit 65 is also used to adjust the AHU water valve opening, the circulating water pump frequency and the heat pump outlet water temperature in sequence when the working mode is the heating mode, if the actual value of the indoor temperature is less than the set value of the indoor temperature, so that the adjusted actual value of the indoor temperature is greater than or equal to the set value of the indoor temperature.

[0168] Further, such as Figure 8 As shown, the target central air-conditioning control system also includes:

[0169] A second acquisition unit 66 is used to acquire a plurality of indoor temperature setting values, a plurality of indoor temperature actual values, a plurality of system total powers and a preset weight coefficient from the actual environment according to a preset duration and a preset rule, and substitute the plurality of indoor temperature setting values, the plurality of indoor temperature actual values, the plurality of system total powers and the preset weight coefficient into a preset reward function formula to calculate an actual reward value;

[0170] A second generating unit 67 is configured to generate a second actual state vector according to the state of the actual environment at a second moment, wherein the second actual state vector includes: a switch state of an uncontrolled device, a target external parameter, an operating parameter of an uncontrolled device, and a target internal parameter, and the second moment is separated from the first moment by the preset time length;

[0171] A storage unit 68, configured to store the first actual state vector, the actual action vector, the actual reward value, and the second actual state vector as training samples in a training sample set;

[0172] The second adjustment unit 69 is used to randomly obtain M training samples from the training sample set, and use the M training samples to optimize and adjust the model parameters in the central air conditioning control model, where M is a positive integer.

[0173] The embodiment of the present application provides a training method for a central air conditioning control model, a central air conditioning control method and a device. The embodiment of the present application can establish a neural network model according to the business rules adopted by the target terminal device in the actual environment, and train the neural network model to obtain the simulation environment model, and then establish a reinforcement learning model according to a preset algorithm, and train the reinforcement learning model based on the simulation environment model until the preset training stop condition is reached, thereby obtaining the central air conditioning control model. After the central air conditioning control model is migrated to the target central air conditioning control system, the target central air conditioning control system can control the target central air conditioning based on the central air conditioning control model. Since the training process of the simulation environment model is simple, training the reinforcement learning model based on the simulation environment model can enable the reinforcement learning model to quickly reach the training stop condition, so the central air conditioning control model can be quickly trained; and the central air conditioning control system can achieve precise control of the central air conditioning based on the central air conditioning control model, thereby maintaining the indoor temperature while reducing the energy consumption of the central air conditioning.

[0174] The training device of the central air conditioning control model includes a processor and a memory. The above-mentioned first establishment unit, first training unit, second establishment unit and second training unit are all stored in the memory as program units, and the processor executes the above-mentioned program units stored in the memory to realize corresponding functions.

[0175] The processor includes a kernel, which retrieves the corresponding program unit from the memory. One or more kernels can be set, and the energy consumption of the central air conditioner can be reduced while maintaining the indoor temperature by adjusting the kernel parameters.

[0176] The target central air-conditioning control system includes a processor and a memory. The first generation unit, the first input unit and the control unit are stored in the memory as program units. The processor executes the program units stored in the memory to implement corresponding functions.

[0177] The processor includes a kernel, which retrieves the corresponding program unit from the memory. One or more kernels can be set, and the energy consumption of the central air conditioner can be reduced while maintaining the indoor temperature by adjusting the kernel parameters.

[0178] An embodiment of the present application provides a storage medium on which a program is stored. When the program is executed by a processor, it implements the training method of the central air conditioning control model as described in the first aspect or the second aspect above; or the central air conditioning control method as described in the third aspect or the fourth aspect above.

[0179] The present application embodiment provides an electronic device 70, such as Fig. 9As shown, the electronic device 70 includes at least one processor 701, and at least one memory 702 and a bus 703 connected to the processor; wherein the processor 701 and the memory 702 communicate with each other through the bus 703; the processor 701 is used to call the program instructions in the memory 702 to execute the training method of the central air-conditioning control model as described in the first aspect or the second aspect above; or the central air-conditioning control method as described in the third aspect or the fourth aspect above.

[0180] The present application also provides a computer program product, which, when executed on a data processing device, is suitable for executing a program for initializing the following method steps:

[0181] Establishing a neural network model according to the business rules adopted in the actual environment, and training the neural network model to obtain a simulation environment model;

[0182] Build a reinforcement learning model based on a preset algorithm;

[0183] The reinforcement learning model is trained based on the simulation environment model until a preset training stop condition is reached to obtain a central air conditioning control model.

[0184] Furthermore, the reinforcement learning model is trained based on the simulation environment model until a preset training stop condition is reached to obtain a central air conditioning control model, including:

[0185] Step S1: using the simulation environment model to generate a first training state vector, wherein the first training state vector is generated by the simulation environment model according to the state of the simulation environment model at a first moment, and the first training state vector includes: a switch state of an uncontrolled device, a target external parameter, an operating parameter of an uncontrolled device, and a target internal parameter, and the first moment is the current moment;

[0186] Step S2: inputting the first training state vector into the reinforcement learning model so that the reinforcement learning model outputs a training action vector, wherein the training action vector includes: a heat pump outlet water temperature setting value, a circulating water pump frequency setting value, and an AHU water valve opening setting value;

[0187] Step S3: inputting the training motion vector into the simulation environment model so that the simulation environment model controls the simulation central air conditioner according to the training motion vector;

[0188] Step S4: acquiring a plurality of indoor temperature setting values, a plurality of indoor temperature actual values, a plurality of system total powers and a preset weight coefficient from the simulation environment model according to a preset duration and a preset rule, and substituting the plurality of indoor temperature setting values, the plurality of indoor temperature actual values, the plurality of system total powers and the preset weight coefficient into a preset reward function formula to calculate a training reward value;

[0189] Step S5: using the simulation environment model to generate a second training state vector, and storing the first training state vector, the training action vector, the training reward value and the second training state vector as training samples in a training sample set, wherein the second training state vector is generated by the simulation environment model according to the state of the simulation environment model at a second moment, and the second training state vector includes: a switch state of an uncontrolled device, target external parameters, operating parameters of an uncontrolled device and target internal parameters, and the second moment is separated from the first moment by the preset time length;

[0190] Step S6: randomly obtaining N training samples from the training sample set, and using the N training samples to optimize and adjust the model parameters in the reinforcement learning model, where N is a positive integer;

[0191] Step S7: Determine whether the preset training stop condition is reached; if reached, confirm the reinforcement learning model as the central air-conditioning control model; if not reached, return to step S1.

[0192] The present application also provides a computer program product, which, when executed on a data processing device, is suitable for executing a program for initializing the following method steps:

[0193] When controlling the target central air conditioner, a first actual state vector is generated according to the state of the actual environment at a first moment, wherein the first actual state vector includes: the switch state of the uncontrolled device, the target external parameter, the operating parameter of the uncontrolled device and the target internal parameter, and the first moment is the current moment;

[0194] Inputting the first actual state vector into the central air conditioning control model so that the central air conditioning control model outputs an actual action vector, wherein the actual action vector includes: a heat pump outlet water temperature setting value, a circulating water pump frequency setting value, and an AHU water valve opening setting value;

[0195] The target central air conditioner is controlled using the actual motion vector.

[0196] Further, after using the actual motion vector to control the target central air conditioner, the method further includes:

[0197] Acquire the working mode, indoor temperature setting value and indoor temperature actual value corresponding to the target central air conditioner from the actual environment;

[0198] When the working mode is cooling mode, if the actual indoor temperature value is greater than the indoor temperature setting value, the AHU water valve opening, the circulating water pump frequency and the heat pump outlet water temperature are adjusted in sequence so that the adjusted actual indoor temperature value is less than or equal to the indoor temperature setting value;

[0199] When the working mode is the heating mode, if the actual value of the indoor temperature is less than the set value of the indoor temperature, the AHU water valve opening, the circulating water pump frequency and the heat pump outlet water temperature are adjusted in sequence so that the adjusted actual value of the indoor temperature is greater than or equal to the set value of the indoor temperature.

[0200] Further, after using the actual motion vector to control the target central air conditioner, the method further includes:

[0201] Acquire multiple indoor temperature setting values, multiple indoor temperature actual values, multiple system total powers, and preset weight coefficients from the actual environment according to a preset duration and a preset rule, and substitute the multiple indoor temperature setting values, multiple indoor temperature actual values, multiple system total powers, and the preset weight coefficients into a preset reward function formula to calculate an actual reward value;

[0202] Generate a second actual state vector according to the state of the actual environment at the second moment, wherein the second actual state vector includes: the switch state of the uncontrolled device, the target external parameter, the operating parameter of the uncontrolled device, and the target internal parameter, and the second moment is separated from the first moment by the preset time length;

[0203] storing the first actual state vector, the actual action vector, the actual reward value, and the second actual state vector as training samples in a training sample set;

[0204] M training samples are randomly obtained from the training sample set, and the M training samples are used to optimize and adjust the model parameters in the central air-conditioning control model, where M is a positive integer.

[0205] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0206] In a typical configuration, the device includes one or more processors (CPU), memory and bus. The device may also include input / output interface, network interface and the like.

[0207] The memory may include non-permanent memory in a computer-readable medium, random access memory (RAM) and / or non-volatile memory in the form of read-only memory (ROM) or flash RAM, and the memory includes at least one memory chip. The memory is an example of a computer-readable medium.

[0208] Computer readable media include permanent and non-permanent, removable and non-removable media that can be implemented by any method or technology to store information. Information can be computer readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, magnetic tape magnetic disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer readable media does not include temporary computer readable media (transitory media), such as modulated data signals and carrier waves.

[0209] It should also be noted that the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, commodity or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, commodity or device. In the absence of more restrictions, the elements defined by the sentence "comprises a ..." do not exclude the existence of other identical elements in the process, method, commodity or device including the elements.

[0210] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment or an embodiment in combination with software and hardware. Moreover, the present application may adopt the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that contain computer-usable program code.

[0211] The above are only embodiments of the present application and are not intended to limit the present application. For those skilled in the art, the present application may have various changes and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application should be included within the scope of the claims of the present application.

Claims

1. A training method for a central air conditioning control model, characterized in that: The method is applied to a target terminal device, comprising: Establishing a neural network model according to the business rules adopted in the actual environment, and training the neural network model to obtain a simulation environment model; Build a reinforcement learning model based on a preset algorithm; Training the reinforcement learning model based on the simulation environment model until a preset training stop condition is reached to obtain a central air conditioning control model; The reinforcement learning model is trained based on the simulation environment model until a preset training stop condition is reached to obtain a central air conditioning control model, including: Step S1: Generate a first training state vector using the simulation environment model, wherein the first training state vector is generated by the simulation environment model according to the state of the simulation environment model at a first moment, and the first training state vector includes: the switch state of the uncontrolled device, the target external parameters, the operating parameters of the uncontrolled device, and the target internal parameters, and the first moment is the current moment; wherein the uncontrolled device is a device that cannot be controlled by the central air-conditioning control system, and the uncontrolled device is a device that can be controlled by the central air-conditioning control system but is not controlled; Step S2: inputting the first training state vector into the reinforcement learning model so that the reinforcement learning model outputs a training action vector, wherein the training action vector includes: a heat pump outlet water temperature setting value, a circulating water pump frequency setting value, and an AHU water valve opening setting value; Step S3: inputting the training motion vector into the simulation environment model so that the simulation environment model controls the simulation central air conditioner according to the training motion vector; Step S4: acquiring a plurality of indoor temperature setting values, a plurality of indoor temperature actual values, a plurality of system total powers and a preset weight coefficient from the simulation environment model according to a preset duration and a preset rule, and substituting the plurality of indoor temperature setting values, the plurality of indoor temperature actual values, the plurality of system total powers and the preset weight coefficient into a preset reward function formula to calculate a training reward value; Step S5: using the simulation environment model to generate a second training state vector, and storing the first training state vector, the training action vector, the training reward value and the second training state vector as training samples in a training sample set, wherein the second training state vector is generated by the simulation environment model according to the state of the simulation environment model at a second moment, and the second training state vector includes: a switch state of an uncontrolled device, target external parameters, operating parameters of an uncontrolled device and target internal parameters, and the second moment is separated from the first moment by the preset time length; Step S6: randomly obtaining N training samples from the training sample set, and using the N training samples to optimize and adjust the model parameters in the reinforcement learning model, where N is a positive integer; Step S7: Determine whether the preset training stop condition is reached; if reached, confirm the reinforcement learning model as the central air-conditioning control model; if not reached, return to step S1.

2. A central air conditioning control method, characterized in that: Used to control a target central air conditioner based on the central air conditioner control model obtained by the training method described in claim 1.

3. The method according to claim 2, characterized in that When controlling the target central air conditioner, a first actual state vector is generated according to the state of the actual environment at a first moment, wherein the first actual state vector includes: the switch state of the uncontrolled device, the target external parameter, the operating parameter of the uncontrolled device and the target internal parameter, and the first moment is the current moment; Inputting the first actual state vector into the central air conditioning control model so that the central air conditioning control model outputs an actual action vector, wherein the actual action vector includes: a heat pump outlet water temperature setting value, a circulating water pump frequency setting value, and an AHU water valve opening setting value; The target central air conditioner is controlled using the actual motion vector.

4. The method according to claim 3, characterized in that After using the actual motion vector to control the target central air conditioner, the method further includes: Acquire the working mode, indoor temperature setting value and indoor temperature actual value corresponding to the target central air conditioner from the actual environment; When the working mode is cooling mode, if the actual indoor temperature value is greater than the indoor temperature setting value, the AHU water valve opening, the circulating water pump frequency and the heat pump outlet water temperature are adjusted in sequence so that the adjusted actual indoor temperature value is less than or equal to the indoor temperature setting value; When the working mode is the heating mode, if the actual value of the indoor temperature is less than the set value of the indoor temperature, the AHU water valve opening, the circulating water pump frequency and the heat pump outlet water temperature are adjusted in sequence so that the adjusted actual value of the indoor temperature is greater than or equal to the set value of the indoor temperature.

5. The method according to claim 3, characterized in that After using the actual motion vector to control the target central air conditioner, the method further includes: Acquire multiple indoor temperature setting values, multiple indoor temperature actual values, multiple system total powers, and preset weight coefficients from the actual environment according to a preset duration and a preset rule, and substitute the multiple indoor temperature setting values, multiple indoor temperature actual values, multiple system total powers, and the preset weight coefficients into a preset reward function formula to calculate an actual reward value; Generate a second actual state vector according to the state of the actual environment at the second moment, wherein the second actual state vector includes: the switch state of the uncontrolled device, the target external parameter, the operating parameter of the uncontrolled device, and the target internal parameter, and the second moment is separated from the first moment by the preset time length; storing the first actual state vector, the actual action vector, the actual reward value, and the second actual state vector as training samples in a training sample set; M training samples are randomly obtained from the training sample set, and the M training samples are used to optimize and adjust the model parameters in the central air-conditioning control model, where M is a positive integer.

6. A training device for a central air conditioning control model, characterized in that: include: A first establishing unit is used to establish a neural network model according to the business rules adopted in the actual environment; A first training unit, used for training the neural network model to obtain a simulation environment model; A second establishing unit is used to establish a reinforcement learning model according to a preset algorithm; A second training unit is used to train the reinforcement learning model based on the simulation environment model until a preset training stop condition is reached to obtain a central air conditioning control model; The second training unit comprises: A first generating module is used to generate a first training state vector using the simulation environment model, wherein the first training state vector is generated by the simulation environment model according to the state of the simulation environment model at a first moment, and the first training state vector includes: a switch state of an uncontrolled device, a target external parameter, an operating parameter of an uncontrolled device, and a target internal parameter, and the first moment is a current moment; wherein the uncontrolled device is a device that cannot be controlled by a central air-conditioning control system, and the uncontrolled device is a device that can be controlled by the central air-conditioning control system but is not controlled; A first input module, used to input the first training state vector into the reinforcement learning model, so that the reinforcement learning model outputs a training action vector, wherein the training action vector includes: a heat pump outlet water temperature setting value, a circulating water pump frequency setting value, and an AHU water valve opening setting value; A second input module, used for inputting the training motion vector into the simulation environment model, so that the simulation environment model controls the simulation central air conditioner according to the training motion vector; an acquisition module, used for acquiring a plurality of indoor temperature setting values, a plurality of indoor temperature actual values, a plurality of system total powers and a preset weight coefficient from the simulation environment model according to a preset duration and a preset rule, and substituting the plurality of indoor temperature setting values, the plurality of indoor temperature actual values, the plurality of system total powers and the preset weight coefficient into a preset reward function formula to calculate a training reward value; A second generating module is used to generate a second training state vector using the simulation environment model, and store the first training state vector, the training action vector, the training reward value and the second training state vector as training samples in a training sample set, wherein the second training state vector is generated by the simulation environment model according to the state of the simulation environment model at a second moment, and the second training state vector includes: a switch state of an uncontrolled device, a target external parameter, an operating parameter of an uncontrolled device and a target internal parameter, and the second moment is separated from the first moment by the preset time length; An adjustment module, used to randomly obtain N training samples from the training sample set, and use the N training samples to optimize and adjust the model parameters in the reinforcement learning model, where N is a positive integer; A judgment module is used to judge whether the preset training stop condition is reached; when the preset training stop condition is reached, the reinforcement learning model is confirmed as the central air-conditioning control model; when the preset training stop condition is not reached, the first generation module, the first input module, the second input module, the acquisition module, the second generation module and the adjustment module are controlled to train the reinforcement learning model.

7. A target central air conditioning control system, characterized in that: Used to control a target central air conditioner based on the central air conditioner control model obtained by the training method described in claim 1.

8. A storage medium, characterized in that: The storage medium includes a stored program, wherein when the program is executed, the device where the storage medium is located is controlled to execute the method according to any one of claims 1 to 5.

9. An electronic device, characterized in that: The device includes at least one processor, and at least one memory and a bus connected to the processor; Wherein, the processor and the memory communicate with each other through the bus; The processor is used to call the program instructions in the memory to execute the method according to any one of claims 1 to 5.

Images