Fan coil control system combined with multi-position control valve and control method

Abstract

The invention relates to the technical field of artificial intelligence and fan coils, and discloses a fan coil control system combined with a multi-position control valve, the multi-position control valve is arranged on a water inlet loop and in front of a fan coil, and the multi-position control valve is connected with the fan coil in series. The multi-position control valve, the fan coil, the equipment state acquisition module and the indoor state acquisition module are all in signal connection with the micro-control unit MCU, and the micro-control unit MCU is in signal connection with the reinforcement learning controller through a network. The invention further provides a method for conducting fan coil control through the fan coil control system combined with the multi-position control valve. According to the method, through multi-valve-position control based on deep learning, cooling capacity output of the fan coil has diversity, the method better adapts to load changes, and great fluctuation of the room temperature is reduced.

Description

Technical Field

[0001] The present invention belongs to the technical field of artificial intelligence and fan coil units, and in particular relates to a fan coil unit control system and a control method combined with a multi-position control valve. Background Art

[0002] Fan coil units are widely used in building air conditioning systems. The constant air volume control method is generally used in control, that is, the room temperature is controlled to fluctuate around the set value by turning the water valve on and off based on rule control. Taking summer working conditions as an example, when the room temperature reaches the upper limit, the water valve opens, and the fan maintains constant air volume operation; when the room temperature reaches the lower limit, the water valve closes, and the interlocked fan is also closed at the same time; when the room temperature is between the upper and lower limits, the current water valve on and off state is maintained. Although this control method is simple, it has the following disadvantages: 1) The single temperature feedback signal makes it impossible for the controller to make optimization decisions based on more information; 2) The on and off actions are too simple to meet the diverse control needs, resulting in more frequent on and off phenomena, affecting thermal comfort. In addition, when the valve is opened, the water volume is always supplied at the rated flow rate to ensure the most unfavorable situation. In the transition season, it will cause a large flow and small temperature difference phenomenon, which makes the energy consumption of the distribution system larger.

[0003] The above-mentioned on-off control method belongs to a model-free rule control method. In comparison, the on-off control method of fan coil based on rule control has the advantage of simple implementation process. For example, the patent "Fan coil control system and method based on deep reinforcement learning of prior knowledge" with publication number CN116989384A uses the state and demand cooling load as the input of the control module to control the water flow of the water pump and the air flow of the fan. However, its disadvantages are also quite obvious: 1) The single temperature feedback signal makes it impossible for the controller to make optimization decisions based on more information; 2) The on and off actions are too simple and cannot meet the diverse control needs. For example, in the transition season, when the valve is open, the cooling capacity provided by the fan coil is often much higher than the indoor load, causing the room temperature to quickly drop to the lower limit, which triggers the water valve and fan coil to close. After that, the room temperature increases to the upper limit, and the water valve and fan coil are opened again, resulting in more frequent on and off phenomena, large fluctuations in indoor temperature, and affected thermal comfort; 3) When the valve is open, supplying water with the designed amount will cause a large flow and small temperature difference under partial load, resulting in higher energy consumption of the distribution system and a lower cold water delivery coefficient, which reduces the economic efficiency of the system operation.

[0004] In summary, the current two-way valve fan coil control action is too simple and cannot match different load conditions well. At the same time, there is a lack of adaptive control measures, which adversely affects the indoor comfort and system operation economy. Therefore, it is necessary to propose a fan coil with greater control freedom and its adaptive optimization control method. Summary of the invention

[0005] The technical problem to be solved by the present invention is to provide a fan coil control system and a control method combined with a multi-position control valve, so as to realize optimized control of the fan coil system to save energy consumption of the distribution system and load energy consumption.

[0006] In order to solve the above technical problems, the present invention provides a fan coil control system combined with a multi-position control valve, comprising an upper computer unit and a lower computer unit, the upper computer unit comprising a reinforcement learning controller, and the lower computer unit comprising a multi-position control valve, a fan coil, an equipment status acquisition module, an indoor status acquisition module and a microcontroller unit MCU;

[0007] The multi-position control valve is arranged on the water inlet circuit, before the fan coil unit and connected in series with the fan coil unit;

[0008] The multi-position control valve, fan coil unit, equipment status acquisition module and indoor status acquisition module are all connected to the micro-control unit MCU signal, and the micro-control unit MCU is connected to the reinforcement learning controller signal through the network.

[0009] As an improvement of the fan coil control system combined with a multi-position control valve of the present invention:

[0010] The indoor state acquisition module includes a temperature sensor for acquiring indoor temperature information;

[0011] The equipment status acquisition module is connected to the multi-position control valve signal to obtain valve position information;

[0012] The reinforcement learning controller includes a buffer, a policy network, a target network and a reward function. Both the policy network and the target network adopt a DQN network. The reinforcement learning controller sends valve position action instructions to a multi-position control valve through a microcontroller unit MCU.

[0013] The present invention also provides a control method for a fan tube coil:

[0014] S1. Obtain the state space and action space of the reinforcement learning controller, where the state space includes the set temperature difference, the gear position of the multi-position control valve and the temperature variation difference, and the action space is the valve position of the multi-position control valve;

[0015] S2. Set the reward function of the reinforcement learning controller;

[0016] S3. Initialize network parameters

[0017] The room and fan coil model are simulated, the simulation time interval and the start and end time of each cycle are set, and the exploration is carried out according to the ε-greedy strategy. A set of state transition samples are obtained through simulation at each step, and the state transition samples are saved in the buffer; the state transition samples include the current state (s i ), action (ai ), reward (r i+1 ) and the next state (s i+1 );

[0018] Every time a set of state transition samples is generated, the DQN network is trained once to optimize the network parameters. When the training reaches the convergence requirement, the preliminary policy network and target network weight parameters are obtained.

[0019] S4. Online operation and training

[0020] Set the ε initial value and decay constant of the ε-greedy strategy; collect and calculate the state transition samples of the current time step, and then send them to the reinforcement learning controller. The DQN network selects random actions with ε probability according to the ε-greedy algorithm and selects the best action predicted by the strategy network with 1-ε probability; feed the generated actions back to the multi-position control valve for online control; then, collect and calculate the state transition samples of the next time step and save them in the buffer;

[0021] Every time a set of state transition samples is obtained, the policy network is trained online to obtain new policy network weights and update the target network weights.

[0022] As an improvement of the fan coil control method of the present invention:

[0023] The temperature difference Δt i for:

[0024] Δt i =t i -t i-1 (1)

[0025] As a further improvement of the fan coil control method of the present invention:

[0026] The set temperature difference Δt set，i In cooling condition:

[0027] Δt set,i =t i -(t set +t band / twenty two)

[0028] Under heating conditions:

[0029] Δt set,i =t i -(t set -t band / twenty three)

[0030] Among them, t i is the room temperature obtained by the i-th sampling, t setis the room temperature setting value, t band To control the dead zone, t i-1 is the i-1th room temperature sampling value.

[0031] As a further improvement of the fan coil control method of the present invention:

[0032] The gear position of the multi-position control valve is the action value a given by the i-1th instruction i-1 .

[0033] As a further improvement of the fan coil control method of the present invention:

[0034] The reward function r i for:

[0035] r i =r temp,i +gr energy,i (4)

[0036] r energy,i =a max -a i-1 (5)

[0037] Among them, r energy,i represents the energy consumption reward, γ represents the weight factor of the water energy consumption reward, a max is the maximum action value;

[0038] r temp,i Represents room temperature bonus, under cooling conditions:

[0039]

[0040] Under heating conditions:

[0041]

[0042] As a further improvement of the fan coil control method of the present invention:

[0043] The buffer adopts a first-in-first-out method.

[0044] The beneficial effects of the present invention are mainly reflected in:

[0045] 1. The present invention can make the cooling output of the fan coil unit diverse through multi-valve position control based on deep learning, better adapt to load changes, and reduce large fluctuations in room temperature.

[0046] 2. The fan coil control method of the present invention reduces water transportation energy consumption while ensuring comfort.

[0047] 3. The control method of the fan coil of the present invention does not rely on the precise mathematical and physical model of the specific equipment, but reflects the system operation characteristics and realizes adaptive control through online data drive. BRIEF DESCRIPTION OF THE DRAWINGS

[0048] The specific implementation modes of the present invention are further described in detail below with reference to the accompanying drawings.

[0049] Figure 1 A schematic diagram of a fan coil control system combined with a multi-position control valve according to the present invention;

[0050] Figure 2 It is a schematic diagram of the control effect of the fan coil control system combined with the multi-position control valve based on the present invention;

[0051] Figure 3 Schematic diagram of the fan coil control effect based on on-off control. DETAILED DESCRIPTION

[0052] The present invention is further described below in conjunction with specific embodiments, but the protection scope of the present invention is not limited thereto:

[0053] Embodiment 1: A fan coil control system combined with a multi-position control valve, such as Figure 1 As shown, it is mainly composed of a multi-position control valve, a fan coil unit, an equipment status acquisition module, an indoor status acquisition module and a reinforcement learning controller. The multi-position control valve is arranged on the water inlet circuit, before the fan coil unit and in series with the fan coil unit. The multi-position control valve can provide control actions of more than two valve positions to achieve water volume closing, full opening and one or more intermediate water volume states.

[0054] The reinforcement learning controller is mainly composed of a buffer, a policy network, a target network and a reward function, where both the policy network and the target network use a deep Q_learning network (DQN). The reinforcement learning controller sends valve position action instructions to the multi-position control valve through the microcontroller unit MCU. The equipment status acquisition module and the indoor status acquisition module are connected to the reinforcement learning controller signal and feed back the acquisition status to the reinforcement learning controller.

[0055] The equipment status acquisition module, indoor status acquisition module, fan coil unit and multi-position control valve constitute the lower computer unit, which is responsible for data acquisition and task execution. The reinforcement learning controller is implemented in the upper computer unit, which is responsible for information aggregation, logic processing and action command issuance. The upper computer unit and the lower computer unit are connected through wireless network signals to realize data and command interaction between each other.

[0056] The equipment status acquisition module collects the valve position information of the multi-position control valve and the fan gear position information of the fan coil unit, and sends it to the host computer system for reception through the micro-control unit MCU connected to the wireless module.

[0057] The indoor status acquisition module collects the current indoor temperature information through the temperature sensor, and sends it to the host computer system through the micro-control unit MCU connected to the wireless module.

[0058] The specific process of the fan coil control method based on the fan coil control system combined with a multi-position control valve is as follows:

[0059] 1. Obtain the state space and action space of the reinforcement learning controller

[0060] The state space is mainly determined by the set temperature difference Δt set,i , multi-position control valve position a i-1 , temperature variation Δt i Specifically:

[0061] Δt i =t i -t i-1 (1)

[0062] Under cooling conditions:

[0063] Δt set,i =t i -(t set +t band / twenty two)

[0064] Under heating conditions:

[0065] Δt set,i =t i -(t set -t band / twenty three)

[0066] Among them, t i is the room temperature obtained by the i-th sampling, t set is the room temperature setting value, t band To control the dead zone, usually 2℃, t i-1 is the i-1th room temperature sampling value.

[0067] a i-1 is the action value given by the i-1th instruction.

[0068] The action space refers to the valve position of a multi-position control valve. For example, the action space for a 4-position control valve is [0, 1, 2, 3], where 0, 1, 2, and 3 represent the 4 openings of the control valve, corresponding to the closed water volume, the 1st water volume, the 2nd water volume, and the fully open water volume.

[0069] 2. Reward function setting for reinforcement learning controller

[0070] The reward function calculates the immediate reward consisting of room temperature and energy consumption based on the state information:

[0071] r i =r temp,i +gr energy,i (4)

[0072] r energy,i =a max -a i-1 (5)

[0073] Among them, r energy,i Represents the energy consumption reward, which is replaced by the action value. The higher the action value, the less the reward. γ represents the weight factor of the water energy consumption reward, such as 0.1. max is the maximum action value.

[0074] r temp,i Represents room temperature bonus, under cooling conditions:

[0075]

[0076] Under heating conditions:

[0077]

[0078] 3. Initialize network parameters

[0079] Using building environment simulation tools, we build room and fan coil models, export the models as FMU files, use matlab or python to call FMU for joint simulation, set the simulation time interval and the start and end time of each cycle (Episode), explore and utilize according to the ε-greedy strategy, and obtain a set of state transition samples (Transition) through virtual simulation at each step. The state transition samples (Transition) are filled into a buffer of a certain size for storage. When the data volume reaches the set size of the buffer, the redundant data is squeezed out in a first-in-first-out manner.

[0080] Each group of state transition samples (Transition) includes the current state (s i ), action (a i ), reward (r i+1 ) and the next state (s i+1 )4 parts of data, including the current state (s i ) corresponds to the state space established in step 1, and the action (a i ) corresponds to the action space, the reward (r i+1) is obtained by calculation in step 2. Every time a state transition sample (Transition) is generated, the DQN network is trained once to optimize the network parameters. When the training reaches the convergence requirement, the preliminary policy network and target network weight parameters are obtained, so that the DQN network of the reinforcement learning controller can approximately achieve a more reasonable action selection in the initial stage of operation.

[0081] 4. Online operation and training

[0082] In the actual online operation, both the policy network and the target network load the network weight parameters trained in step 3, set the initial value of ε and the decay constant of the ε-greedy strategy, and select a smaller value for ε at this time to avoid excessive interference of large randomness on the actual operation. In each time step, the state transition sample (Transition) of the current time step is collected and calculated, and then sent to the reinforcement learning controller. The DQN network selects random actions with a probability of ε according to the ε-greedy algorithm, and selects the best action predicted by the policy network with a probability of 1-ε. The generated action is fed back to the multi-position control valve for online control. In the next time step, the new state information is collected and the reward value is calculated to obtain a state transition sample (s t , a t , r t+1 ,s t+1 ), fill the state transition sample into the buffer and save it.

[0083] Each time a state transition sample is obtained, the policy network is trained online to obtain a new policy network weight. After a certain number of trainings, the weight of the target network is updated using the policy network weight. The ε factor is slightly larger at the beginning. As the online training progresses, ε decreases exponentially according to the decay constant. After it drops to a certain value, it no longer decreases, that is, a minimum exploration function is maintained, so that the policy network and the target network are used to the maximum extent.

[0084] As the operation data accumulates, the samples in the buffer are continuously and dynamically updated to reflect the actual response of the environment (including rooms and fan coils) to the actions in real time, and their connections are reflected in the network model through optimization, thereby achieving adaptive control.

[0085] As can be seen from the above, the present invention proposes a fan coil unit combined with a multi-position control valve and a control method thereof. A general fan coil unit uses an on-off control valve in its water path and adopts an on-off control strategy based on rule control. The present invention is an improvement on the on-off control valve, proposes a multi-valve position fan coil unit (a combination of a multi-position control valve + a fan coil unit) and provides its control steps. There are mature products for separate multi-valve position control valves and fan coil units, but the combination of a multi-valve position control valve and a fan coil unit has never been seen before.

[0086] experiment:

[0087] The calculation parameters of implementation example 1 are shown in Table 1. The room object is a light office building that accommodates 4 people, with a construction area of ​​16 square meters, a floor height of 3 meters, and a 3-square-meter window on the south wall. The fan coil unit is designed to have a cooling capacity of 2000w, a rated water flow of 0.095kg / s, a rated air volume of 0.185kg / s, a pressure difference of the cold water distribution system set to 300000kpa (30mH2o), and a water pump efficiency set to 0.7. The calculation time is the transition season in early June, from 8:00 am to 18:00 pm, a total of 10 hours. During the operation period, the maximum outdoor temperature is 32℃, the minimum temperature is 21℃, the upper limit of the room temperature is set to 25℃, and the lower limit is set to 23℃.

[0088] Figure 2 The control effect of fan coil control based on the fan coil control system combined with a multi-position control valve of the present invention is shown. It can be seen that under reinforcement learning control, during the morning period when the load is low, the valve position of the multi-valve position controller fluctuates between 0 and 1, making the room temperature basically stable between 24-25°C. In the afternoon period, due to the increase in load, the valve position of the multi-valve position controller fluctuates between 2 and 1, making the room temperature fluctuate close to the upper limit of 25°C. It is worth noting that the full-open valve position of the third gear is not used throughout the day, because the third gear valve position will make the room temperature low and is not conducive to energy saving. The full-day cooling consumption during reinforcement learning control is 24128.9 kJ, the energy consumption of cold water distribution is 309.5 kJ, and the cold water transmission coefficient (ratio of cooling capacity to water pump energy consumption) is 78.

[0089] As a comparison, Figure 3 The control effect of the fan coil unit using the usual on-off control is demonstrated. The fan coil unit switches between full on and off, and the indoor temperature fluctuates greatly, fluctuating between 23-25°C. The total cooling consumption throughout the day is 26473.1 kJ, the total cold water distribution energy consumption throughout the day is 489.9 kJ, and the cold water delivery coefficient is 54.

[0090] Table 1 Comparison of results of implementation example 1 and conventional on-off control

[0091]

[0092]

[0093] It can be seen that after adopting the reinforcement learning control of the multi-valve position of the present invention, the amount of water required by the system is reduced, and the corresponding energy consumption of cold water transportation is reduced by 36.8%; because the reinforcement learning controls the temperature at the set upper limit, the air conditioning load is reduced, so the cooling consumption is reduced by 8.8%. Therefore, the multi-valve position fan coil control system and method proposed by the present invention based on reinforcement learning combined with a multi-position control valve realizes the diversified and adaptive optimization control of the fan coil system, saves the energy consumption of the distribution system and the load energy consumption, and effectively realizes the original intention of the present invention.

[0094] In the above implementation examples, the design parameters of the system can be reasonably determined by comprehensively considering specific usage conditions and requirements, technical and economic performance and other factors to take into account both the applicability and economy of the system.

[0095] Finally, it should be noted that the above examples are only some specific embodiments of the present invention. Obviously, the present invention is not limited to the above embodiments, and there are many variations. All variations that can be directly derived or associated with the content disclosed by a person skilled in the art should be considered as the protection scope of the present invention.

Claims

1. A fan coil control system combined with a multi-position control valve, characterized in that: It includes an upper computer unit and a lower computer unit. The upper computer unit includes a reinforcement learning controller, and the lower computer unit includes a multi-position control valve, a fan coil unit, an equipment status acquisition module, an indoor status acquisition module and a microcontroller unit MCU; The multi-position control valve is arranged on the water inlet circuit, before the fan coil unit and connected in series with the fan coil unit; The multi-position control valve, fan coil unit, equipment status acquisition module and indoor status acquisition module are all connected to the micro-control unit MCU signal, and the micro-control unit MCU is connected to the reinforcement learning controller signal through the network.

2. A fan coil control system in combination with a multi-position control valve according to claim 1, characterized in that: The indoor state acquisition module includes a temperature sensor for acquiring indoor temperature information; The equipment status acquisition module is connected to the multi-position control valve signal to obtain valve position information; The reinforcement learning controller includes a buffer, a policy network, a target network and a reward function. Both the policy network and the target network adopt a DQN network. The reinforcement learning controller sends valve position action instructions to a multi-position control valve through a microcontroller unit MCU.

3. A method for controlling a fan coil unit using the fan coil unit control system according to claim 1 or 2, characterized in that: S1. Obtain the state space and action space of the reinforcement learning controller, where the state space includes the set temperature difference, the gear position of the multi-position control valve and the temperature variation difference, and the action space is the valve position of the multi-position control valve; S2. Set the reward function of the reinforcement learning controller; S3. Initialize network parameters The room and fan coil model are simulated, the simulation time interval and the start and end time of each cycle are set, and the exploration is carried out according to the ε-greedy strategy. A set of state transition samples are obtained through simulation at each step, and the state transition samples are saved in the buffer; the state transition samples include the current state (s i ), action (a i ), reward (r i+1 ) and the next state (s i+1 ); Each time a set of state transition samples is generated, the DQN network is trained to optimize the network parameters. When the training reaches the convergence requirement, the preliminary policy network and target network weight parameters are obtained. S4. Online operation and training Set the ε initial value and decay constant of the ε-greedy strategy; collect and calculate the state transition samples of the current time step, and then send them to the reinforcement learning controller. The DQN network selects random actions with ε probability according to the ε-greedy algorithm and selects the best action predicted by the strategy network with 1-ε probability; feed the generated actions back to the multi-position control valve for online control; then, collect and calculate the state transition samples of the next time step and save them in the buffer; Every time a set of state transition samples is obtained, the policy network is trained online to obtain new policy network weights and update the target network weights.

4. The fan coil control method according to claim 3, characterized in that: The temperature difference Δt i for: Δt i =t i -t i-1 (1)。 5. The fan coil control method according to claim 4, characterized in that: The set temperature difference Δt set，i In cooling condition: Δt set,i =t i -(t set +t band / 2) (2) Under heating conditions: Δt set,i =t i -(t set -t band / 2) (3) Among them, t i is the room temperature obtained by the i-th sampling, t set is the room temperature setting value, t band To control the dead zone, t i-1 is the i-1th room temperature sampling value.

6. The fan coil control method according to claim 5, characterized in that: The gear position of the multi-position control valve is the action value a given by the i-1th instruction i-1 .

7. The fan coil control method according to claim 6, characterized in that: The reward function r i for: r i ＝r temp,i +gr energy,i (4) r energy,i =a max -a i-1 (5) Among them, r energy,i represents the energy consumption reward, γ represents the weight factor of the water energy consumption reward, and a max is the maximum action value; r temp,i Represents room temperature bonus, under cooling conditions: Under heating conditions:

8. The fan coil control method according to claim 7, characterized in that: The buffer adopts a first-in-first-out method.

Images