A ventilation and exhaust coordinated control method and device based on multi-agent system
Through the coordinated control method of ventilation ventilation of multi-intelligent systems, the reward functions and strategies of the blower and exhaust fan are optimized, which solves the problem that traditional control methods cannot cope with complex environment changes, and realizes environmental parameter stability, energy consumption efficiency and response time The improvement ensures the robustness of the system and overall optimal performance.
Patent Information
- Application Number
- CN202410554557.3
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2024-05-07
- Publication Date
- 2024-11-22
- Estimated Expiration
- 2044-05-07
AI Technical Summary
Traditional control methods cannot effectively deal with complex environmental changes. A single blower or exhaust fan cannot meet the requirements of multiple environmental parameters at the same time, and lack considerations for environmental parameter stability, energy consumption efficiency and response time.
Adopt the hybrid ventilation and exhaust control method based on multi-agent system, collect multiple environmental parameters through environmental agents, define and optimize the reward functions of the blower and exhaust fan, and use the Q-learning or Deep Q-Networks algorithm optimization strategy, combined with Pareto Optimize and Nash equilibrium strategies, adjust weights to balance various indicators, and achieve overall optimization of the system.
Improves the stability, energy consumption efficiency and response time of environmental parameters, ensures that the fan and exhaust fans choose the optimal strategy under given conditions, and improves the robustness and overall performance of the system.
Smart Images

Figure CN118224713B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of computer technology, and in particular to a ventilation and exhaust collaborative control method and device based on a multi-agent system. Background Art
[0002] Traditional control methods such as PID control cannot effectively cope with complex environmental changes; moreover, a single supply fan or exhaust fan cannot meet the requirements of multiple environmental parameters at the same time. The prior art lacks consideration of environmental parameter stability, energy efficiency and response time. Therefore, the present invention provides a ventilation and exhaust coordinated control method and device based on a multi-agent system, aiming to improve environmental parameter stability, energy efficiency and response time. Summary of the invention
[0003] The purpose of the present invention is to provide a ventilation and exhaust collaborative control method and device based on a multi-agent system, which can improve the stability of environmental parameters, energy efficiency and response time, and provide a personalized control strategy.
[0004] In a first aspect, the present invention provides a ventilation and exhaust coordinated control method based on a multi-agent system, comprising:
[0005] S1. The environmental agent collects seven environmental parameters, including temperature T, humidity RH, wind speed V, CO 2 Concentration C, PM2.5 concentration P, fan noise N s And exhaust fan noise N p , fan energy consumption E s And exhaust fan energy consumption E p ;
[0006] S2. Define the fan agent reward function R s =(w 11 ×T+w 12 ×RH+w 13 ×V+w 14 ×C+w 15 ×P 16 ×N s -w 17 ×E s )+Δw s , Δw s Reward Δw represents the adjustment of the weight coefficient of the fan agent s , w 11 tow 17 is the weight coefficient, which is used to balance the weights of various indicators;
[0007] S3, exhaust fan agent R p =(w 21 ×C+w 22 ×P23 ×E p -w 24 ×N p )+Δw p , Δw p represents the reward for adjusting the weight coefficient of the exhaust fan agent, w 21 tow 24 is the weight coefficient, which is used to balance the weights of various indicators;
[0008] S4. Use Q-learning or Deep Q-Networks to optimize the reward function R s and R p , find the function that maximizes the reward R s and R p action; implement the ε-greedy strategy, at each decision moment, randomly select an action with probability ε, and select the best action indicated in the Q table or neural network with probability 1-ε; set the learning rate and discount factor;
[0009] S5. Adjust the reward function R based on environmental parameter stability, energy efficiency and response time. s , R p The weight of
[0010] S6. Use Pareto to optimize the reward function and set the reward function R = R s +R p -λ×(||S s -S p || 2 )+Δw c , Δw c represents the weight coefficient adjustment reward during the negotiation process between the supply fan and the exhaust fan, λ is the weight coefficient used to balance the goals of the supply fan and the exhaust fan; ||S s -S p || 2 It is the square of the Euclidean distance between the supply and exhaust fan strategies, taking into account the objectives of both the supply and exhaust fans.
[0011] The method of the present invention has the following beneficial effects:
[0012] A reward function based on environmental parameter stability, energy efficiency and response time is adopted to guide the supply and exhaust fan agents to make decisions that are conducive to environmental parameter stability, high energy efficiency and short response time.
[0013] The Pareto optimization joint reward function is adopted to make a trade-off between the supply fan and exhaust fan agents to achieve the overall optimality of the system.
[0014] Genetic algorithm or particle swarm optimization algorithm is used to effectively find the strategy that maximizes the joint reward function R.
[0015] The Nash equilibrium strategy is adopted to ensure that each agent chooses the optimal strategy given the strategies of other agents.
[0016] Kalman filtering is used to estimate the noise of environmental parameters and user commands to improve the robustness of the system.
[0017] Optionally, the step S1 includes:
[0018] S11, use temperature and humidity sensor, wind speed sensor, CO 2 Sensors, PM2.5 sensors, noise sensors, and electric meters collect 7 environmental parameters to obtain real-time data, including current temperature, current humidity, current wind speed, current CO 2 concentration, current PM2.5 concentration, current noise level and current energy consumption;
[0019] S12. The frequency of data collection is determined according to the rate of environmental change and the requirements of the control strategy; temperature and humidity data are collected every 0.5-2 seconds; CO is collected every 2-10 seconds 2 Concentration data; PM2.5 data is collected every 30-90 seconds; noise data is collected every 30-90 seconds; energy consumption data is collected every 0.5-15 minutes;
[0020] S13, data alignment, for high-frequency data, directly use the original data; for low-frequency data, use the interpolation method to interpolate it to the time point of the high-frequency data; merge the interpolated data with the high-frequency data; and use maximum-minimum scaling or standard deviation scaling for data alignment;
[0021] S14. Store the seven environmental parameters in a relational database; the environmental agent continuously senses the environmental parameters and updates the relational database.
[0022] Optionally, step S2 includes:
[0023] Δw s =k s ×(||w t -w t-1 || 2 ), k s is the adjustment coefficient of the fan agent, which is used to control the adjustment amplitude of its weight coefficient; ||w t -w t-1 || 2is the square of the distance between the current weight coefficient and the weight coefficient at the previous moment; by adjusting the weight coefficient, the fan agent finds the optimal strategy; among them, T, RH, V, C, and P are positive feedbacks, and the higher they are, the more the environment needs to be purified, and the fan needs to work more actively; N s 、E s It is negative feedback. The higher it is, the greater the negative impact of the blower operation, and it should be reduced as much as possible.
[0024] Optionally, step S3 includes:
[0025] The reward function of the exhaust fan agent is more focused on removing pollutants and noise control, R p =(w 21 ×C+w 22 ×P 23 ×E p -w 24 ×N p )+Δw p , Δw p Reward for adjusting the weight coefficient of the exhaust fan agent: Δw p =k p ×(||w t -w t-1 || 2 );k p is the adjustment coefficient of the exhaust fan agent, which is used to control the amplitude of the weight coefficient adjustment. Among them, C and P are positive feedback. The higher the value, the more the environment needs to be purified, and the exhaust fan should work harder. p 、N p It is negative feedback. The higher the value, the lower the exhaust fan efficiency, and the need to reduce energy consumption and noise.
[0026] Optionally, step S4 includes:
[0027] S41. Optimize the reward function and use Q-learning or Deep Q-Networks algorithm to optimize the reward function R s and R p ;
[0028] Q-learning algorithm:
[0029] Initialize the Q table to store the expected reward for performing action a in each state s; select actions according to the ε-greedy strategy, observe the results and update the Q table:
[0030]
[0031] α is the learning rate, γ is the discount factor;
[0032] Deep Q-Networks algorithm:
[0033] A deep neural network is used to estimate the Q function and trained through an experience replay pool; the training goal is to minimize the difference between the predicted Q value and the target Q value:
[0034]
[0035] S42: Implementing the ε-greedy strategy
[0036] At each decision moment, an action is randomly selected with probability ε, and the best action indicated in the Q table or neural network is selected with probability 1-ε; as time goes by, ε gradually decreases and tends to select the optimal action.
[0037] S43: Set the learning rate and discount factor. Use the learning rate to control the speed of updating the Q table or neural network, and the discount factor to control the current value of future rewards.
[0038] Optionally, step S5 includes:
[0039] S51, Environmental parameter stability, calculate standard deviation σ, and evaluate fluctuations:
[0040]
[0041] x i is a single measurement value, μ is the average value, N is the number of measurements, and the standard deviation σ of temperature T and humidity RH is calculated. t and σ h , set the temperature fluctuation range to ±1℃ to ±2℃ with σ t By comparison, the humidity fluctuation range is set to ±3% to ±5% with σ h Compare; if σ t and σ h If the fluctuation range exceeds the set value, the weight w related to temperature T and humidity RH will be increased. 11 and w 12 , reduce volatility;
[0042] S52, energy efficiency, using reward function R s and R p To calculate the ratio of energy consumption of the supply fan and the exhaust fan: Energy efficiency = (R s +R p ) / (E s +E p ), E s and E p The energy consumption of the fan and exhaust fan is set to a target value of energy efficiency greater than 0.7-0.9; if the energy efficiency does not meet the standard, increase the energy consumption E s and E pThe associated negative feedback weight w 17 and w 23 ;
[0043] S53, response time, records the time required from the user's instruction to the environmental parameter reaching the target value: response time = t 目标 -t 指令 , t 目标 , is the time when the environmental parameter reaches the target value; t 指令 is the time when the user command is issued; set the target value of the response time to be less than 30-120 seconds; if the response time does not meet the standard, increase the weight w related to the rate of change of environmental parameters 11 、w 12 and w 13 .
[0044] Optionally, step S6 includes:
[0045] S61, reward function R = R s +R p -λ×(||S s -S p || 2 )+Δw c , Δw c represents the weight coefficient adjustment reward during the negotiation process between the supply fan and the exhaust fan, λ is the weight coefficient used to balance the goals of the supply fan and the exhaust fan; ||S s -S p || 2 is the distance between the supply fan and exhaust fan strategies, reflecting the difference between the two strategies; Δw c =Δw s +Δw p , the weighted adjustment rewards of the fan and exhaust agents themselves, or Δw c =min(Δw s ,Δw p ) Only the reward of the party with better adjustment effect is considered.
[0046] S62, use genetic algorithm or particle swarm algorithm to search for the maximum value of the joint reward function; when PM2.5 concentration or CO 2 When the concentration exceeds the standard, the penalty value is increased;
[0047] S63. Each agent chooses strategy s* as the Nash equilibrium strategy if and only if, given the strategies s* of other agents, for all s' and s'':
[0048] R s (s*,s')≤R s (s*,s*)
[0049] Rp (s*,s”)≤R p (s*,s*)
[0050] For all possible fan strategies s', when the exhaust fan agent strategy is fixed to s*, the reward that the fan agent can obtain by choosing strategy s* will not be lower than choosing any other strategy s';
[0051] For all possible exhaust fan strategies s', when the supply fan agent strategy is fixed to s*, the reward that the exhaust fan agent can obtain by choosing strategy s* will not be lower than choosing any other strategy s'.
[0052] S64, the supply fan and exhaust fan agents exchange the desired air volume and wind speed information, use Deep Q-Networks to iteratively update their respective air volume and wind speed strategies, and repeat the steps until a Nash equilibrium is reached;
[0053] S65, calculating control instructions and sending them to the air supply fan and the exhaust fan;
[0054] S66. Use Kalman filtering to estimate the noise of environmental parameters and user instructions; repeat the steps until the control target is achieved.
[0055] In a second aspect, the present invention further provides a device, comprising:
[0056] Data collection module: responsible for collecting temperature, humidity, wind speed, CO 2 Concentration, PM2.5 concentration, noise level, energy consumption of supply fans and energy consumption of exhaust fans;
[0057] Fan control module: define the reward function R s and optimize air supply strategies;
[0058] Exhaust fan control module: define the reward function R p and optimize ventilation strategies;
[0059] Pareto Optimization Module: Use Pareto optimization to balance the objectives of the supply fan and the exhaust fan and find the optimal strategy R;
[0060] Performance evaluation module: evaluates system performance based on environmental parameter stability, energy efficiency, and response time, and adjusts the weight value in the reward function;
[0061] Data processing module: functions such as determination of data acquisition frequency, data alignment, data scaling and storage;
[0062] Learning strategy module: implements the ε-greedy strategy and the setting of learning rate and discount factor to adjust the importance of future rewards;
[0063] Strategy optimization module: Use genetic algorithm or particle swarm algorithm to search for the maximum value of the joint reward function and calculate the Nash equilibrium strategy;
[0064] Noise estimation module: Use Kalman filtering to estimate the noise of environmental parameters and user commands.
[0065] For the beneficial effects of the second aspect, reference may be made to the description of the first aspect above. BRIEF DESCRIPTION OF THE DRAWINGS
[0066] Figure 1 A schematic diagram of a flow chart of a ventilation and exhaust collaborative control method based on a multi-agent system provided in an embodiment of the present invention;
[0067] Figure 2 A schematic diagram of the structure of a ventilation and exhaust collaborative control device based on a multi-agent system provided in an embodiment of the present invention. DETAILED DESCRIPTION
[0068] In order to make the purpose, technical solutions and advantages of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, rather than all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention. Unless otherwise defined, the technical terms or scientific terms used herein should be understood by people with general skills in the field to which the present invention belongs. "Including" and similar words used in this article mean that the elements or objects appearing before the word include the elements or objects listed after the word and their equivalents, without excluding other elements or objects.
[0069] like Figure 1 As shown, the present invention provides a ventilation and exhaust coordinated control method based on a multi-agent system, comprising:
[0070] S1. The environmental agent collects seven environmental parameters, including temperature T, humidity RH, wind speed V, CO 2 Concentration C, PM2.5 data P, fan noise N s And exhaust fan noise N p , fan energy consumption E s And exhaust fan energy consumption E p ;
[0071] In some embodiments, step S1 includes:
[0072] S11, use temperature and humidity sensor, wind speed sensor, CO 2Sensors, PM2.5 sensors, noise sensors, and electric meters collect 7 environmental parameters to obtain real-time data, including current temperature, current humidity, current wind speed, current CO 2 concentration, current PM2.5 concentration, current noise level and current energy consumption;
[0073] S12. The frequency of data collection is determined according to the rate of environmental change and the control strategy requirements. The frequency of data collection is higher for parameters with faster change rates. Temperature and humidity data are collected every 0.5-2 seconds; CO data is collected every 2-10 seconds. 2 Concentration data; for parameters with slower change rates, the collection frequency is less frequent; PM2.5 concentration data is collected every 30-90 seconds; noise data is collected every 30-90 seconds; energy consumption data is collected every 0.5-15 minutes;
[0074] S13, data alignment, for high-frequency data, use the original data directly; for low-frequency data, use the interpolation method to interpolate it to the time point of the high-frequency data; merge the interpolated data with the high-frequency data; and use maximum-minimum scaling or standard deviation scaling for data alignment;
[0075] S14. Store the seven environmental parameters in a relational database; the environmental agent continuously senses the environmental parameters and updates the relational database.
[0076] S2. Define the reward function R s =(w 11 ×T+w 12 ×RH+w 13 ×V+w 14 ×C+w 15 ×P 16 ×N s -w 17 ×E s )+Δw s , Δw s represents the reward for the fan agent weight coefficient adjustment, Δw s =k s ×(||w t -w t-1 || 2 ), w 11 tow 17 is the weight coefficient, which is used to balance the weights of various indicators; k s is the adjustment coefficient of the fan agent, which is used to control the adjustment amplitude of its weight coefficient; ||w t -w t-1 || 2is the square of the distance between the current weight coefficient and the weight coefficient at the previous moment; by adjusting the weight coefficient, the fan agent finds the optimal strategy. Among them, T, RH, V, C, and P are positive feedbacks. The higher the value, the more the environment needs to be purified, and the fan needs to work more actively; N s 、E s It is negative feedback. The higher it is, the greater the negative impact of the blower operation, and it should be reduced as much as possible.
[0077] S3. Define the exhaust fan agent R p =(w 21 ×C+w 22 ×P 23 ×E p -w 24 ×N p )+Δw p , Δw p represents the reward for adjusting the weight coefficient of the exhaust fan agent, w 21 tow 24 is the weight coefficient, which is used to balance the weights of various indicators;
[0078] In some embodiments, Δw in step S3 p for:
[0079] Δw p =k p ×(||w t -w t-1 || 2 );k p is the adjustment coefficient of the exhaust fan agent, which is used to control the amplitude of the weight coefficient adjustment; among them, C and P are positive feedback, and the higher the value, the more the environment needs to be purified, and the exhaust fan should work harder; E p 、N p It is negative feedback. The higher the value, the lower the exhaust fan efficiency, and the need to reduce energy consumption and noise.
[0080] S4. Use Q-learning or Deep Q-Networks to optimize the reward function R s , R p , find the function that maximizes the reward R s and R p action; implement the ε-greedy strategy, at each decision moment, randomly select an action with probability ε, and select the best action indicated in the Q table or neural network with probability 1-ε; set the learning rate and discount factor;
[0081] In some embodiments, step S4 includes:
[0082] S41. Optimize the reward function and use Q-learning or Deep Q-Networks algorithm to optimize the reward function R s and R p ;
[0083] Q-learning algorithm:
[0084] Initialize the Q table to store the expected reward for performing action a in each state s; select actions according to the ε-greedy strategy, observe the results and update the Q table:
[0085]
[0086] α is the learning rate, γ is the discount factor;
[0087] Deep Q-Networks algorithm:
[0088] A deep neural network is used to estimate the Q function and trained through an experience replay pool; the training goal is to minimize the difference between the predicted Q value and the target Q value:
[0089]
[0090] S42: Implementing the ε-greedy strategy
[0091] At each decision moment, an action is randomly selected with probability ε, and the best action indicated in the Q table or neural network is selected with probability 1-ε; as time goes by, ε gradually decreases and tends to select the optimal action.
[0092] S43: Set the learning rate and discount factor. Use the learning rate to control the speed of updating the Q table or neural network, and the discount factor to control the current value of future rewards.
[0093] S5. Adjust the reward function R based on environmental parameter stability, energy efficiency and response time. s , R p The weight of
[0094] In some embodiments, step S5 includes:
[0095] S51. Calculate the standard deviation σ based on the stability of environmental parameters and evaluate the fluctuation:
[0096]
[0097] x i is a single measurement value, μ is the average value, N is the number of measurements, and the standard deviation σ of temperature T and humidity RH is calculated. t and σ h , set the temperature T fluctuation range to ±1℃ to ±2℃ with σt By comparison, the humidity fluctuation range is set to ±3% to ±5% with σ h Compare; if σ t and σ h If the fluctuation range exceeds the set value, the weight w related to temperature T and humidity RH will be increased. 11 and w 12 , reduce volatility;
[0098] S52, energy efficiency, using reward function R s and R p To calculate the ratio of energy consumption of the supply fan and the exhaust fan: Energy efficiency = (R s +R p ) / (E s +E p ), E s and E p The energy consumption of the fan and exhaust fan is set to a target value of energy efficiency greater than 0.7-0.9; if the energy efficiency does not meet the standard, increase the energy consumption E s and E p The associated negative feedback weight w 17 and w 23 ;
[0099] S53, response time, records the time required from the user's instruction to the environmental parameter reaching the target value: response time = t 目标 -t 指令 , t 目标 , is the time when the environmental parameter reaches the target value; t 指令 , is the time when the user command is issued; set the target value of the response time to be less than 30-120 seconds; if the response time does not meet the standard, increase the weight w related to the rate of change of environmental parameters 11 、w 12 and w 13 .
[0100] S6. Use Pareto to optimize the reward function and set the reward function R = R s +R p -λ×(||S s -S p || 2 )+Δw c , Δw c represents the weight coefficient adjustment reward during the negotiation process between the supply fan and the exhaust fan, λ is the weight coefficient used to balance the goals of the supply fan and the exhaust fan; ||S s -S p || 2 It is the square of the Euclidean distance between the supply and exhaust fan strategies, taking into account the objectives of both the supply and exhaust fans.
[0101] In some embodiments, step S6 includes:
[0102] S61. Define the joint reward function R = R s +R p -λ×(||S s -S p || 2 )+Δw c , Δw c represents the weight coefficient adjustment reward during the negotiation process between the supply fan and the exhaust fan, λ is the weight coefficient used to balance the goals of the supply fan and the exhaust fan; ||S s -S p || 2 is the square of the Euclidean distance between the supply fan and exhaust fan strategies, reflecting the difference between the two strategies; Δw c =Δw s +Δw p , the weighted adjustment rewards of the fan and exhaust agents themselves, or Δw c =min(Δw s ,Δw p ), only the reward of the party with better adjustment effect is considered.
[0103] S62, use genetic algorithm or particle swarm algorithm to search for the maximum value of the joint reward function; when PM2.5 concentration or CO 2 When the concentration exceeds the standard, the penalty value is increased;
[0104] S63. Each agent chooses strategy s* as the Nash equilibrium strategy if and only if, given the strategies s* of other agents, for all s' and s'':
[0105] R s (s*,s')≤R s (s*,s*)
[0106] R p (s*,s”)≤R p (s*,s*)
[0107] S64, the supply fan and exhaust fan agents exchange the desired air volume and wind speed information, use Deep Q-Networks to iteratively update their respective air volume and wind speed strategies, and repeat the steps until a Nash equilibrium is reached;
[0108] S65, calculating control instructions and sending them to the air supply fan and the exhaust fan;
[0109] S66. Use Kalman filtering to estimate the noise of environmental parameters and user instructions; repeat the steps until the control target is achieved.
[0110] Compared with the prior art, the advantages of the present invention are:
[0111] Improved the stability of environmental parameters: By adjusting the weight of the reward function, the control accuracy of environmental parameters such as temperature and humidity is improved, making the environmental parameters more stable.
[0112] Improved energy efficiency: The present invention effectively improves the energy efficiency of the system by calculating the energy consumption ratio of the supply fan and the exhaust fan and taking it as part of the reward function.
[0113] Shortened response time: The time required to reach the target value is included as part of the reward function, which effectively shortens the response time of the system.
[0114] Improved system robustness: Using the Nash equilibrium strategy ensures that each agent chooses the optimal strategy given the strategies of other agents, thereby improving the robustness of the system.
[0115] Table 1 Sensor types and data collection
[0116] sensor Collecting data Data Types unit Data Description Collection frequency Temperature and humidity sensor temperature T ℃ Indoor temperature 0.5-2 seconds Temperature and humidity sensor humidity RH % Indoor humidity 0.5-2 seconds Wind speed sensor Wind speed V m / s Indoor wind speed 2-10 seconds CO2 Sensor CO2 concentration C ppm Indoor CO2 concentration 2-10 seconds PM2.5 Sensor PM2.5 concentration P <![CDATA[μg / m 3 ]]> Indoor PM2.5 concentration 30-90 seconds Noise Sensor Noise level N dB Indoor noise level 30-90 seconds Electricity Meter Energy consumption E kWh Energy consumption of supply and exhaust fans 0.5-15 minutes
[0117] For ease of understanding, this embodiment further describes the specific implementation method of the above method in combination with a specific application scenario system. Taking the robotic arm production scenario as an example, it specifically includes the following steps:
[0118] In step a, the environmental agent collects temperature, humidity, wind speed, CO2 concentration, PM2.5, noise, and energy consumption data in real time through sensors. These data are used to monitor the production environment and ensure that the robot arm operates under optimal conditions.
[0119] In step b, the goal of the fan agent is to adjust the air speed and direction to maintain temperature and humidity within the optimal range while minimizing noise and energy consumption. The reward function takes these factors into account and optimizes the performance of the fan through weight adjustments.
[0120] In step c, the exhaust fan agent is responsible for maintaining air quality by controlling CO2 and PM2.5 concentrations while reducing energy consumption and noise. Its reward function is also adjusted based on real-time data of environmental parameters.
[0121] In step d, using the Q-learning algorithm, the agent learns how to adjust its behavior according to environmental changes to maximize its reward function. The ε-greedy strategy ensures that the agent maintains a balance between exploring new strategies and exploiting known strategies.
[0122] In step e, based on the stability of environmental parameters, energy efficiency, and response time, the agent adjusts the weights in its reward function to improve performance.
[0123] In step f, through Pareto optimization, the agent finds an equilibrium point that satisfies both the objectives of the supply fan and the exhaust fan, while taking into account the difference in strategies between the two.
[0124] The above-mentioned embodiments can further improve the efficiency and reliability of the ventilation and exhaust collaborative control system of the multi-agent system, and provide users with high-quality services and support.
[0125] like Figure 2 As shown, a ventilation and exhaust coordination control device 1 based on a multi-agent system comprises:
[0126] Data collection module 101: responsible for collecting data on temperature, humidity, wind speed, CO2 concentration, PM2.5 concentration, noise level, energy consumption of the supply fan and energy consumption of the exhaust fan;
[0127] The fan control module 102 defines a reward function Rs and optimizes the air supply strategy;
[0128] Exhaust fan control module 103: define reward function Rp and optimize exhaust strategy;
[0129] Pareto optimization module 104: using Pareto optimization to balance the objectives of the supply fan and the exhaust fan and find the best strategy R;
[0130] Performance evaluation module 105: evaluates system performance according to environmental parameter stability, energy efficiency and response time, and adjusts the weight value in the reward function;
[0131] Data processing module 106: functions such as determination of data acquisition frequency, data alignment, data scaling and storage;
[0132] Learning strategy module 107: implements the ε-greedy strategy and the setting of learning rate and discount factor to adjust the importance of future rewards;
[0133] Strategy optimization module 108: using a genetic algorithm or a particle swarm algorithm to search for the maximum value of the joint reward function and calculate the Nash equilibrium strategy;
[0134] Noise estimation module 109: uses Kalman filtering to estimate the noise of environmental parameters and user instructions.
[0135] Although the embodiments of the present invention are described in detail above, it is obvious to those skilled in the art that various modifications and variations can be made to these embodiments. However, it should be understood that such modifications and variations are within the scope and spirit of the present invention as described in the claims. Moreover, the present invention described herein may have other embodiments and may be implemented or realized in a variety of ways.
Claims
1. A ventilation and exhaust collaborative control method based on a multi-agent system, characterized in that: include: S1. The environmental agent collects seven environmental parameters, including temperature T, humidity RH, wind speed V, CO2 concentration C, PM2.5 concentration P, and fan noise N. s And exhaust fan noise N p , fan energy consumption E s And exhaust fan energy consumption E p ; S2. Define the fan agent reward function R s =(w 11 ×T+w 12 ×RH+w 13 ×V+w 14 ×C+w 15 ×P 16 ×N s -w 17 ×E s )+Δw s , Δw s represents the reward for the fan agent weight coefficient adjustment, Δw s =k s ×(||w t -w t-1 || 2 ), k s is the adjustment coefficient of the fan agent, which is used to control the adjustment amplitude of its weight coefficient; ||w t -w t-1 || 2 is the square of the distance between the current weight coefficient and the weight coefficient at the previous moment; by adjusting the weight coefficient, the fan agent finds the optimal strategy; w 11 tow 17 is the weight coefficient, which is used to balance the weights of various indicators; S3. Define the exhaust fan agent reward function R p =(w 21 ×C+w 22 ×P 23 ×E p -w 24 ×N p )+Δw p , Δw p represents the reward for adjusting the weight coefficient of the exhaust fan agent, Δw p =k p ×(||w t -w t-1 || 2 );k p is the adjustment coefficient of the exhaust fan agent, which is used to control the adjustment amplitude of its weight coefficient; 21 tow 24 is the weight coefficient, which is used to balance the weights of various indicators; S4. Use Q-learning or Deep Q-Networks to optimize the reward function R s , R p , find the function that maximizes the reward R s and R p action; implement the ε-greedy strategy, at each decision moment, randomly select an action with a probability of ε, and select the best action indicated in the Q table or neural network with a probability of 1-ε; the Q table is a table that stores Q values, which indicates the expected value of the long-term cumulative reward of the agent performing an action in the current state and following the optimal strategy; the higher the Q value, the higher the long-term return of the action; set the learning rate and discount factor; S5. Adjust the reward function R based on environmental parameter stability, energy efficiency and response time. s , R p The weight of S6. Use Pareto to optimize the reward function and set the joint reward function R = R s +R p -λ×(||S s -S p || 2 )+Δw c , Δw c represents the weight coefficient adjustment reward during the negotiation process between the supply fan and the exhaust fan, Δw c =Δw s +Δw p or Δw c =min(Δw s ,Δw p ); λ is the weight coefficient used to balance the objectives of the supply fan and the exhaust fan; ||S s -S p || 2 It is the square of the Euclidean distance between the supply and exhaust fan strategies, taking into account the objectives of both the supply and exhaust fans.
2. The method according to claim 1, characterized in that The step S1 comprises: S11. Use temperature and humidity sensors, wind speed sensors, CO2 sensors, PM2.5 sensors, noise sensors, and electric meters to collect seven environmental parameters and obtain real-time data, including current temperature, current humidity, current wind speed, current CO2 concentration, current PM2.5 concentration, current noise level, and current energy consumption; S12. The frequency of data collection is determined according to the rate of environmental change and the requirements of the control strategy; temperature data and humidity data are collected every 0.5-2 seconds; CO2 concentration data are collected every 2-10 seconds; PM2.5 concentration data and noise data are collected every 30-90 seconds; energy consumption data are collected every 0.5-15 minutes; S13, data alignment, for high-frequency data, directly use the original data; for low-frequency data, use the interpolation method to interpolate it to the time point of the high-frequency data; merge the interpolated data with the high-frequency data; and use maximum-minimum scaling or standard deviation scaling for data alignment; S14. Store the seven environmental parameters in a relational database; the environmental agent continuously senses the environmental parameters and updates the relational database.
3. The method according to claim 1, characterized in that: The step S5 comprises: S51. Calculate the standard deviation σ based on the stability of environmental parameters and evaluate the fluctuation: x i is a single measurement value, μ is the average value, N is the number of measurements, and the standard deviation σ of temperature T and humidity RH is calculated. t and σ h , set the temperature T fluctuation range to ±1℃ to ±2℃ with σ t By comparison, the humidity RH fluctuation range is set to ±3% to ±5% with σ h Compare; if σ t and σ h If the fluctuation range exceeds the set value, the weight w related to temperature T and humidity RH will be increased. 11 and w 12 , reduce volatility; S52, energy efficiency, using reward function R s and R p To calculate the ratio of energy consumption of the supply fan and the exhaust fan: Energy efficiency = (R s +R p ) / (E s +E p ), E s and E p The energy consumption of the fan and exhaust fan is set to a target value of energy efficiency greater than 0.7-0.9; if the energy efficiency does not meet the standard, increase the energy consumption E s and E p The associated negative feedback weight w 17 and w 23 ; S53, response time, records the time required from the user's instruction to the environmental parameter reaching the target value: response time = t 目标 -t 指令 , t 目标 is the time when the environmental parameter reaches the target value; t 指令 is the time when the user command is issued; set the target value of the response time to be less than 30-120 seconds; if the response time does not meet the standard, increase the weight w related to the rate of change of environmental parameters 11 、w 12 and w 13 .
4. The method according to claim 1, characterized in that: The step S6 comprises: S61, Δw c =Δw s +Δw p , the weighted adjustment rewards of the fan and exhaust agents themselves, or Δw c =min(Δw s ,Δw p ), only the reward of the party with better adjustment effect is considered; S62, using a genetic algorithm or a particle swarm optimization algorithm to find a strategy to maximize the joint reward function R; when the PM2.5 concentration or the CO2 concentration exceeds a predetermined threshold, increasing the penalty value; S63. Define the Nash equilibrium strategy s* so that it holds for all s' and s" given the strategies s* of other agents. This ensures that each agent chooses the optimal strategy given the strategies of other agents: R s (s*,s')≤R s (s*,s*) R p (s*,s”)≤R p (s*,s*) S64, the supply fan and exhaust fan agents exchange the desired air volume and wind speed information and use Deep Q-Networks to iteratively update their respective strategies, repeating the steps until a Nash equilibrium is reached; S65, calculating control instructions and sending them to the air supply fan and the exhaust fan; S66. Use Kalman filtering to estimate the noise of environmental parameters and user instructions; repeat the steps until the control target is achieved.
5. A ventilation and exhaust coordination control device based on a multi-agent system, characterized in that: For the method according to claim 1, the device comprises: Data collection module: responsible for collecting data on temperature, humidity, wind speed, CO2 concentration, PM2.5 concentration, noise level, energy consumption of the supply fan and the energy consumption of the exhaust fan; Fan control module: define the reward function R s And optimize the air supply strategy; comprehensively consider T, RH, V, C, P, Ns, Es, and introduce weight coefficients to balance the influence of various factors, and maximize Rs through Q-learning or DQN algorithm; Exhaust fan control module: define the reward function R p And optimize the exhaust strategy; focus on C, P, Ep, Np, and use Q-learning or Deep Q-Networks algorithm to maximize Rp; Pareto optimization module: Use Pareto optimization to balance the goals of the supply fan and the exhaust fan and find the best strategy R; perform a weighted combination of the reward functions of the supply fan and the exhaust fan, introduce a penalty term to constrain the conflict between the two, and use the Pareto optimization method to find the best strategy R; Performance evaluation module: evaluates system performance based on environmental parameter stability, energy efficiency, and response time, and adjusts the weight value in the reward function; Data processing module: functions such as determination of data acquisition frequency, data alignment, data scaling and storage; Learning strategy module: implements the ε-greedy strategy and the setting of learning rate and discount factor to adjust the importance of future rewards; Strategy optimization module: Use genetic algorithm or particle swarm algorithm to search for the maximum value of the joint reward function and calculate the Nash equilibrium strategy; Noise estimation module: Use Kalman filtering to estimate the noise of environmental parameters and user commands.
Citation Information
Patent Citations
Intelligent control method and device for constant humidity machine
CN114110824A
Central air conditioner control method and control system based on reinforcement learning
CN114234381A
Multi-target refrigeration side water supply temperature control optimization method based on Q-learning
CN116592546A
Energy-saving optimization control method of dehumidification unit system based on reinforcement learning
CN117190405A
Fresh air control system and method based on Internet of Things
CN117663390A
