Intelligent air flow control system based on deep Q network

Abstract

The invention belongs to the technical field of air flow control, and particularly relates to an intelligent air flow control system based on a deep Q network, and the system comprises a sensing layer which is provided with a high-definition camera and a particle sensor, fuses time information, generates a high-dimensional state vector, and carries out the preprocessing; the decision-making layer is used for deploying a decision-making model based on an Actor-Critic framework, receiving the high-dimensional state vector preprocessed by the sensing layer and generating a continuous rotating speed control instruction of the FFU; the execution layer is connected with the decision-making layer and adopts a multi-agent architecture, each agent performs communication and information interaction through a graph neural network so as to cooperatively control the rotating speed of each FFU, and meanwhile, the execution layer is provided with an edge reasoning module so as to realize distributed control; and the optimization layer is connected with the decision-making layer and the execution layer and is used for adjusting parameters of a decision-making model through a dynamic reward function according to the concentration change rate of the particulate matters and the microorganisms and the energy consumption data of the FFU. According to the invention, efficient, intelligent and energy-saving control of the environment of the clean room is realized.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of air flow control, and particularly relates to an intelligent air flow control system based on a deep Q network. BACKGROUND

[0002] In the field of clean room intelligent control technology, traditional FFU (Fan Filter Unit) speed control relies on discrete action space (such as fixed gear adjustment), which has defects such as insufficient control accuracy and delayed response, and is difficult to adapt to dynamic scenarios such as load fluctuations and particle concentration mutations. In modern industrial production, the application range of clean rooms is expanding from semiconductor manufacturing to biopharmaceutical production, and the requirements for air cleanliness are becoming increasingly stringent. However, traditional FFU (Fan Filter Unit) speed control usually uses fixed gear adjustment, which not only cannot achieve precise speed adjustment, but also causes system response delay, making it difficult to quickly respond to load fluctuations and particle concentration mutations caused by personnel activities, goods movement and external environmental changes in the clean room. This not only affects production efficiency and product quality, but also increases production costs and energy consumption.

[0003] In the face of high-dimensional data generated by the expansion of sensor array scale, traditional full-quantity feature processing methods are prone to cause "dimension disaster", resulting in exponential growth of computational complexity, which seriously affects the operation efficiency of decision-making models. With the rapid development of sensing technology, the number of sensors deployed in clean rooms has increased dramatically, allowing real-time collection of various data including particle concentration, three-dimensional position and motion parameters of personnel and goods, temperature, humidity, etc. Although these data provide rich information for precise control of air flow, they also pose great challenges to data processing and decision-making models. Traditional data processing methods often need to perform tedious feature extraction and dimension reduction when facing high-dimensional data, which not only increases the computational burden, but also may cause loss of key information, thereby affecting the accuracy and real-time performance of the control strategy.

[0004] In addition, in part of the reinforcement learning control strategy, the reward function is insufficient in the fusion evaluation of pollution control and energy saving goals, and the exploration strategy lacks dynamic adaptability, which cannot achieve effective balance between "exploration" and "utilization", further restricting the stability and efficiency of the control strategy. When designing the reward function, the traditional reinforcement learning method often has difficulty in balancing the two contradictory goals of pollution control and energy saving, resulting in the system consuming a large amount of energy while pursuing high cleanliness, or failing to respond to pollution changes in energy-saving mode. At the same time, the fixed exploration strategy cannot adjust the exploration intensity according to the dynamic changes of the environment, resulting in low learning efficiency of the model in the complex and variable clean room environment, and it is difficult to achieve the ideal control effect. Therefore, it is imminent to develop an intelligent air flow control system that can efficiently process high-dimensional data, dynamically balance pollution control and energy saving goals, and has self-adaptive exploration ability. SUMMARY

[0005] In view of the deficiencies in the prior art above, the purpose of the present application is to provide an intelligent air flow control system based on deep Q network, which realizes efficient, intelligent and energy-saving control of clean room environment.

[0006] To achieve the above purpose, the present application provides an intelligent air flow control system based on deep Q network, comprising: a perception layer configured with a high-definition camera and a particle sensor, which collects three-dimensional space information of personnel / goods in the clean room, personnel / goods motion parameters, cargo volume, particulate matter / microorganism concentration data, fan filter unit (FFU) rotation speed, fuses time information to generate a high-dimensional state vector, and pre-processes the high-dimensional state vector; a decision layer deploying a decision model based on an Actor-Critic framework, which receives the pre-processed high-dimensional state vector from the perception layer and generates a continuous rotation speed control instruction for the FFU; an execution layer connected to the decision layer, which adopts a multi-agent architecture, each agent communicates and exchanges information through a graph neural network to cooperatively control the rotation speed of each FFU, and has an edge reasoning module to realize distributed control; an optimization layer connected to the decision layer and the execution layer, which adjusts the parameters of the decision model through a dynamic reward function according to the concentration change rate of particulate matter and microorganisms and the energy consumption data of the FFU.

[0007] As a preferred scheme of the present application, in the perception layer, n monitoring points, m monitored personnel, k goods and q FFUs are provided in the clean room, and the high-dimensional state vector contains the following data: concentration data C1, C2, …, Cn of the n monitoring points n wherein each data point contains the concentration values of particulate matter and microorganisms at the monitoring point; State information of m monitored persons, including three-dimensional position information of the monitored persons and motion direction and speed information ; State information of k goods, including three-dimensional position information of the goods , motion direction and speed information , and volume information V1, V2, …, V k ; Current rotating speed information F1, F2, …, F q of q FFUs Time step Δt Particle concentration change rate Δc and microorganism concentration change rate Δb

[0008] As a preferred scheme of the present application, in the perception layer, for the high-dimensional state vector, first, preliminary preprocessing of data cleaning, missing value processing, and data standardization is performed, and then, key feature screening based on SHAP value and data compression based on dynamic PCA are used to perform feature screening and dimensionality reduction processing on the preliminary preprocessed high-dimensional state vector.

[0009] As a preferred scheme of the present application, in the decision layer, in the decision model based on the Actor-Critic framework, an action policy is generated by the Actor network, and an action value is evaluated by the Critic network The Actor network adopts a fully connected neural network to map the FFU rotating speed adjustment range to the normalized interval [0, 100], corresponding to 0%-100% rated rotating speed; the Critic network evaluates the Q value based on the state-action pair, which is used to quantitatively evaluate the expected cumulative reward of the Actor network executing the action a in the state s, and the action a is the rotating speed action value of the FFU, so as to generate a continuous rotating speed control instruction of the FFU, and a mean square error loss function is used to optimize the network parameters A deterministic gradient strategy is used to train the decision model based on the Actor-Critic framework.

[0010] As a preferred scheme of the present application, for the decision model of the Actor-Critic framework, a rotating speed change rate constraint and a safety interval constraint are set, specifically: Rotating speed change rate constraint: For the rotating speed change rate ΔF of the FFU, a change rate threshold F max_delta is set, and a clipping operation is performed on the output of the Actor network to realize the constraint: ; In the formula, represents the clipped rotating speed action value; clip represents the clipping operation This represents the rotational speed action value output by the Actor network; This indicates the rotational speed value at the previous moment; based on Ensure that the current rate of change of rotational speed meets the requirements. ; Safe range constraints: Define the safe speed range {F min F max},right Perform secondary cutting: ; In the formula, Indicates the rotational speed value after secondary cutting; F min F max These represent the lower and upper limits of the rotational speed, respectively. The rotational speed value is cut to a safe rotational speed range to ensure that the action is performed within the safe operating range of the equipment.

[0011] As a preferred embodiment of the present invention, in the execution layer, the multi-agent architecture employs a greedy strategy to select the action with the largest Q-value for execution, and introduces an adaptive exploration strategy to incorporate the greedy strategy... The value is related to the pollution change rate ΔC, where ΔC is the particulate matter concentration change rate Δc or the microbial concentration change rate Δb. The formula for adjusting the value is: ; In the formula, , They are respectively The minimum and maximum values; To control The coefficient of the rate of change; The set threshold; At each time step, calculate based on the current pollution change rate. Value, in The probability of randomly selecting an action, in order to The probability is to choose the action with the highest Q value.

[0012] As a preferred embodiment of the present invention, the edge inference module in the execution layer includes: Edge nodes are used to deploy quantization models and process data collected by sensors in real time. Fog nodes are used to store historical experience data and periodically update the decision-making model; A multi-agent system is responsible for global coordination and fault tolerance.

[0013] As a preferred embodiment of the present invention, in the optimization layer, the dynamic reward function Rt The concentration recovery reward function R1 and the energy saving reward function R2 are included; R1 is expressed as: ; In the formula, is a positive weight coefficient; ΔC represents the pollution change rate, which is the particulate matter concentration change rate Δc or the microorganism concentration change rate Δb; is a constant to avoid zero denominator; Δt is the time step; R2 is expressed as: ; In the formula, represents the weight coefficient of energy saving reward; represents the penalty weight when the energy consumption exceeds the benchmark; represents the environmental state factor; represents the time factor; represents the energy consumption at the current time; represents the benchmark energy consumption; R t is expressed as: ; The energy consumption refers to the electric power consumed by the FFU when it is running.

[0014] As a preferred scheme of the present application, the environmental state factor reflects the activity level of personnel and goods, and is calculated as follows: ; In the formula, k1 is a positive coefficient; represents the average movement speed of personnel and goods; represents the maximum allowed speed; represents the average volume of goods; represents the maximum allowed volume; The time factor adjusts the priority of energy saving according to different time periods, and is expressed as: ; In the formula, k2 is a positive adjustment coefficient; represents the current time; represents the peak period of the day.

[0015] The algorithm involved in the present application can be executed by an electronic device, which includes a memory, a processor, and a computer program stored on the memory and executable on the processor, and the above-mentioned algorithm calculation is realized by executing the software on the processor.

[0016] The present application has the beneficial effects that: This invention employs a Deep Q-Network (DQN) and multi-agent architecture, combining high-definition camera and particle sensor data to generate high-dimensional state vectors. It utilizes SHAP-PCA joint dimensionality reduction technology to screen key features, effectively solving the challenges of high-dimensional data processing and avoiding the "curse of dimensionality." Simultaneously, it improves computational efficiency and response speed, ensuring rapid adjustment of FFU rotation speed in response to changes in particulate matter and microbial concentrations, achieving rapid recovery of pollutant concentrations. Its millisecond-level real-time response capability can promptly address unexpected situations within the cleanroom, ensuring a stable and clean air environment. Employing a real-time data-driven dynamic control strategy, compared to traditional fixed-speed or simple feedback control methods, it can more accurately respond to dynamic environmental changes, avoiding energy waste caused by excessive FFU operation.

[0017] This invention introduces a dynamic reward function that comprehensively considers concentration recovery rate and energy-saving effect, adjusting the exploration strategy based on the pollution change rate to achieve a balance between pollution control and energy-saving goals. This function incentivizes the model to reduce energy consumption while meeting purification requirements through both concentration recovery rewards and energy-saving rewards. It employs an adaptive greedy exploration strategy, dynamically adjusting the exploration probability based on the current state to balance "exploration" and "utilization," thereby improving the stability and efficiency of the control strategy. Simultaneously, the system features a distributed edge computing architecture. Through a three-tiered architecture of the edge inference module, it achieves real-time data processing and periodic model updates, optimizing long-term operational performance and ensuring efficient and stable operation in various cleanroom scenarios. Attached Figure Description

[0018] Figure 1 This is a system architecture diagram of the present invention; Figure 2 This is a schematic diagram of the Actor-Critic network architecture in this invention; Figure 3 This is a diagram of the multi-agent collaborative control and edge inference architecture in this invention. Detailed Implementation

[0019] The embodiments of the present invention will be further described below with reference to the accompanying drawings: Example 1: As Figure 1 As shown, an intelligent airflow control system based on a deep Q-network includes: The perception layer is equipped with high-definition cameras and particle sensors to collect three-dimensional spatial information of personnel / goods in the clean room, personnel / goods motion parameters, goods volume, particulate matter / microorganism concentration data, and fan filter unit (FFU) rotation speed. It integrates time information to generate a high-dimensional state vector and preprocesses the high-dimensional state vector. The decision-making layer deploys a decision model based on the Actor-Critic framework, receives the high-dimensional state vector preprocessed by the perception layer, and generates continuous speed control commands for the FFU. The execution layer is connected to the decision layer and adopts a multi-agent architecture. Each agent communicates and exchanges information through a graph neural network to cooperatively control the rotation speed of each FFU. The edge inference module is also provided to realize distributed control. The optimization layer is connected to the decision layer and the execution layer. According to the concentration change rates of particulate matter and microorganisms and the energy consumption data of the FFU, the parameters of the decision model are adjusted through a dynamic reward function.

[0020] In the perception layer, there are n monitoring points, m monitored personnel, k goods, and q FFUs in the clean room. The high-dimensional state vector includes the following data: The concentration data C1, C2, …, Cn of the n monitoring points n Each data point contains the concentration values of particulate matter and microorganisms at the monitoring point, which are obtained through particle sensors. The state information of the m monitored personnel, including three-dimensional position information and motion direction and speed information of the monitored personnel, which are obtained through high-definition cameras. The state information of the k goods, including three-dimensional position information , motion direction and speed information , and volume information V1, V2, …, Vk of the goods k , which are obtained through high-definition cameras (volume can be obtained based on active binocular stereo imaging RGBD cameras, multi-view camera systems, etc.). The current rotation speed information F1, F2, …, Fq of the q FFUs q , which are obtained through rotation speed sensors. The time step Δt of the current decision cycle. The particulate matter concentration change rate Δc and the microorganism concentration change rate Δb, which can be calculated by existing methods.

[0021] All the above data form a high-dimensional state vector State.

[0022] The equipment configuration of the clean room has significant differences (e.g., the number of FFUs can vary from tens to thousands). Although the modular design of State provides good scalability, directly processing full-dimensional high-dimensional feature data will cause "dimension disaster", leading to exponential growth of computational complexity, which severely restricts the operation efficiency and response speed of the model. Based on this, a two-stage dimension reduction strategy is proposed to simplify the state space variables.

[0023] In the perception layer, for the high-dimensional state vector, first, preliminary preprocessing such as data cleaning, missing value processing, and data standardization is performed, and then, key feature screening based on SHAP values and data compression based on dynamic PCA are used to screen and reduce the dimension of the high-dimensional state vector after preliminary preprocessing.

[0024] First, the SHAP (Shapley Additive exPlanations) value is used to quantitatively analyze the contribution of each state variable to the model decision, and by setting a threshold, key features with strong correlation are screened out, redundant information is removed, and the state dimension is preliminarily reduced. Then, data compression based on dynamic PCA is performed, and for the screened high-dimensional key feature set, dynamic principal component analysis (PCA) technology is further introduced. By analyzing the data eigenvalues and variances, high-dimensional data is mapped to a low-dimensional space, which effectively improves the operation efficiency and response speed of the subsequent decision model while maximizing the retention of core information. By retaining the first few principal components, the cumulative variance ratio is ensured to be ≥95%, and the data dimension is compressed with minimal information loss.

[0025] For the dynamic characteristics of clean room FFU speed control, traditional discrete action space (such as fixed gear adjustment) has inherent defects such as insufficient control precision and response delay. To overcome the above limitations, a continuous action space design is adopted, which maps the FFU speed adjustment range to the normalized interval [0, 100] (corresponding to 0%-100% rated speed), achieving fine control. This continuous action space not only ensures physical feasibility but also significantly reduces the exploration complexity of the reinforcement learning model, enabling the system to maintain stable response under dynamic scenarios such as clean room load fluctuations and particle concentration mutations.

[0026] As shown in Figure 2 In the decision layer, in the decision model based on the Actor-Critic framework, the Actor network generates action strategies, and the Critic network evaluates action values. The Actor network uses a fully connected neural network to map the FFU speed adjustment range to the normalized interval [0, 100], corresponding to 0%-100% rated speed. The Critic network evaluates Q values based on state-action pairs, which are used to quantitatively evaluate the expected cumulative reward of the Actor network executing action a in state s. Action a is the FFU speed action value, thus generating the continuous speed control instruction of the FFU, and the mean square error loss function is used to optimize the network parameters. The Actor network performs linear transformation and activation function operation through the first layer of full connection layer to realize preliminary feature extraction, and the second layer of full connection layer further extracts high-order features, and finally the output layer linear transformation, hyperbolic tangent function processing and multiplication by 100 are performed to output the original action value. The value represents the target percentage of FFU speed, and the range is limited to [0, 100].

[0027] The Critic network takes the vector obtained by splicing the action a and the state s output by the Actor network as input, first performs linear transformation on the input through the first layer of full connection layer, and extracts preliminary features through activation function processing. The second layer of full connection layer further performs linear transformation and activation operation on the output features of the previous layer to deepen the feature abstraction. Finally, the output layer linear transformation is performed to output the Q value of the state-action pair. The Q value is used to quantitatively evaluate the expected cumulative reward of the Actor network executing the action a in the state s, and provides a value feedback basis for the optimization of the action strategy of the Actor network, forming a closed-loop optimization mechanism of "Actor generates action-Critic evaluates value-Actor optimizes strategy".

[0028] The deterministic gradient strategy is adopted to train the decision model based on the Actor-Critic framework.

[0029] For the decision model of the Actor-Critic framework, the speed change rate constraint and the safety interval constraint are set, specifically: Speed change rate constraint: For the speed change rate ΔF of the FFU, the change rate threshold F max_delta is set, and the output of the Actor network is clipped to realize the constraint: ; In the formula, represents the clipped speed action value; clip represents the clipping operation; represents the speed action value output by the Actor network; represents the speed action value at the previous moment; Based on , it is ensured that the current speed change rate satisfies ; Safety interval constraint: Define the safety speed interval {F min , F max}, and perform secondary clipping on : ; In the formula, represents the secondary clipped speed action value; F min , F maxrespectively represent the lower limit and the upper limit of the rotation speed; The rotation speed action value is clipped to the safe rotation speed interval to ensure that the execution action is within the safe operation range of the equipment.

[0030] The action space constraint algorithm including the rotation speed change rate constraint and the safe interval constraint can perform dual constraint processing on the original action value output by the Actor: first, the rotation speed change rate constraint is implemented by clipping to ensure that the rotation speed change rate meets certain requirements and prevent the FFU rotation speed from suddenly changing and damaging the equipment; second, a safe rotation speed interval (such as 30%-80% of the rated rotation speed) is defined, and a secondary clipping is implemented using a formula to limit the rotation speed action value within the safe interval, which ensures the safe operation of the equipment while meeting the clean room air quality control and energy saving requirements.

[0031] After generating the safe and controllable FFU rotation speed control instruction through the action space constraint algorithm, a dynamic action selection strategy needs to be introduced in the training phase to achieve continuous optimization of the model in a complex clean room environment. The greedy mechanism is used to balance the "exploration-exploitation" contradiction, that is, to dynamically switch between the optimal action selection (exploitation) driven by existing experience and unknown strategy exploration. The core is to ensure sufficient coverage of the strategy space through a probabilistic decision framework.

[0032] In the execution layer, the multi-agent architecture selects the action with the maximum Q value for execution using the greedy strategy, and an adaptive exploration strategy is introduced to adjust the value in the greedy strategy based on the pollution change rate ΔC, where ΔC is the particle concentration change rate Δc or the microorganism concentration change rate Δb, The adjustment formula for the ; wherein , are the minimum and maximum values of respectively; is a coefficient for controlling the change rate of ; is a set threshold value; e is the natural constant; At each time step, the value is calculated based on the current pollution change rate, and an action is randomly selected with a probability of , and an action with the maximum Q value is selected with a probability of .

[0033] The edge reasoning module includes: Edge nodes for deploying quantization models to perform real-time processing on data collected by sensors; Fog nodes for storing historical experience data and periodically updating decision-making models; Multi-agent systems responsible for global coordination and fault handling.

[0034] The architecture of multi-agent collaborative control and edge inference is as shown in Figure 3 .

[0035] The design of the reward function is a core component of the system, which aims to provide a quantitative scalar feedback signal R t to evaluate the impact of the action performed by the agent on the overall performance of the system in a specific state. The dynamic reward function R t quantifies and balances two core technical indicators: the control efficiency of the pollutant concentration in the clean room and the total energy consumption of the system. By using this reward signal as a driving force, the reinforcement learning algorithm is guided to optimize autonomously, so that the entire control system can achieve a unified response to the changes in pollutants and efficient energy-saving operation in a dynamic environment.

[0036] In the optimization layer, the dynamic reward function R t includes the concentration recovery reward function R1 and the energy-saving reward function R2. R1 is expressed as: ; In the formula, is a positive weight coefficient for adjusting the importance of concentration recovery reward; ΔC represents the pollution change rate (the change in pollutant concentration between adjacent time steps), which is the particulate matter concentration change rate Δc or the concentration change rate Δb of microorganisms; is a constant to avoid a zero denominator, usually a very small positive number (such as 10 -6 ); this formula can effectively quantify and motivate the intelligent air flow control system to quickly reduce particulate matter and microorganisms to a minimum.

[0037] The energy-saving reward aims to guide the model to minimize energy consumption under the premise of meeting the cleanliness requirements. For situations where energy consumption exceeds the benchmark, R2 is expressed as: ; In the formula, represents the weight coefficient of energy-saving reward; represents the penalty weight when energy consumption exceeds the benchmark; represents the environmental state factor; represents the time factor; represents the energy consumption at the current time; represents the benchmark energy consumption; R t is expressed as: ; Energy consumption refers to the electric power consumed by the FFU when it is running.

[0038] R1 has a significant impact on the overall excitation function result. The focus is on whether the control system can quickly restore the concentration to the ideal level when the concentration of microorganisms and particulate matter in the cleanroom changes. This can be achieved by adjusting the weighting factors. , It can meet the needs of different cleanrooms. When the total reward R t When R is positive, it indicates that the current action (such as the FFU speed adjustment strategy) performs well in terms of both concentration recovery and energy saving, and the model will strengthen the selection bias of this action; if R t A negative R1 indicates that the model's actions have not met expectations in pollutant concentration control or energy management. If the negative reward is due to an excessively low R1, the model's exploration strategy can be adjusted (e.g., increasing R1). If R2 is negative, try new action combinations (such as increasing the FFU speed); if R2 is negative, assess whether the FFU speed is consuming too much energy. You can try reducing the speed to balance cleanliness and energy consumption, or adjust the energy-saving weight parameters (such as increasing the value). Strengthen the penalty for high energy consumption). Furthermore, if negative rewards occur frequently, the weighting coefficient of the reward function can be optimized: increase... (Concentration recovery weights) guide the model to prioritize reducing pollution; adjust or (Energy-saving related weights), balancing energy saving and purification priorities.

[0039] Environmental state factors It reflects the level of activity of people and goods, and is calculated as follows: ; In the formula, k1 is a positive coefficient; This indicates the average speed of movement of people and goods; Indicates the maximum permissible speed; Indicates the average volume of the goods; Indicates the maximum allowable volume; Time factor The energy-saving priority is adjusted according to different time periods, as shown below: ; In the formula, k2 is a positive adjustment coefficient; Indicates the current time; This indicates the peak hours of the day (in a typical work setting, the morning peak is from 8:00 to 10:00, the noon peak is from 12:00 to 13:00, and the afternoon peak is from 17:00 to 19:00).

[0040] Embodiment 2: For the reinforcement learning system consisting of a decision model based on an Actor-Critic framework, a multi-agent architecture and a dynamic reward function, based on Δb, when the preset microbial concentration change rate threshold is exceeded, the training priority is dynamically adjusted: ; In the formula, represents the training priority of the i-th experience; is a proportional coefficient for quantifying the influence degree of Δb on the experience training priority, which is set according to the requirements; is the maximum time step threshold; Set the energy consumption threshold , adjust the experience priority: ; In the formula, represents the storage priority of the j-th experience tuple, which is used as the sampling weight when the reinforcement learning experience is played back; is an adjustment coefficient for controlling the amplitude of priority adjustment. The greater the energy consumption gap, the more significant the influence of the energy consumption gap on the priority, and the stronger the force guiding the model to optimize towards the energy-saving goal. is an energy consumption indicator, which represents the current energy consumption level of the system in real time.

Claims

1. A smart airflow control system based on deep Q-networks, characterized in that, include: The perception layer is equipped with high-definition cameras and particle sensors to collect three-dimensional spatial information of personnel / goods in the clean room, personnel / goods motion parameters, goods volume, particulate matter / microorganism concentration data, and fan filter unit (FFU) rotation speed. It integrates time information to generate a high-dimensional state vector and preprocesses the high-dimensional state vector. The decision-making layer deploys a decision-making model based on the Actor-Critic framework, receives the high-dimensional state vector preprocessed by the perception layer, and generates continuous speed control commands for the FFU. The execution layer connects to the decision-making layer and adopts a multi-agent architecture. Each agent communicates and interacts with information through a graph neural network to coordinate the control of the rotation speed of each FFU. It also has an edge inference module to achieve distributed control. The optimization layer connects the decision-making layer and the execution layer. Based on the concentration change rate of particulate matter and microorganisms and the energy consumption data of FFU, it adjusts the parameters of the decision-making model through a dynamic reward function.

2. The intelligent airflow control system based on a deep Q-network according to claim 1, characterized in that, In the aforementioned perception layer, the cleanroom is equipped with n monitoring points, m monitored personnel, k goods, and q FFUs. The high-dimensional state vector contains the following data: Concentration data C1, C2, ..., C at n monitoring points n Each data point contains the concentration values ​​of particulate matter and microorganisms at that monitoring point; Status information of m monitored individuals, including their three-dimensional location information. and information on direction and speed of motion ; Status information of k goods, including their three-dimensional location information. Information on direction and speed of motion And volume information V1, V2, ..., V k ; Current rotational speed information of q FFUs: F1, F2, ..., F q ; Time step Δt; The rate of change of particulate matter concentration Δc and the rate of change of microbial concentration Δb.

3. The intelligent airflow control system based on a deep Q-network according to claim 1, characterized in that, In the perception layer, the preprocessing is as follows: for high-dimensional state vectors, firstly, preliminary preprocessing such as data cleaning, missing value handling, and data standardization is performed. Then, feature filtering and dimensionality reduction are performed on the high-dimensional state vectors after preliminary preprocessing by using key feature filtering based on SHAP values ​​and data compression based on dynamic PCA.

4. The intelligent airflow control system based on a deep Q-network according to claim 1, characterized in that, In the aforementioned decision-making layer, the decision-making model based on the Actor-Critic framework generates action strategies through the Actor network and evaluates the value of actions through the Critic network. The Actor network employs a fully connected neural network to map the FFU speed adjustment range to a normalized interval of [0, 100], corresponding to 0%-100% of the rated speed. The Critic network evaluates the Q value based on the state-action pair. This Q value is used to quantify the expected cumulative reward of the Actor network executing action a in state s. Action a is the speed action value of the FFU, thereby generating continuous speed control commands for the FFU. The mean square error loss function is used to optimize the network parameters. A deterministic gradient strategy is used to train the decision model based on the Actor-Critic framework.

5. The intelligent airflow control system based on a deep Q-network according to claim 4, characterized in that, For the decision-making model within the Actor-Critic framework, constraints on the rate of change of rotational speed and a safe interval are set, specifically as follows: Speed ​​change rate constraint: For the rotational speed change rate ΔF of the FFU, set its change rate threshold F. max_delta To implement constraints, perform a pruning operation on the output of the Actor network. ； In the formula, This indicates the rotation speed value after clipping; clip indicates the clipping operation. This represents the rotational speed action value output by the Actor network; This indicates the rotational speed value at the previous moment; based on Ensure that the current rate of change of rotational speed meets the requirements. ; Safe range constraints: Define the safe speed range {F min F max },right Perform secondary cutting: ； In the formula, Indicates the rotational speed value after secondary cutting; F min F max These represent the lower and upper limits of the rotational speed, respectively. The rotational speed value is cut to a safe rotational speed range to ensure that the action is performed within the safe operating range of the equipment.

6. The intelligent airflow control system based on a deep Q-network according to claim 1, characterized in that, In the execution layer, the multi-agent architecture employs a greedy strategy to select the action with the highest Q-value for execution. Furthermore, an adaptive exploration strategy is introduced to incorporate the elements of the greedy strategy. The value is related to the pollution change rate ΔC, where ΔC is the particulate matter concentration change rate Δc or the microbial concentration change rate Δb. The formula for adjusting the value is: ； In the formula, , They are respectively The minimum and maximum values; To control The coefficient of the rate of change; The set threshold; At each time step, calculate based on the current pollution change rate. Value, in The probability of randomly selecting an action, in order to The probability is to choose the action with the highest Q value.

7. The intelligent airflow control system based on a deep Q-network according to claim 1, characterized in that, In the execution layer, the edge inference module includes: Edge nodes are used to deploy quantization models and process data collected by sensors in real time. Fog nodes are used to store historical experience data and periodically update the decision-making model; A multi-agent system is responsible for global coordination and fault tolerance.

8. The intelligent airflow control system based on a deep Q-network according to claim 1, characterized in that, In the optimization layer, the dynamic reward function R t This includes the concentration recovery reward function R1 and the energy-saving reward function R2; R1 is represented as: ； In the formula, It is a positive weighting coefficient; ΔC represents the pollution change rate, which is the particulate matter concentration change rate Δc or the microbial concentration change rate Δb. It is a constant to avoid the denominator being zero; Δt is the time step; R2 is represented as: ； In the formula, Indicates the weighting coefficient of energy-saving rewards; This indicates the penalty weight when energy consumption exceeds the benchmark; Indicates environmental state factors; Indicates the time factor; This indicates the energy consumption at the current moment; Indicates baseline energy consumption; R t Represented as: ； Energy consumption refers to the electrical power consumed by the FFU during operation.

9. A smart airflow control system based on a deep Q-network according to claim 8, characterized in that, Environmental state factors It reflects the level of activity of people and goods, and is calculated as follows: ； In the formula, k1 is a positive coefficient; This indicates the average speed of movement of people and goods; Indicates the maximum permissible speed; Indicates the average volume of the goods; Indicates the maximum allowed volume; Time factor The energy-saving priority is adjusted according to different time periods, as shown below: ； In the formula, k2 is a positive adjustment coefficient; Indicates the current time; This indicates the peak time of the day.

Images