Clean room air supply adjustment method and system based on reinforcement learning
By applying a reinforcement learning-based air supply regulation method in a clean room, extracting and analyzing sensing data, generating and updating air supply strategies, the problem of low air supply regulation efficiency in the existing technology is solved, and more efficient energy use is achieved and dynamic adaptability.
Patent Information
- Application Number
- CN202411225052.9
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2024-09-03
- Publication Date
- 2024-11-05
- Estimated Expiration
- 2044-09-03
AI Technical Summary
The existing clean room air supply regulation method is inefficient, resulting in increased energy consumption and cannot adapt to changes in clean room load.
Using reinforcement learning-based method, by extracting personnel timing characteristics and environmental timing characteristics from clean room sensing data, performing linear data analysis and nonlinear error correction, generating air supply strategies, and using radial propagation algorithms for strategy evaluation and update , realize dynamic air supply adjustment.
It improves the efficiency of air supply regulation, reduces energy consumption, and can adjust the air supply strategy in real time to adapt to changes in clean rooms.
Smart Images

Figure CN118729516B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field of air supply regulation, and in particular to a clean room air supply regulation method and system based on reinforcement learning. Background Art
[0002] With the continuous development of various industries and the increasing requirements of society for product quality and safety, the application of clean rooms in the fields of medicine, semiconductors, food, aerospace and aviation is becoming more and more promising. The air supply airflow assembly is an important part of the clean room airflow assembly. Its purpose is to deliver the treated air into the clean room in a certain way. In order to ensure the cleanliness and uniformity of the air in the clean room, the clean room needs to be regulated.
[0003] Most of the existing clean room air supply regulation methods are based on manual control, in which personnel manually control the air supply volume to maintain the pressure and air quality in the clean room. In actual applications, the manual control-based air supply regulation method requires personnel to make adjustments many times, which increases the workload of personnel, and most personnel basically do not make adjustments after the initial setting. Since the air volume is fixed, even if the load in the clean room changes, the system will still run at a constant air volume, resulting in unnecessary energy consumption, which may lead to low efficiency when performing air supply regulation. Summary of the invention
[0004] The present invention provides a clean room air supply adjustment method and system based on reinforcement learning, the main purpose of which is to solve the problem of low efficiency when performing air supply adjustment.
[0005] To achieve the above object, the present invention provides a clean room air supply adjustment method based on reinforcement learning, comprising:
[0006] The personnel time series features and the environment time series features are extracted from the clean room sensor data acquired in real time respectively;
[0007] Performing linear data analysis and nonlinear error correction on the personnel time series characteristics and the environment time series characteristics respectively to obtain analysis environment data and analysis personnel data;
[0008] Generate analysis sensor data according to the analysis environment data and the analysis personnel data, forward propagate the analysis sensor data, and obtain the analysis air supply strategy, wherein the forward propagation of the analysis sensor data to obtain the analysis air supply strategy includes: vectorizing the analysis sensor data into sensor state features; forward propagating the sensor state features using the following radial propagation algorithm to obtain air supply strategy features:
[0009]
[0010] in, It refers to the first one in the air supply strategy characteristics The features of the time step, is the time step index, is the node index, is the total number of nodes in the hidden layer corresponding to the radial propagation algorithm, is the radial propagation algorithm corresponding to The weights of the hidden layers, is the radial propagation algorithm corresponding to The center vector of the hidden layer, is the exponential function symbol, is the radial basis kernel width of the radial propagation algorithm, is the first of the sensor state characteristics The characteristics of each time step; feature mapping the characteristics of the air supply strategy to obtain an analysis air supply strategy;
[0011] Performing a strategy evaluation on the analyzed air supply strategy according to the analyzed sensor data to obtain a strategy reward;
[0012] The analyzed air supply strategy is updated into a standard air supply strategy by using the strategy reward, and the air supply of the clean room is adjusted according to the standard air supply strategy.
[0013] Optionally, extracting the personnel time series features and the environment time series features from the clean room sensor data acquired in real time respectively includes:
[0014] Extract sensor environment data and sensor personnel data from the clean room sensor data acquired in real time respectively;
[0015] Performing a stationarity check and a differential operation on the sensor environment data and the sensor personnel data respectively to obtain stationary environment data and stationary personnel data;
[0016] Performing autoregressive operation on the stable environment data and the stable personnel data to obtain a lag operator matrix;
[0017] Using the lag operator matrix, a maximum likelihood operation is performed on the stationary environment data and the stationary personnel data to obtain a moving average matrix;
[0018] Recursively fitting the stationary personnel data using the lag operator matrix and the moving average matrix to obtain personnel time series characteristics;
[0019] The stationary environmental data are recursively fitted using the lag operator matrix and the moving average matrix to obtain environmental time series characteristics.
[0020] Optionally, performing a stationarity check and a differential operation on the sensor environment data and the sensor personnel data respectively to obtain stationary environment data and stationary personnel data includes:
[0021] Performing data trend fitting on the sensor environment data and the sensor personnel data respectively to obtain fitting environment data and fitting personnel data;
[0022] The environmental stability coefficient of the sensor environment data is calculated based on the fitting environment data using the following residual statistical algorithm:
[0023]
[0024] in, refers to the environmental stability coefficient, is the data length of the sensing environment data, , is the ordinal index, is the first Environmental data, is the first Environmental data, is the first Environmental data, is the first Environmental data;
[0025] Performing a differential operation on the sensor environment data according to the environment stability coefficient to obtain stable environment data;
[0026] Calculating a personnel stability coefficient of the sensed personnel data according to the fitted personnel data;
[0027] The sensor personnel data is subjected to a differential operation according to the personnel stability coefficient to obtain stable personnel data.
[0028] Optionally, performing linear data analysis and nonlinear error correction on the personnel time series characteristics and the environment time series characteristics respectively to obtain analysis environment data and analysis personnel data includes:
[0029] Performing linear data analysis and reverse differential operations on the personnel time series characteristics and the environment time series characteristics respectively to obtain linear environment data and linear personnel data;
[0030] Splitting the linear environment data into linear sensing environment data and linear analysis environment data, and splitting the linear personnel data into linear sensing personnel data and linear analysis personnel data;
[0031] Respectively calculating the personnel residual data corresponding to the linear sensing personnel data and the environment residual data corresponding to the linear sensing environment data;
[0032] Respectively extracting nonlinear personnel time series features corresponding to the personnel residual data and nonlinear environment time series features corresponding to the environment residual data;
[0033] Using the nonlinear personnel time series characteristics, the linear analysis personnel data is subjected to nonlinear error correction to obtain analysis personnel data;
[0034] The nonlinear environmental time series characteristics are utilized to perform nonlinear error correction on the linear analysis environmental data to obtain analysis environmental data.
[0035] Optionally, the extracting respectively the nonlinear personnel time series features corresponding to the personnel residual data and the nonlinear environment time series features corresponding to the environment residual data includes:
[0036] Respectively extracting the long-term personnel time series features and the short-term personnel time series features corresponding to the personnel residual data;
[0037] Connect the long-term personnel time series characteristics and the short-term personnel time series characteristics to the long-term and short-term personnel time series characteristics;
[0038] Performing self-attention weighting on the long and short personnel time series features to obtain nonlinear personnel time series features;
[0039] Respectively extracting the long-term environmental time series features and the short-term environmental time series features corresponding to the environmental residual data;
[0040] Connecting the long-term environmental time series characteristics and the short-term environmental time series characteristics to the long- and short-term environmental time series characteristics;
[0041] The long and short environmental time series features are weighted by self-attention to obtain nonlinear environmental time series features.
[0042] Optionally, the using the nonlinear personnel time series characteristics to perform nonlinear error correction on the linear analysis personnel data to obtain the analysis personnel data includes:
[0043] Performing full connection mapping on the nonlinear personnel time series features to obtain mapped nonlinear personnel features;
[0044] Linearly activating the mapped nonlinear personnel features to obtain nonlinear analysis personnel data;
[0045] The linear analyst data and the nonlinear analyst data are weightedly fused to obtain analyst data.
[0046] Optionally, performing a strategy evaluation on the analyzed air supply strategy according to the analyzed sensor data to obtain a strategy reward includes:
[0047] Performing energy consumption analysis on the analyzed air supply strategy to obtain analyzed air supply energy consumption;
[0048] Extracting analytical sensing temperature, analytical sensing humidity and analytical sensing pressure from the analytical sensing data;
[0049] The strategy reward is calculated based on the analyzed air supply energy consumption, the analyzed sensor temperature, the analyzed sensor humidity, and the analyzed sensor pressure using the following strategy reward algorithm:
[0050]
[0051] in, is the strategy reward, is the time step length of analyzing the sensor data, and the time step length of analyzing the sensor data is equal to the time step length of analyzing the air supply strategy, is the time step index, It is the default The discount factor for time steps is is the first of the analytical sensing temperatures The temperature value of the time step, It is the preset suitable temperature. The first The humidity value of the time step, It is the preset suitable humidity. The analytical sensing pressure The pressure value of the time step, is the preset appropriate pressure. , is the preset reward weight, It is the analysis of air supply energy consumption.
[0052] Optionally, the updating the analyzed air supply strategy to a standard air supply strategy by using the strategy reward includes:
[0053] Determining whether the strategy reward is greater than a preset reward threshold;
[0054] If yes, then the parameters of the radial propagation algorithm are updated according to the strategy reward, and the step of forward propagating the analyzed sensor data to obtain the analyzed air supply strategy is returned;
[0055] If not, the analyzed air supply strategy is used as the standard air supply strategy.
[0056] Optionally, the adjusting the air supply to the clean room according to the standard air supply strategy includes:
[0057] Extracting a standard air supply volume and a standard air supply angle from the standard air supply strategy;
[0058] Measuring the total pressure of the air outlet of the clean room to obtain the total pressure of the air outlet;
[0059] Subtract the preset outlet wind static pressure from the outlet total pressure to obtain the outlet dynamic pressure;
[0060] Calculating the primary air supply velocity according to the air outlet dynamic pressure;
[0061] Calculate the standard air outlet area according to the standard air supply volume and the primary air supply speed;
[0062] Air is supplied to the clean room according to the standard air outlet area and the standard air supply angle.
[0063] In order to solve the above problems, the present invention also provides a clean room air supply adjustment system based on reinforcement learning, the system comprising:
[0064] A feature extraction module is used to extract personnel time series features and environment time series features from the clean room sensor data acquired in real time;
[0065] An error correction module, used to perform linear data analysis and nonlinear error correction on the personnel time series characteristics and the environment time series characteristics respectively, to obtain analysis environment data and analysis personnel data;
[0066] A strategy generation module is used to generate analysis sensor data according to the analysis environment data and the analysis personnel data, and forward propagate the analysis sensor data to obtain the analysis air supply strategy, wherein the forward propagation of the analysis sensor data to obtain the analysis air supply strategy includes: vectorizing the analysis sensor data into sensor state features; forward propagating the sensor state features using the following radial propagation algorithm to obtain the air supply strategy features:
[0067]
[0068] in, It refers to the first one in the air supply strategy characteristics The features of the time step, is the time step index, is the node index, is the total number of nodes in the hidden layer corresponding to the radial propagation algorithm, is the radial propagation algorithm corresponding to The weights of the hidden layers, is the radial propagation algorithm corresponding to The center vector of the hidden layer, is the exponential function symbol, is the radial basis kernel width of the radial propagation algorithm, is the first of the sensor state characteristics The characteristics of each time step; feature mapping the characteristics of the air supply strategy to obtain an analysis air supply strategy;
[0069] A strategy evaluation module, used for performing strategy evaluation on the analyzed air supply strategy according to the analyzed sensing data to obtain a strategy reward;
[0070] The air supply adjustment module is used to update the analyzed air supply strategy into a standard air supply strategy by using the strategy reward, and adjust the air supply of the clean room according to the standard air supply strategy.
[0071] The embodiments of the present invention can extract time series features by extracting personnel time series features and environmental time series features, and can extract time series features in combination with the interactive relationship between personnel data and environmental data, and use the difference and moving average methods to extract the linear time series relationship between personnel data and environmental data, thereby improving the details of the time series features. By performing linear data analysis and nonlinear error correction, it is possible to extract some time series features of nonlinear changes in clean room sensor data, thereby further correcting the analysis results and improving the accuracy of sensor data analysis. By forward propagating the analyzed sensor data, it is possible to use radial basis functions to improve the nonlinear characteristics of the air supply strategy, and achieve the flexibility of the air supply strategy through a reinforcement learning model.
[0072] By conducting strategy evaluation, it is possible to combine the difference between the analyzed sensor parameters and the appropriate parameters of the personnel and the energy consumption data to conduct strategy judgment, thereby improving the efficiency of the air supply strategy. By updating the strategy, the reinforcement learning model can be updated and adjusted in real time to obtain a more energy-efficient air supply strategy. By adjusting the clean room air supply according to the standard air supply strategy, the air supply adjustment can be achieved by adjusting the guide air vents, thereby saving the unit energy consumption by adjusting the air supply fan frequency and improving the efficiency of air supply adjustment. Therefore, the clean room air supply adjustment method and system based on reinforcement learning proposed in the present invention can solve the problem of low efficiency when performing air supply adjustment. BRIEF DESCRIPTION OF THE DRAWINGS
[0073] Figure 1 A schematic diagram of a flow chart of a clean room air supply adjustment method based on reinforcement learning provided in one embodiment of the present invention;
[0074] Figure 2 A schematic diagram of a process for extracting personnel time series features and environment time series features provided by an embodiment of the present invention;
[0075] Figure 3 A schematic diagram of a flow chart of performing a stability check and a differential operation according to an embodiment of the present invention;
[0076] Figure 4 A functional module diagram of a clean room air supply adjustment system based on reinforcement learning provided by one embodiment of the present invention;
[0077] The realization of the purpose, functional features and advantages of the present invention will be further explained in conjunction with embodiments and with reference to the accompanying drawings. DETAILED DESCRIPTION
[0078] It should be understood that the specific embodiments described herein are only used to explain the present invention, and are not used to limit the present invention.
[0079] The embodiment of the present application provides a clean room air supply regulation method based on reinforcement learning. The execution subject of the clean room air supply regulation method based on reinforcement learning includes but is not limited to at least one of the electronic devices such as a server, a terminal, etc. that can be configured to execute the method provided in the embodiment of the present application. In other words, the clean room air supply regulation method based on reinforcement learning can be executed by software or hardware installed on a terminal device or a server device, and the software can be a blockchain platform. The server includes but is not limited to: a single server, a server cluster, a cloud server or a cloud server cluster, etc. The server can be an independent server, or it can be a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, content delivery networks (Content Delivery Network, CDN), and big data and artificial intelligence platforms.
[0080] Reference Figure 1 FIG. 1 is a flow chart of a clean room air supply adjustment method based on reinforcement learning provided in one embodiment of the present invention. In this embodiment, the clean room air supply adjustment method based on reinforcement learning includes:
[0081] S1. Extract personnel time series features and environment time series features from the clean room sensor data acquired in real time.
[0082] In detail, the clean room sensor data is time series data obtained by performing various types of data monitoring on the clean room, and the clean room sensor data includes sensor environment data and sensor personnel data, wherein the sensor environment data includes environmental data such as temperature, humidity and pressure of the clean room over a period of time, and the sensor personnel data includes personnel data such as the number of personnel in the clean room over a period of time, personnel location and personnel activity frequency.
[0083] Specifically, the personnel time series feature refers to the linear feature of the sensor personnel data changing over time, and the environment time series feature refers to the linear feature of the sensor environment data changing over time.
[0084] In the embodiment of the present invention, refer to Figure 2 As shown, the personnel time series features and the environment time series features are extracted from the clean room sensor data acquired in real time, including:
[0085] S21, extracting sensor environment data and sensor personnel data from the clean room sensor data acquired in real time respectively;
[0086] S22, performing a stability check and a differential operation on the sensor environment data and the sensor personnel data respectively to obtain stable environment data and stable personnel data;
[0087] S23, performing an autoregressive operation on the stable environment data and the stable personnel data to obtain a lag operator matrix;
[0088] S24, using the lag operator matrix to perform maximum likelihood operation on the stable environment data and the stable personnel data to obtain a moving average matrix;
[0089] S25, recursively fitting the stationary personnel data using the lag operator matrix and the moving average matrix to obtain personnel time series characteristics;
[0090] S26. Use the lag operator matrix and the moving average matrix to recursively fit the stationary environmental data to obtain environmental time series characteristics.
[0091] In detail, the data in the sensor environment data and the sensor personnel data are sorted in time series. In order to better extract the time series characteristics of the personnel data and the environment data, it is necessary to stabilize the personnel data and the environment data. The stabilized environment data and the stabilized personnel data are respectively the sensor environment data and the sensor personnel data after stabilization.
[0092] Specifically, refer to Figure 3 As shown, the sensing environment data and the sensing personnel data are respectively subjected to a stationary check and a differential operation to obtain stationary environment data and stationary personnel data, including:
[0093] S31, performing data trend fitting on the sensor environment data and the sensor personnel data respectively to obtain fitting environment data and fitting personnel data;
[0094] S32, calculating the environmental stability coefficient of the sensor environment data according to the fitting environment data;
[0095] S33, performing a differential operation on the sensor environment data according to the environment stability coefficient to obtain stable environment data;
[0096] S34, calculating the personnel stability coefficient of the sensed personnel data according to the fitted personnel data;
[0097] S35. Perform a differential operation on the sensor personnel data according to the personnel stability coefficient to obtain stable personnel data.
[0098] In detail, the least squares method or linear regression model can be used to fit the data trend, and the fitted environmental data is a fitted environmental data sequence having the trend of the sensored environmental data, and the fitted personnel data is a fitted personnel data sequence having the trend of the sensored personnel data.
[0099] Specifically, the environmental stability coefficient is calculated using the following residual statistical algorithm:
[0100]
[0101] in, refers to the environmental stability coefficient, is the data length of the sensing environment data, , is the ordinal index, is the first Environmental data, is the first Environmental data, is the first Environmental data, is the first Environmental data.
[0102] In detail, the residual statistical algorithm can use the cumulative sum of residuals to reflect the cumulative deviation of environmental data, and measure the volatility of the sequence according to the ratio between the square of the cumulative sum of residuals and the variance of the residuals, thereby improving the intuitiveness of the calculation of the environmental stability coefficient.
[0103] Specifically, the differential operation is a commonly used time series data preprocessing method, which is used to eliminate the trend and seasonal components in the time series to make the data more stable. The differential operation calculates the difference between the current value of the time series and its lagged value, thereby removing the trend change or seasonal component in the time series.
[0104] In detail, performing a differential operation on the sensed environment data according to the environment stability coefficient refers to determining whether the environment stability coefficient is greater than a preset stability threshold. If so, performing a differential operation; if not, treating the sensed environment data as stable environment data.
[0105] In detail, the method for calculating the personnel stability coefficient of the sensed personnel data based on the fitted personnel data is the same as the method for calculating the environmental stability coefficient of the sensed environment data based on the fitted environment data in the above steps. The method for performing a differential operation on the sensed personnel data based on the personnel stability coefficient to obtain stable personnel data and stable environment data is the same as the method for performing a differential operation on the sensored environment data based on the environmental stability coefficient in the above steps, which will not be repeated here.
[0106] Specifically, the vector autoregression model (VAR) can be used to perform autoregression operations. The lag operator matrix (Lag Operator Matrix) is a mathematical tool for representing the lag relationship in time series data. The lag operator matrix can capture the dependency between the stationary environmental data and the stationary personnel data. The moving average matrix (Moving Average Matrix) is a mathematical tool for representing the relationship between the current value and the past error terms in a multivariate time series model.
[0107] In an embodiment of the present invention, by extracting personnel time series features and environmental time series features, the time series features can be extracted in combination with the interactive relationship between personnel data and environmental data, and the linear time series relationship between personnel data and environmental data can be extracted using differential and moving average methods, thereby improving the details of the time series features.
[0108] S2. Perform linear data analysis and nonlinear error correction on the personnel time series characteristics and the environment time series characteristics respectively to obtain analysis environment data and analysis personnel data.
[0109] In detail, the analyzed environment data refers to the analyzed sensor environment data for a future time period, and the analyzed personnel data refers to the analyzed sensor personnel data for a future time period.
[0110] In the embodiment of the present invention, the linear data analysis and nonlinear error correction are performed on the personnel time series characteristics and the environment time series characteristics respectively to obtain the analysis environment data and the analysis personnel data, including:
[0111] Performing linear data analysis and reverse differential operations on the personnel time series characteristics and the environment time series characteristics respectively to obtain linear environment data and linear personnel data;
[0112] Splitting the linear environment data into linear sensing environment data and linear analysis environment data, and splitting the linear personnel data into linear sensing personnel data and linear analysis personnel data;
[0113] Respectively calculating the personnel residual data corresponding to the linear sensing personnel data and the environment residual data corresponding to the linear sensing environment data;
[0114] Respectively extracting nonlinear personnel time series features corresponding to the personnel residual data and nonlinear environment time series features corresponding to the environment residual data;
[0115] Using the nonlinear personnel time series characteristics, the linear analysis personnel data is subjected to nonlinear error correction to obtain analysis personnel data;
[0116] The nonlinear environmental time series characteristics are utilized to perform nonlinear error correction on the linear analysis environmental data to obtain analysis environmental data.
[0117] Specifically, the linear data analysis refers to iterative linear operations on the personnel time series characteristics using the lag operator matrix, and iterative linear operations on the environment time series characteristics using the lag operator matrix, and the inverse differential operation is the inverse operation of the differential operation.
[0118] In detail, the linear sensing environment data is the environment data in the linear environment data with the same time step as the sensing environment data, the linear analysis environment data is the environment data in the linear environment data after the time step of the sensing environment data, the linear sensing personnel data is the personnel data in the linear personnel data with the same time step as the sensing personnel data, and the linear analysis personnel data is the personnel data in the linear personnel data after the time step of the sensing personnel data.
[0119] Specifically, the personnel residual data is the residual data of the sensed personnel data and the linear analysis personnel data, and the environment residual data is the residual data of the sensed environment data and the linear analysis environment data.
[0120] In detail, respectively extracting the nonlinear personnel time series features corresponding to the personnel residual data and the nonlinear environment time series features corresponding to the environment residual data includes:
[0121] Respectively extracting the long-term personnel time series features and the short-term personnel time series features corresponding to the personnel residual data;
[0122] Connect the long-term personnel time series characteristics and the short-term personnel time series characteristics to the long-term and short-term personnel time series characteristics;
[0123] Performing self-attention weighting on the long and short personnel time series features to obtain nonlinear personnel time series features;
[0124] Respectively extracting the long-term environmental time series features and the short-term environmental time series features corresponding to the environmental residual data;
[0125] Connecting the long-term environmental time series characteristics and the short-term environmental time series characteristics to the long- and short-term environmental time series characteristics;
[0126] The long and short environmental time series features are weighted by self-attention to obtain nonlinear environmental time series features.
[0127] In detail, the memory unit and gate structure of the Long Short-Term Memory (LSTM) neural network model can be used to extract long-term personnel time series features, short-term personnel time series features, long-term environment time series features and short-term environment time series features, and the self-attention mechanism can be used for self-attention weighting.
[0128] Specifically, the nonlinear error correction of the linear analysis personnel data by using the nonlinear personnel time series characteristics to obtain the analysis personnel data includes:
[0129] Performing full connection mapping on the nonlinear personnel time series features to obtain mapped nonlinear personnel features;
[0130] Linearly activating the mapped nonlinear personnel features to obtain nonlinear analysis personnel data;
[0131] The linear analyst data and the nonlinear analyst data are weightedly fused to obtain analyst data.
[0132] In detail, the fully connected layer of the output layer of the LSTM model can be used for fully connected mapping, and the softmax function or the sigmoid function can be used for linear activation.
[0133] Specifically, the method of using the nonlinear environment time series characteristics to perform nonlinear error correction on the linear analysis environment data to obtain analysis environment data is the same as the method of using the nonlinear personnel time series characteristics to perform nonlinear error correction on the linear analysis personnel data to obtain analysis personnel data in the above steps, and will not be repeated here.
[0134] In the embodiment of the present invention, by performing linear data analysis and nonlinear error correction, partial time series characteristics of nonlinear changes in clean room sensor data can be extracted, so as to further correct the analysis results and improve the accuracy of sensor data analysis.
[0135] S3. Generate analysis sensor data according to the analysis environment data and the analysis personnel data, and forward propagate the analysis sensor data to obtain an analysis air supply strategy.
[0136] In detail, the analytical sensor data is a combination of the analytical environment data and the analytical personnel data, and the analytical air supply strategy is a strategy generated based on the analytical sensor data for adjusting the air supply angle and air supply flow rate of the clean room air supply component within the time period of the analytical sensor data.
[0137] In the embodiment of the present invention, the forward propagation of the analyzed sensor data to obtain the analyzed air supply strategy includes:
[0138] vectorizing the analyzed sensor data into sensor state features;
[0139] Performing forward propagation on the sensor state characteristics to obtain air supply strategy characteristics;
[0140] Feature mapping is performed on the air supply strategy features to obtain an analyzed air supply strategy.
[0141] Specifically, the radial propagation algorithm is an algorithm determined by a deep deterministic policy gradient model (DDPG) in a pre-trained reinforcement learning network, wherein the weight of the hidden layer and the center vector of the hidden layer are parameters obtained through training.
[0142] In detail, the air supply strategy characteristics are calculated using the following radial propagation algorithm:
[0143]
[0144] in, Refers to the first of the air supply strategy characteristics The features of the time step, is the time step index, is the node index, is the total number of nodes in the hidden layer corresponding to the radial propagation algorithm, is the radial propagation algorithm corresponding to The weights of the hidden layers, is the radial propagation algorithm corresponding to The center vector of the hidden layer, is the exponential function symbol, is the radial basis kernel width of the radial propagation algorithm, is the first of the sensor state characteristics The features of the time step, is the absolute value symbol.
[0145] Specifically, the radial propagation algorithm can map input features to a high-dimensional feature space, thereby capturing nonlinear features in the features and improving the generalization ability of the deep deterministic policy gradient model and its adaptability to complex environments.
[0146] In detail, the feature mapping refers to mapping the air supply strategy feature into a preset air supply strategy database, and each air supply strategy in the air supply strategy database corresponds to an air supply strategy feature.
[0147] In the embodiment of the present invention, by forward propagating the analyzed sensor data, the nonlinear characteristics of the air supply strategy can be improved by using radial basis functions, and the flexibility of the air supply strategy can be achieved through a reinforcement learning model.
[0148] S4. Performing a strategy evaluation on the analyzed air supply strategy according to the analyzed sensor data to obtain a strategy reward.
[0149] In detail, the strategy reward is a numerical value used to reflect the pros and cons of the analyzed air supply strategy, and the strategy reward can implement adjustments to the model parameters of the deep deterministic policy gradient model.
[0150] In the embodiment of the present invention, the step of performing a strategy evaluation on the analyzed air supply strategy according to the analyzed sensor data to obtain a strategy reward includes:
[0151] Performing energy consumption analysis on the analyzed air supply strategy to obtain analyzed air supply energy consumption;
[0152] Extracting analytical sensing temperature, analytical sensing humidity and analytical sensing pressure from the analytical sensing data;
[0153] A strategy reward is calculated based on the analyzed air supply energy consumption, the analyzed sensed temperature, the analyzed sensed humidity, and the analyzed sensed pressure.
[0154] In detail, energy consumption analysis may be performed based on an energy consumption model of the clean room air supply component, wherein the analytical sensing temperature, the analytical sensing humidity, and the analytical sensing pressure are respectively the temperature, humidity, and pressure analyzed from the analytical sensing data.
[0155] Specifically, the strategy reward is calculated using the following strategy reward algorithm:
[0156]
[0157] in, is the strategy reward, is the time step length of analyzing the sensor data, and the time step length of analyzing the sensor data is equal to the time step length of analyzing the air supply strategy, is the time step index, It is the default The discount factor for time steps is is the first of the analytical sensing temperatures The temperature value of the time step, It is the preset suitable temperature. The first The humidity value of the time step, It is the preset suitable humidity. The analytical sensing pressure The pressure value of the time step, is the preset appropriate pressure. , is the preset reward weight, It is the analysis of air supply energy consumption.
[0158] In detail, the suitable temperature, suitable humidity and suitable pressure are the temperature, humidity and pressure that people feel are suitable in the preset environment. The strategy reward algorithm can combine the difference between the analyzed sensor parameters and the suitable parameters of the personnel and the energy consumption data to make strategy judgments, thereby improving the efficiency of the air supply strategy.
[0159] In the embodiment of the present invention, by performing strategy evaluation, it is possible to perform strategy judgment in combination with the difference between the analyzed sensor parameters and the appropriate parameters of the personnel and the energy consumption data, thereby improving the efficiency of the air supply strategy.
[0160] S5. Using the strategy reward, the analyzed air supply strategy is updated into a standard air supply strategy, and the air supply of the clean room is adjusted according to the standard air supply strategy.
[0161] In the embodiment of the present invention, the step of updating the analyzed air supply strategy to a standard air supply strategy by using the strategy reward includes:
[0162] Determining whether the strategy reward is greater than a preset reward threshold;
[0163] If yes, then the parameters of the radial propagation algorithm are updated according to the strategy reward, and the step of forward propagating the analyzed sensor data to obtain the analyzed air supply strategy is returned;
[0164] If not, the analyzed air supply strategy is used as the standard air supply strategy.
[0165] In detail, the reward threshold is a preset threshold for measuring whether the strategy reward of the air supply strategy analysis is qualified, and the parameters of the radial propagation algorithm can be updated using the gradient descent algorithm.
[0166] Specifically, the air supply adjustment for the clean room according to the standard air supply strategy includes:
[0167] Extracting a standard air supply volume and a standard air supply angle from the standard air supply strategy;
[0168] Measuring the total pressure of the air outlet of the clean room to obtain the total pressure of the air outlet;
[0169] Subtract the preset outlet wind static pressure from the outlet total pressure to obtain the outlet dynamic pressure;
[0170] Calculating the primary air supply velocity according to the air outlet dynamic pressure;
[0171] Calculate the standard air outlet area according to the standard air supply volume and the primary air supply speed;
[0172] Air is supplied to the clean room according to the standard air outlet area and the standard air supply angle.
[0173] In detail, the standard air supply volume refers to the air volume required to supply air to the clean room corresponding to the standard air supply strategy, and the standard air supply angle refers to the angle required to supply air to the clean room corresponding to the standard air supply strategy.
[0174] Specifically, the total pressure can be measured using the Pitot tube principle. The total pressure at the air outlet represents the total energy density of the air fluid at the air outlet of the clean room air supply assembly. The static pressure at the air outlet is the internal pressure of the air fluid. The dynamic pressure at the air outlet is the additional pressure caused by the movement speed of the air fluid.
[0175] In detail, the primary air supply speed can be calculated according to the dynamic pressure of the air outlet using the dynamic pressure formula. The standard air outlet area is equal to the standard air supply volume divided by the primary air supply speed. The guide vane angle of the air outlet can be changed so that the effective air supply area of the air outlet is the standard air outlet area, and the effective air supply angle of the air outlet is the standard air supply angle, thereby achieving air supply regulation.
[0176] In an embodiment of the present invention, by performing strategy updates, the reinforcement learning model can be updated and adjusted in real time to obtain a more energy-efficient air supply strategy. By adjusting the air supply to the clean room according to the standard air supply strategy, the air supply adjustment can be achieved by adjusting the guide air vents, thereby saving unit energy consumption by adjusting the air supply fan frequency and improving the efficiency of air supply adjustment.
[0177] The embodiments of the present invention can extract time series features by extracting personnel time series features and environmental time series features, and can extract time series features in combination with the interactive relationship between personnel data and environmental data, and use the difference and moving average methods to extract the linear time series relationship between personnel data and environmental data, thereby improving the details of the time series features. By performing linear data analysis and nonlinear error correction, it is possible to extract some time series features of nonlinear changes in clean room sensor data, thereby further correcting the analysis results and improving the accuracy of sensor data analysis. By forward propagating the analyzed sensor data, it is possible to use radial basis functions to improve the nonlinear characteristics of the air supply strategy, and achieve the flexibility of the air supply strategy through a reinforcement learning model.
[0178] By conducting strategy evaluation, it is possible to combine the difference between the analyzed sensor parameters and the appropriate parameters of the personnel and the energy consumption data to conduct strategy judgment, thereby improving the efficiency of the air supply strategy. By updating the strategy, the reinforcement learning model can be updated and adjusted in real time to obtain a more energy-efficient air supply strategy. By adjusting the clean room air supply according to the standard air supply strategy, the air supply adjustment can be achieved by adjusting the guide air vents, thereby saving the unit energy consumption by adjusting the air supply fan frequency and improving the efficiency of air supply adjustment. Therefore, the clean room air supply adjustment method based on reinforcement learning proposed in the present invention can solve the problem of low efficiency when performing air supply adjustment.
[0179] like Figure 4 , which is a functional module diagram of a clean room air supply adjustment system based on reinforcement learning provided by one embodiment of the present invention.
[0180] The clean room air supply adjustment system 100 based on reinforcement learning of the present invention can be installed in an electronic device. According to the functions to be implemented, the clean room air supply adjustment system 100 based on reinforcement learning can include a feature extraction module 101, an error correction module 102, a strategy generation module 103, a strategy evaluation module 104 and an air supply adjustment module 105. The module of the present invention can also be called a unit, which refers to a series of computer program segments that can be executed by an electronic device processor and can complete fixed functions, which are stored in the memory of the electronic device.
[0181] In this embodiment, the functions of each module / unit are as follows:
[0182] The feature extraction module 101 is used to extract personnel time series features and environment time series features from the clean room sensor data acquired in real time;
[0183] The error correction module 102 is used to perform linear data analysis and nonlinear error correction on the personnel time series characteristics and the environment time series characteristics respectively to obtain analysis environment data and analysis personnel data;
[0184] The strategy generation module 103 is used to generate analysis sensor data according to the analysis environment data and the analysis personnel data, and forward propagate the analysis sensor data to obtain the analysis air supply strategy, wherein the forward propagation of the analysis sensor data to obtain the analysis air supply strategy includes: vectorizing the analysis sensor data into sensor state features; forward propagating the sensor state features using the following radial propagation algorithm to obtain the air supply strategy features:
[0185]
[0186] in, It refers to the first one in the air supply strategy characteristics The features of the time step, is the time step index, is the node index, is the total number of nodes in the hidden layer corresponding to the radial propagation algorithm, is the radial propagation algorithm corresponding to The weights of the hidden layers, is the radial propagation algorithm corresponding to The center vector of the hidden layer, is the exponential function symbol, is the radial basis kernel width of the radial propagation algorithm, is the first of the sensor state characteristics The characteristics of each time step; feature mapping the characteristics of the air supply strategy to obtain an analysis air supply strategy;
[0187] The strategy evaluation module 104 is used to perform strategy evaluation on the analyzed air supply strategy according to the analyzed sensor data to obtain a strategy reward;
[0188] The air supply adjustment module 105 is used to update the analyzed air supply strategy into a standard air supply strategy by using the strategy reward, and adjust the air supply of the clean room according to the standard air supply strategy.
[0189] In detail, each module described in the clean room air supply adjustment system 100 based on reinforcement learning in the embodiment of the present invention is used in the same manner as described above. Figures 1 to 3 The clean room air supply adjustment method based on reinforcement learning described in the invention has the same technical means and can produce the same technical effects, so it will not be repeated here.
[0190] In the several embodiments provided by the present invention, it should be understood that the disclosed devices, systems and methods can be implemented in other ways. For example, the system embodiments described above are only schematic, for example, the division of the modules is only a logical function division, and there may be other division methods in actual implementation.
[0191] The modules described as separate components may or may not be physically separated, and the components shown as modules may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of this embodiment.
[0192] In addition, each functional module in each embodiment of the present invention may be integrated into one processing unit, each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of hardware plus software functional modules.
[0193] It is obvious to those skilled in the art that the present invention is not limited to the details of the above exemplary embodiments, and that the present invention can be implemented in other specific forms without departing from the spirit or essential characteristics of the present invention.
[0194] The embodiments of the present application can acquire and process relevant data based on artificial intelligence technology. Artificial Intelligence (AI) is the theory, method, technology and application system that uses digital computers or machines controlled by digital computers to simulate, extend and expand human intelligence, perceive the environment, acquire knowledge and use knowledge to obtain the best results.
[0195] In addition, it is obvious that the word "comprising" does not exclude other units or steps, and the singular does not exclude the plural. The multiple units or systems stated in the system embodiment can also be implemented by one unit or system through software or hardware. The words first, second, etc. are used to indicate names, and do not indicate any specific order.
[0196] Finally, it should be noted that the above embodiments are only used to illustrate the technical solution of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical solution of the present invention can be modified or replaced by equivalents without departing from the spirit and scope of the technical solution of the present invention.
Claims
1. A clean room air supply adjustment method based on reinforcement learning, It is characterized in that The method comprises: The personnel time series features and the environment time series features are extracted from the clean room sensor data acquired in real time respectively; Performing linear data analysis and nonlinear error correction on the personnel time series characteristics and the environment time series characteristics respectively to obtain analysis environment data and analysis personnel data; Generate analysis sensor data according to the analysis environment data and the analysis personnel data, forward propagate the analysis sensor data, and obtain the analysis air supply strategy, wherein the forward propagation of the analysis sensor data to obtain the analysis air supply strategy includes: vectorizing the analysis sensor data into sensor state features; forward propagating the sensor state features using the following radial propagation algorithm to obtain air supply strategy features: in, It refers to the first one in the air supply strategy characteristics The features of the time step, is the time step index, is the node index, is the total number of nodes in the hidden layer corresponding to the radial propagation algorithm, is the radial propagation algorithm corresponding to The weights of the hidden layers, is the radial propagation algorithm corresponding to The center vector of the hidden layer, is the exponential function symbol, is the radial basis kernel width of the radial propagation algorithm, is the first of the sensor state characteristics The characteristics of each time step; feature mapping the characteristics of the air supply strategy to obtain an analysis air supply strategy; The analytical air supply strategy is strategically evaluated according to the analytical sensor data to obtain a strategic reward, wherein the strategic evaluation of the analytical air supply strategy according to the analytical sensor data to obtain a strategic reward includes: performing energy consumption analysis on the analytical air supply strategy to obtain analytical air supply energy consumption; extracting analytical sensor temperature, analytical sensor humidity and analytical sensor pressure from the analytical sensor data; and calculating a strategic reward according to the analytical air supply energy consumption, the analytical sensor temperature, the analytical sensor humidity and the analytical sensor pressure using the following strategic reward algorithm: in, is the strategy reward, is the time step length of analyzing the sensor data, and the time step length of analyzing the sensor data is equal to the time step length of analyzing the air supply strategy, is the time step index, It is the default The discount factor for each time step is is the first of the analytical sensing temperatures The temperature value of the time step, It is the preset suitable temperature. The first The humidity value of the time step, It is the preset suitable humidity. The analytical sensing pressure The pressure value of the time step, is the preset appropriate pressure. , is the preset reward weight, is the energy consumption of air supply analyzed; The analyzed air supply strategy is updated into a standard air supply strategy by using the strategy reward, and the air supply of the clean room is adjusted according to the standard air supply strategy.
2. The clean room air supply adjustment method based on reinforcement learning as claimed in claim 1, It is characterized in that The extracting of personnel time series features and environment time series features from the clean room sensor data acquired in real time respectively includes: Extract sensor environment data and sensor personnel data from the clean room sensor data acquired in real time respectively; Performing a stationarity check and a differential operation on the sensor environment data and the sensor personnel data respectively to obtain stationary environment data and stationary personnel data; Performing autoregressive operation on the stable environment data and the stable personnel data to obtain a lag operator matrix; Using the lag operator matrix, a maximum likelihood operation is performed on the stationary environment data and the stationary personnel data to obtain a moving average matrix; Recursively fitting the stationary personnel data using the lag operator matrix and the moving average matrix to obtain personnel time series characteristics; The stationary environmental data are recursively fitted using the lag operator matrix and the moving average matrix to obtain environmental time series characteristics.
3. The clean room air supply adjustment method based on reinforcement learning as claimed in claim 2, It is characterized in that The step of performing a stationarity check and a differential operation on the sensor environment data and the sensor personnel data respectively to obtain stationary environment data and stationary personnel data includes: Performing data trend fitting on the sensor environment data and the sensor personnel data respectively to obtain fitting environment data and fitting personnel data; The environmental stability coefficient of the sensor environment data is calculated based on the fitting environment data using the following residual statistical algorithm: in, refers to the environmental stability coefficient, is the data length of the sensing environment data, , is the ordinal index, is the first Environmental data, is the first Environmental data, is the first Environmental data, is the first Environmental data; Performing a differential operation on the sensor environment data according to the environment stability coefficient to obtain stable environment data; Calculating a personnel stability coefficient of the sensed personnel data according to the fitted personnel data; The sensor personnel data is subjected to a differential operation according to the personnel stability coefficient to obtain stable personnel data.
4. The clean room air supply adjustment method based on reinforcement learning as claimed in claim 1, It is characterized in that The performing linear data analysis and nonlinear error correction on the personnel time series characteristics and the environment time series characteristics respectively to obtain analysis environment data and analysis personnel data includes: Performing linear data analysis and reverse differential operations on the personnel time series characteristics and the environment time series characteristics respectively to obtain linear environment data and linear personnel data; Splitting the linear environment data into linear sensing environment data and linear analysis environment data, and splitting the linear personnel data into linear sensing personnel data and linear analysis personnel data; Respectively calculating the personnel residual data corresponding to the linear sensing personnel data and the environment residual data corresponding to the linear sensing environment data; Respectively extracting nonlinear personnel time series features corresponding to the personnel residual data and nonlinear environment time series features corresponding to the environment residual data; Using the nonlinear personnel time series characteristics, nonlinear error correction is performed on the linear analysis personnel data to obtain analysis personnel data; The nonlinear environmental time series characteristics are utilized to perform nonlinear error correction on the linear analysis environmental data to obtain analysis environmental data.
5. The clean room air supply adjustment method based on reinforcement learning as claimed in claim 4, It is characterized in that The extracting the nonlinear personnel time series features corresponding to the personnel residual data and the nonlinear environment time series features corresponding to the environment residual data respectively includes: Respectively extracting the long-term personnel time series features and the short-term personnel time series features corresponding to the personnel residual data; Connecting the long-term personnel time series characteristics and the short-term personnel time series characteristics to the long-term and short-term personnel time series characteristics; Performing self-attention weighting on the long and short personnel time series features to obtain nonlinear personnel time series features; Respectively extracting the long-term environmental time series features and the short-term environmental time series features corresponding to the environmental residual data; Connecting the long-term environmental time series characteristics and the short-term environmental time series characteristics to the long- and short-term environmental time series characteristics; The long and short environmental time series features are weighted by self-attention to obtain nonlinear environmental time series features.
6. The clean room air supply adjustment method based on reinforcement learning as claimed in claim 4, It is characterized in that The method of performing nonlinear error correction on the linear analysis personnel data by using the nonlinear personnel time series characteristics to obtain the analysis personnel data includes: Performing full connection mapping on the nonlinear personnel time series features to obtain mapped nonlinear personnel features; Linearly activating the mapped nonlinear personnel features to obtain nonlinear analysis personnel data; The linear analyst data and the nonlinear analyst data are weightedly fused to obtain analyst data.
7. The clean room air supply adjustment method based on reinforcement learning as claimed in claim 1, It is characterized in that The updating of the analyzed air supply strategy into a standard air supply strategy by using the strategy reward includes: Determining whether the strategy reward is greater than a preset reward threshold; If yes, then the parameters of the radial propagation algorithm are updated according to the strategy reward, and the step of forward propagating the analyzed sensor data to obtain the analyzed air supply strategy is returned; If not, the analyzed air supply strategy is used as the standard air supply strategy.
8. The clean room air supply adjustment method based on reinforcement learning as claimed in claim 1, It is characterized in that The step of adjusting the air supply to the clean room according to the standard air supply strategy includes: Extracting a standard air supply volume and a standard air supply angle from the standard air supply strategy; Measuring the total pressure of the air outlet of the clean room to obtain the total pressure of the air outlet; Subtract the preset outlet wind static pressure from the outlet total pressure to obtain the outlet dynamic pressure; Calculating the primary air supply velocity according to the air outlet dynamic pressure; Calculate the standard air outlet area according to the standard air supply volume and the primary air supply speed; Air is supplied to the clean room according to the standard air outlet area and the standard air supply angle.
9. A clean room air supply adjustment system based on reinforcement learning, It is characterized in that The system comprises: A feature extraction module is used to extract personnel time series features and environment time series features from the clean room sensor data acquired in real time; An error correction module, used to perform linear data analysis and nonlinear error correction on the personnel time series characteristics and the environment time series characteristics respectively, to obtain analysis environment data and analysis personnel data; A strategy generation module is used to generate analysis sensor data according to the analysis environment data and the analysis personnel data, and forward propagate the analysis sensor data to obtain the analysis air supply strategy, wherein the forward propagation of the analysis sensor data to obtain the analysis air supply strategy includes: vectorizing the analysis sensor data into sensor state features; forward propagating the sensor state features using the following radial propagation algorithm to obtain the air supply strategy features: in, It refers to the first one in the air supply strategy characteristics The features of the time step, is the time step index, is the node index, is the total number of nodes in the hidden layer corresponding to the radial propagation algorithm, is the radial propagation algorithm corresponding to The weights of the hidden layers, is the radial propagation algorithm corresponding to The center vector of the hidden layer, is the exponential function symbol, is the radial basis kernel width of the radial propagation algorithm, is the first of the sensor state characteristics The characteristics of each time step; feature mapping the characteristics of the air supply strategy to obtain an analysis air supply strategy; A strategy evaluation module is used to perform a strategy evaluation on the analytical air supply strategy according to the analytical sensor data to obtain a strategy reward, wherein the strategy evaluation on the analytical air supply strategy according to the analytical sensor data to obtain a strategy reward includes: performing an energy consumption analysis on the analytical air supply strategy to obtain analytical air supply energy consumption; extracting analytical sensor temperature, analytical sensor humidity and analytical sensor pressure from the analytical sensor data; and calculating a strategy reward according to the analytical air supply energy consumption, the analytical sensor temperature, the analytical sensor humidity and the analytical sensor pressure using the following strategy reward algorithm: in, is the strategy reward, is the time step length of analyzing the sensor data, and the time step length of analyzing the sensor data is equal to the time step length of analyzing the air supply strategy, is the time step index, It is the default The discount factor for each time step is is the first of the analytical sensing temperatures The temperature value of the time step, It is the preset suitable temperature. The first The humidity value of the time step, It is the preset suitable humidity. The analytical sensing pressure The pressure value of the time step, is the preset appropriate pressure. , is the preset reward weight, is the energy consumption of air supply analyzed; The air supply adjustment module is used to update the analyzed air supply strategy into a standard air supply strategy by using the strategy reward, and adjust the air supply of the clean room according to the standard air supply strategy.
