CNN-LSTM-based particle anomaly monitoring system and monitoring method

Abstract

The invention discloses a CNN-LSTM-based particle anomaly monitoring system and monitoring method. The CNN-LSTM-based particle anomaly monitoring system comprises a data acquisition module, a target detection module, a data processing module, a model construction module, an anomaly prediction module and a data storage module. According to the system, environmental parameters such as the temperature, humidity and pressure difference of a clean room are collected through sensors, and personnel access information is detected through YoloV11; after data fusion and normalization preprocessing, inputting the data into a CNN-LSTM neural network, automatically identifying an abnormal reason and predicting a particle concentration trend; particle abnormity real-time accurate monitoring and intelligent early warning are realized, the operation and maintenance cost is reduced, and the clean room management efficiency is improved.

Description

Technical Field

[0001] The present invention relates to the field of cleanroom environmental monitoring, and particularly to a particle anomaly monitoring system and monitoring method based on CNN-LSTM. Background Art

[0002] With the continuous improvement of the requirements for the cleanliness of the production environment in industries such as the electronics industry, aerospace, medical health, and biopharmaceuticals, the cleanroom, as a key facility for controlling and maintaining the particle concentration within a specific grade standard, its monitoring and maintenance have become an important part of the industry's quality management. Environmental parameters such as temperature, humidity, and differential pressure in the cleanroom, as well as factors such as personnel entry and exit and equipment operation status, can directly or indirectly affect the dynamic changes in the indoor particle concentration. Maintaining the long-term stable compliance of the cleanroom is not only related to product quality and experimental safety but also determines the control of operating costs and energy consumption.

[0003] Currently, the diagnosis of abnormal particle concentration in the cleanroom mainly relies on manual inspections or alarm based on empirical thresholds. Operators judge by timed sampling or visual inspection, consuming a large amount of manpower and material resources, and the detection accuracy is closely related to the experience level, making it difficult to achieve rapid response and accurate positioning for sudden anomalies. To improve the monitoring efficiency and intelligent level, several online monitoring and automatic control solutions have been proposed at home and abroad, but each still has limitations to varying degrees:

[0004] In the "Cleanroom Dust Particle Online Monitoring System" disclosed in the patent publication number CN202256115U, the solution uses multiple optoelectronic sampling dust particle counting sensors and realizes data upload through photovoltaic cell power supply and wireless access. This system solves the problems of sensor wiring and power supply access limitations, but it can only perform real-time counting and statistics on the particle concentration, lacking the intelligent analysis of the causes of anomalies and the prediction function of future trends.

[0005] In the "Control Method, Device and System for Cleanroom" disclosed in the patent publication number CN118816304A, based on the comparison of the real-time obtained particle data with a preset threshold, the rotation speed of the purification module is dynamically adjusted to ensure that the cleanroom maintains at a qualified level. This method can achieve closed-loop control to a certain extent, but its control strategy is only related to a single particle concentration threshold, without involving the correlation analysis of multi-source environmental parameters and personnel behavior, unable to trace the cause of anomalies and unable to give early warnings about future particle concentration trends.

[0006] In the "Cleanliness Control System of Clean Room" disclosed in the patent publication number CN211375437U, cleanliness data is collected through particle concentration sensors, and the fan filter unit is driven to achieve dynamic speed regulation to save energy and reduce consumption. This utility model focuses on cleanliness control and energy efficiency optimization, but does not adopt deep learning or advanced time series models, lacking in-depth exploration of the comprehensive influence of multiple factors and the time series changes of particle concentration, making it difficult to achieve intelligent diagnosis and decision support under abnormal conditions.

[0007] Although the above technologies have made certain progress in clean room monitoring and control, there are still the following main deficiencies: existing systems mostly focus on single particle concentration measurement or fan speed control, ignoring the combined effects of environmental parameters such as temperature, humidity, pressure difference and personnel activities on particle concentration fluctuations; abnormal diagnosis mostly relies on empirical thresholds or simple statistics, unable to conduct intelligent analysis and positioning of the specific causes of abnormalities; lacking multi-source time series feature extraction and fusion processing based on deep learning, unable to provide accurate prediction and early warning of future particle concentration trends; lacking complete data visualization and online management functions, making it difficult to provide fast and intuitive decision support for clean room maintenance personnel.

[0008] Therefore, there is an urgent need for a particle anomaly monitoring system and method that can collect clean room environmental parameters and personnel entry and exit data in real time, fuse multi-source time series information, and automatically identify the causes of anomalies and predict the trend of particle concentration based on object detection and CNN-LSTM deep learning models, and finally achieve visual management and intelligent decision-making. The present invention is proposed under this technical background to overcome the above deficiencies of the prior art and improve the intelligent level and prediction accuracy of clean room anomaly monitoring. Summary of the Invention

[0009] Aiming at the deficiencies of the existing clean room particle monitoring system that only relies on manual experience or threshold alarm, unable to fuse environmental parameters and personnel behavior for intelligent analysis, and also difficult to predict the future trend of particle concentration, the present invention proposes a particle anomaly monitoring system and monitoring method based on CNN-LSTM; real-time environmental time series data such as temperature, humidity, and pressure difference are collected through sensors, and combined with YoloV11 object detection to obtain personnel entry, exit and stay information. After data preprocessing, it is input into the CNN-LSTM network for multi-source feature extraction and time series modeling to automatically identify the causes of anomalies and predict the future trend of particle concentration; achieving real-time and accurate positioning of the causes of anomalies, early warning of concentration fluctuations and realizing online visual management, thus significantly improving the monitoring accuracy and response speed. The system of the present invention specifically includes the following modules:

[0010] T1: The acquisition module is used to collect clean room environmental parameters in real time through sensors, including time series data of temperature, humidity and pressure difference;

[0011] T2: The target detection module is used to perform real-time detection of personnel in the cleanroom monitoring video by using a target detection neural network, and obtain the moments when personnel enter and leave the cleanroom and the time that personnel stay in the cleanroom, and organize them into structured data;

[0012] T3: The data processing module is used to merge and integrate the environmental parameter data obtained by the data acquisition module and the personnel data obtained by the target detection module into a human-environment data set, and perform preprocessing on the data. The data preprocessing includes missing value processing, outlier processing, duplicate value removal, and Min-Max normalization processing;

[0013] T4: The model construction module is used to construct a CNN-LSTM neural network model. The model includes an input layer, a feature extraction layer, a time series processing layer, a fully connected layer, and an output layer connected in sequence, and specifically includes the following features:

[0014] The feature extraction layer contains a convolutional layer, an activation function, and a pooling layer, and is used to perform convolutional feature extraction, non-linear mapping, and dimensionality reduction on the data. The convolutional feature extraction calculation formula in the convolutional layer is:

[0015]

[0016] where: y l(i,j) is the mapping value of the local area, is the j'-th weight of the i-th convolutional kernel in the l-th layer, x l(j) is the j-th area in the l-th layer, and C is the convolutional kernel width;

[0017] The time series processing layer adopts an LSTM network structure, and captures long-term dependencies in the sequence data by a gating mechanism. Its internal calculation formula is:

[0018] f t = σ(W f · [h t-1 , x t + b f )

[0019] i t = σ(W i · [h t-1 , x t + b i )

[0020]

[0021] o t = σ(W o · [h t-1 , x t + b o )

[0022] ht = o t *tanh(C t )

[0023] Where: f t is the forgetting gate, σ is the activation function, W f is the forgetting gate parameter, h t-1 is the output of the hidden layer at time t-1, x t is the input at time t, b f is the forgetting gate parameter, i t is the input gate, W i is the input gate parameter, b i is the input gate parameter, is the input state at time t, tanh is the activation function, W c is the input state parameter, b c is the input state parameter, C t is the internal state at time t, C t-1 is the internal state at time t-1, o t is the output gate, W o is the output gate parameter, b o is the output gate parameter, h t is the output of the hidden layer at time t;

[0024] T5: The anomaly prediction module is used to map the feature vector output by the CNN-LSTM neural network model to the prediction space through a fully connected layer, obtain the probability distribution of each anomaly category using the Softmax function, and select the category with the highest probability as the prediction result of the particle anomaly cause. At the same time, it predicts the future particle concentration trend in the clean room;

[0025] T6: The data storage module is used to store the environmental parameters, personnel detection data, and anomaly prediction results in the local database and the cloud server respectively, and provide an online visualization management function for the data;

[0026] The above modules are connected through a data bus. The outputs of the data acquisition module and the target detection module are connected to the data processing module. The output of the data processing module is connected to the input of the model construction module. The output of the model construction module is connected to the anomaly prediction module. The output of the anomaly prediction module is connected to the data storage module. Through the cooperation of the above modules, the real-time monitoring, anomaly cause identification, and trend prediction of the particle anomaly monitoring system are realized.

[0027] As a preferred technical solution of the present invention, the data processing module specifically includes the following steps:

[0028] Merge the cleanroom environmental parameter data obtained by the data acquisition module with the personnel data obtained by the target detection module to integrate into a human-environment dataset, where the personnel data specifically includes the duration of personnel staying in the cleanroom within the detection window, the number of times entering the cleanroom within the time window, and the number of times leaving the cleanroom within the time window;

[0029] Organize the human-environment dataset into time series data according to timestamps;

[0030] Use the interpolation method to supplement the missing values in the time series data;

[0031] Use the quartile method to automatically identify the outliers in the time series data, and replace the identified outliers with the mean of the original data;

[0032] Remove the duplicate values in the time series data and only keep one;

[0033] Perform Min-Max normalization on the data to scale the data to the range of [0,1]. The formula is:

[0034]

[0035] Where: x is a sample value in the sequence data, Min is the minimum value in the sequence data, Max is the maximum value in the sequence data, and x' is the output value after normalization;

[0036] Use the sliding window method to construct a data matrix and corresponding labels for the input of the CNN-LSTM neural network model.

[0037] As a preferred technical solution of the present invention, the feature extraction layer in the model construction module further includes performing a non-linear transformation on the extracted features using the ReLU activation function. The formula is:

[0038] ReLU(y l(i,j) ) = max{0, y l(i,j)}

[0039] Where: l is the number of layers of the neural network, (i,j) is the position identifier of the neuron, and y l(i,j) is the output value after the convolution operation.

[0040] As a preferred technical solution of the present invention, the CNN-LSTM neural network model in module T4 specifically includes a first convolutional layer, a first activation layer, a first pooling layer, a second convolutional layer, a second activation layer, a second pooling layer, an LSTM layer, and a fully connected layer connected in sequence from the input end to the output end.

[0041] As a preferred technical solution of the present invention, the kernel size of the first convolutional layer is kernel_size = 3, and the padding parameter is padding = 1; the first activation layer uses the ReLU activation function; the first pooling layer uses a pooling kernel size of pool_size = 2; the kernel size of the second convolutional layer is kernel_size = 3, the stride is stride = 1, and the padding parameter is padding = 1; the second activation layer uses the ReLU activation function; the second pooling layer uses a pooling kernel size of pool_size = 2; the number of hidden units in the LSTM layer is hidden_size = 64, and the dropout rate is dropout = 0.2; the fully connected layer is used to map the output of the LSTM layer to the target prediction space to obtain an output vector.

[0042] As a preferred technical solution of the present invention, the training process of the CNN-LSTM neural network model includes:

[0043] P1: Input the preprocessed data into the first convolutional layer through the input layer for convolutional feature extraction, and output it after being processed by the first activation layer and the first pooling layer;

[0044] P2: Transmit the output of step P1 to the second convolutional layer for further feature extraction, and form a feature vector after being processed by the second activation layer and the second pooling layer;

[0045] P3: Input the feature vector obtained in step P2 into the LSTM layer, and the LSTM layer captures the long-term dependence relationship of the data;

[0046] P4: Input the output of the LSTM layer into the fully connected layer to perform mapping to obtain an output vector;

[0047] P5: Map the output vector obtained in step P4 to an abnormal cause prediction result through the output layer;

[0048] P6: Compare the actual value with the prediction result, and evaluate the prediction performance of the CNN-LSTM model. The evaluation criteria include accuracy, precision, recall, and F1 score; among them, the calculation formulas for accuracy, precision, recall, and F1 score are respectively:

[0049]

[0050] Where: TP represents the number of samples predicted as positive and actually positive; FP represents the number of samples predicted as positive but actually negative; FN represents the number of samples predicted as negative but actually positive; TN represents the number of samples predicted as negative and actually negative; accuracy represents accuracy; precision represents precision; recall represents recall, and F1 represents the F1 score.

[0051] A particle anomaly monitoring method based on CNN-LSTM for operating the above-mentioned system, specifically including the following steps:

[0052] S1: Use the sensors of the data acquisition module to collect time series data of environmental parameters such as temperature, humidity, and differential pressure in the clean room in real time;

[0053] S2: Use the target detection module to detect the monitoring video in the clean room, collect the moments when people enter and leave the clean room and the time people stay in the clean room, and convert them into structured data;

[0054] S3: Use the data processing module to merge the data obtained in the above steps S1 and S2 to form an integrated human-environment dataset, and perform data preprocessing, specifically including interpolation method for handling missing values, quartile method for handling outliers, removing duplicate values, and Min-Max normalization to normalize the data to the range of [0,1];

[0055] S4: Input the human-environment dataset preprocessed in step S3 into the CNN-LSTM neural network model constructed by the model construction module for feature extraction and sequence prediction. First, perform local feature extraction through the CNN network, then capture the long-term dependence relationship in the data through the LSTM network, and finally obtain the prediction output through the mapping of the fully connected layer;

[0056] S5: Use the anomaly prediction module to process the feature vector output by the CNN-LSTM neural network model through the softmax function to obtain the probability distribution, select the category with the highest probability as the anomaly cause prediction result, and simultaneously output the future particle concentration trend of the clean room;

[0057] S6: Use the data storage module to store the collected data and prediction results in the database, and upload them to the cloud server for online visualization management.

[0058] The present invention has the following beneficial effects compared with the prior art:

[0059] The present invention not only collects environmental parameters such as temperature, humidity, and differential pressure in the clean room, but also combines YoloV11 target detection to obtain personnel entry, exit and stay information. Through the construction of the human-environment dataset, the deep integration of the environmental state and personnel behavior is realized, and the coverage range is wider and the monitoring is more comprehensive than a single particle counting or threshold alarm system.

[0060] Introduce the automatic extraction of spatial features by CNN, and capture the time dependence relationship of environmental and personnel activity data through LSTM, so that the system has higher accuracy, precision, recall rate and F1 score in anomaly cause classification and positioning than traditional manual experience discrimination or simple statistical methods.

[0061] The CNN-LSTM model can not only diagnose current anomalies in real time, but also perform regression prediction on the future trend of particle concentration to achieve early warning, helping operation and maintenance personnel take measures in a timely manner and significantly shortening the response time.

[0062] Through the dual storage of the local database and the cloud server, and providing real-time monitoring, historical playback and trend charts on the online page, operation and maintenance personnel can intuitively view environmental changes and anomaly statistics, support rapid positioning of problem areas, and improve operation and maintenance efficiency. Brief Description of the Drawings

[0063] Figure 1 It is a schematic structural diagram of a particle anomaly monitoring system based on CNN-LSTM provided by the present invention;

[0064] Figure 2 It is a schematic flowchart of the data processing method of the embodiment provided by the present invention;

[0065] Figure 3 It is a structural diagram of the CNN-LSTM neural network model of the embodiment provided by the present invention;

[0066] Figure 4 It is a training flowchart of the CNN-LSTM neural network model of the embodiment provided by the present invention. Detailed Embodiments

[0067] The present invention will be further described below with reference to the accompanying drawings. However, the present invention can be implemented in many different ways and should not be construed as limited to the illustrated embodiments; on the contrary, these embodiments provide implementation manners that meet the applicable legal requirements for those skilled in the art.

[0068] Embodiment 1: This embodiment provides a particle anomaly monitoring system based on CNN-LSTM, combined with Figure 1 As shown, the system specifically includes: a data acquisition module, a target detection module, a data processing module, a model construction module, an anomaly prediction module and a data storage module. The specific implementation processes of each module are as follows:

[0069] T1: The data acquisition module collects environmental parameters such as temperature, humidity, and differential pressure in the clean room in real time through sensors arranged at key positions in the clean room, and records them as time-series data with timestamps. Specifically, the arrangement of sensors includes the following methods: Determine the positions of key monitoring points in the clean room, install temperature sensors, humidity sensors, and differential pressure sensors respectively, and set a unified sampling frequency, such as once per minute. At the same time, the data acquisition module also includes encoding and collecting equipment parameters such as the operating mode and instrument status of the equipment in the clean room to form a complete environmental data set. The collected data is transmitted back to the data acquisition server in real time via wireless or wired means and recorded in the database.

[0070] T2: The target detection module uses the YoloV11 target detection model to analyze the clean room monitoring video in real time, detect and record the specific moments and residence durations of personnel entering and leaving the clean room. The specific implementation steps include:

[0071] First, install fixed-position monitoring cameras at different angles in the clean room to collect images of personnel with the characteristics of the clean room working environment (such as wearing dust-proof clothing, headgear, etc.).

[0072] Manually annotate the collected image data to form a customized data set, label the personnel targets and perform training, validation, and test subset division.

[0073] Use the YoloV11 model to train the customized data set. After training, process the monitoring video data in real time on-site in the clean room, detect personnel targets and record the specific moments of each personnel entering and leaving the clean room and the total residence duration of personnel in the clean room to form structured personnel data.

[0074] T3: As Figure 2 shown, the data processing module fuses and preprocesses the environmental parameter data and personnel detection data. The specific steps include:

[0075] Merge the environmental data collected by the above sensors and the personnel data collected by the target detection module to integrate and form a unified human-environment data set, where the personnel data specifically includes the residence duration, entry times, and exit times of personnel within each detection time window.

[0076] Sort the merged data set according to the timestamp to form continuous time-series data.

[0077] For the missing data in the data set, use the interpolation method to supplement it.

[0078] Use the quartile method to automatically identify outliers in the data and correct and replace the outliers with the mean of the same batch of data.

[0079] Remove the duplicate records in the data set and only keep one of them.

[0080] The data is processed by Min - Max normalization and scaled to the range of [0, 1]. The formula is as follows:

[0081]

[0082] By using the sliding window method, the data is divided into matrices and corresponding labels with a set window length to meet the standard format required for the subsequent input of the deep learning model.

[0083] T4: As Figure 3 shown, the model construction module uses a CNN - LSTM deep neural network model to train and predict the data. The model includes an input layer, a feature extraction layer, a time series processing layer, a fully connected layer, and an output layer connected in sequence, and specifically includes the following features:

[0084] The feature extraction layer contains a convolutional layer, an activation function, and a pooling layer, and is used to perform convolutional feature extraction, non - linear mapping, and dimensionality reduction on the data. The calculation formula for convolutional feature extraction in the convolutional layer is:

[0085]

[0086] The time series processing layer uses an LSTM network structure, and the gating mechanism captures the long - term dependencies in the sequence data. The internal calculation formula is:

[0087] f t = σ(W f ·[h t-1 , x t +b f )

[0088] i t = σ(W i ·[h t-1 , x t +b i )

[0089]

[0090] o t = σ(W o ·[h t-1 , x t +b o )

[0091] h t = o t *tanh(C t )

[0092] The abnormal prediction module is used to map the feature vector output by the CNN-LSTM neural network model to the prediction space through a fully connected layer, obtain the probability distribution of each abnormal category using the Softmax function, select the category with the highest probability as the prediction result of the particle abnormal cause, and simultaneously predict the future particle concentration trend in the clean room.

[0093] As Figure 3 shown, the CNN-LSTM neural network model in the model construction module specifically includes a first convolutional layer, a first activation layer, a first pooling layer, a second convolutional layer, a second activation layer, a second pooling layer, an LSTM layer, and a fully connected layer connected in sequence from the input end to the output end. The convolutional kernel size of the first convolutional layer is kernel_size = 3, and the padding parameter is padding = 1; the first activation layer uses the ReLU activation function; the first pooling layer uses a pooling kernel size of pool_size = 2; the convolutional kernel size of the second convolutional layer is kernel_size = 3, the stride is stride = 1, and the padding parameter is padding = 1; the second activation layer uses the ReLU activation function; the second pooling layer uses a pooling kernel size of pool_size = 2; the number of hidden units in the LSTM layer is hidden_size = 64, and the dropout rate is dropout = 0.2; the fully connected layer is used to map the output of the LSTM layer to the target prediction space to obtain an output vector.

[0094] As Figure 4 shown, the training process of the CNN-LSTM neural network model includes:

[0095] P1: Input the preprocessed data into the first convolutional layer through the input layer for convolutional feature extraction, and output it after being processed by the first activation layer and the first pooling layer;

[0096] P2: Transmit the output of step P1 to the second convolutional layer for further feature extraction, and form a feature vector after being processed by the second activation layer and the second pooling layer;

[0097] P3: Input the feature vector obtained in step P2 into the LSTM layer, and the LSTM layer captures the long-term dependence relationship of the data;

[0098] P4: Input the output of the LSTM layer into the fully connected layer to perform mapping to obtain an output vector;

[0099] P5: Map the output vector obtained in step P4 to the abnormal cause prediction result through the output layer;

[0100] P6: Compare the actual values with the predicted results to evaluate the prediction performance of the CNN-LSTM model. The evaluation criteria include accuracy, precision, recall, and F1-score. The calculation formulas for accuracy, precision, recall, and F1-score are as follows:

[0101]

[0102] Where: TP represents the number of samples predicted as positive and actually positive; FP represents the number of samples predicted as positive but actually negative; FN represents the number of samples predicted as negative but actually positive; TN represents the number of samples predicted as negative and actually negative; accuracy represents accuracy; precision represents precision; recall represents recall, and F1 represents the F1-score.

[0103] T5: The anomaly prediction module performs post-processing operations on the model output, specifically including: converting the result output by the fully connected layer into a probability distribution through the Softmax function; obtaining the category with the highest probability in the probability distribution as the predicted anomaly cause; and at the same time determining the future trend of the particle concentration according to the regression output and giving targeted maintenance and processing strategies.

[0104] T6: The data storage module is responsible for storing information such as environmental parameter data, personnel detection data, and anomaly prediction results in the local database and uploading them to the cloud server. Specifically, it includes: recording environmental parameters, personnel data, anomaly causes, and particle concentration trends in the local database; and real-time displaying environmental parameter curves, personnel activity trajectories, anomaly occurrence records, and trend prediction curves through the online page of the cloud server to achieve online monitoring and visualization management, facilitating cleanroom managers to timely grasp the environmental status and make quick decisions.

[0105] Example 2: This example provides a particle anomaly monitoring method based on CNN-LSTM for operating the system described in Example 1, specifically including the following steps:

[0106] S1: Use the sensors of the data acquisition module to continuously collect time series data of environmental parameters such as temperature, humidity, and differential pressure in the cleanroom.

[0107] S2: Use the target detection module to detect the monitoring video in the cleanroom, collect the moments when personnel enter and exit the cleanroom and the time when personnel stay in the cleanroom, and convert them into structured data.

[0108] S3: Use the data processing module to merge the data obtained in the above steps S1 and S2 to form an integrated human-environment dataset, and perform data preprocessing, specifically including handling missing values by interpolation method, handling outliers by quartile method, removing duplicate values, and performing Min-Max normalization to normalize the data to the range of [0, 1];

[0109] S4: Input the human-environment dataset preprocessed in step S3 into the CNN-LSTM neural network model constructed by the model construction module for feature extraction and sequence prediction. First, perform local feature extraction through the CNN network, then capture the long-term dependencies in the data through the LSTM network, and finally obtain the prediction output through mapping by the fully connected layer;

[0110] S5: Use the anomaly prediction module to process the feature vectors output by the CNN-LSTM neural network model through the softmax function to obtain the probability distribution, select the category with the highest probability as the prediction result of the anomaly cause, and simultaneously output the future particle concentration trend of the clean room;

[0111] S6: Use the data storage module to store the collected data and the prediction results in the database, and upload them to the cloud server for online visualization management.

[0112] It should also be noted that the exemplary embodiments mentioned in this application describe some methods or systems based on a series of steps or devices. However, this application is not limited to the order of the above steps. That is, the steps can be executed in the order mentioned in the embodiments, or different from the order in the embodiments, or several steps can be executed simultaneously.

[0113] The above has described various aspects of the present disclosure with reference to the flowcharts and / or block diagrams of methods, apparatuses (systems), and computer program products according to embodiments of the present disclosure. It should be understood that each block in the flowcharts and / or block diagrams, and the combinations of blocks in the flowcharts and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device to generate a machine, such that the instructions executed by the processor of the computer or other programmable data processing device enable the implementation of the functions / actions specified in one or more blocks of the flowchart and / or block diagram. It can also be understood that each block in the block diagram and / or flowchart, and the combinations of blocks in the block diagram and / or flowchart, can also be implemented by dedicated hardware that performs the specified functions or actions, or by a combination of dedicated hardware and computer instructions.

[0114] The above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it; although the present invention has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that: they can still modify the technical solutions recorded in the foregoing embodiments, or perform equivalent replacements on some of the technical features; and these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A particle anomaly monitoring system based on CNN-LSTM, characterized in that: It includes the following modules: T1: Data acquisition module, which collects the environmental parameters of the cleanroom in real time through sensors, including the time series data of temperature, humidity and differential pressure; T2: Target detection module, which uses the target detection neural network to detect the personnel in the cleanroom monitoring video in real time, and obtains the moments when the personnel enter and leave the cleanroom and the time the personnel stay in the cleanroom, and organizes them into structured data; T3: Data processing module, which merges the environmental parameter data collected by the data acquisition module and the personnel data obtained by the target detection module into a human-environment dataset, and preprocesses the data. The data preprocessing includes missing value processing, outlier processing, duplicate value removal and Min-Max normalization processing; T4: Model construction module, which is used to construct a CNN-LSTM neural network model. The model includes an input layer, a feature extraction layer, a time series processing layer, a fully connected layer and an output layer connected in sequence, and specifically includes the following features: The feature extraction layer contains a convolutional layer, an activation function and a pooling layer, which are used to perform convolutional feature extraction, non-linear mapping and dimensionality reduction on the data. The convolutional layer feature extraction calculation formula is: where: y l(i,j) is the mapping value of the local area, is the j'-th weight of the i-th convolutional kernel in the l-th layer, x l(j) is the j-th area in the l-th layer, and C is the convolutional kernel width; The time series processing layer adopts an LSTM network structure, and the gated mechanism captures the long-term dependencies in the sequence data. Its internal calculation formula is: f t = σ(W f · [h t-1 , x t + b f ) i t = σ(W i · [h t-1 , x t + b i ) o t = σ(W o · [h t-1 , x t + b o ) h t = o t *tanh(C t ) Where: f t is the forget gate, σ is the activation function, W f is the forget gate parameter, h t-1 is the output of the hidden layer at time t-1, x t is the input at time t, b f is the forget gate parameter, i t is the input gate, W i is the input gate parameter, b i is the input gate parameter, is the input state at time t, tanh is the activation function, W c is the input state parameter, b c is the input state parameter, C t is the internal state at time t, C t-1 is the internal state at time t-1, o t is the output gate, W o is the output gate parameter, b o is the output gate parameter, h t is the output of the hidden layer at time t; T5: Anomaly prediction module, which maps the feature vector output by the CNN-LSTM neural network model to the prediction space through a fully connected layer, uses the Softmax function to obtain the probability distribution of each anomaly category, and selects the category with the largest probability as the prediction result of the particle anomaly cause. At the same time, it predicts the future particle concentration trend of the cleanroom; T6: Data storage module, which is used to store the environmental parameters, personnel detection data and anomaly prediction results into the local database and the cloud server respectively, and provides an online visualization management function for the data; The above modules are connected through a data bus. The outputs of the data acquisition module and the target detection module are connected to the data processing module. The output of the data processing module is connected to the input of the model construction module. The output of the model construction module is connected to the anomaly prediction module. The output of the anomaly prediction module is connected to the data storage module; Through the cooperation of the above modules, the real-time monitoring, anomaly cause identification and trend prediction of the particle anomaly monitoring system are realized.

2. The particle anomaly monitoring system based on CNN-LSTM according to claim 1, characterized in that: The data processing module specifically includes the following steps: Merge the cleanroom environmental parameter data obtained by the data acquisition module and the personnel data obtained by the target detection module into a human-environment dataset. The personnel data specifically includes the duration of personnel staying in the cleanroom within the detection window, the number of times of entering the cleanroom within the time window and the number of times of leaving the cleanroom within the time window; Organize the human-environment dataset into time series data according to the time stamp; Use the interpolation method to supplement the missing values in the time series data; Automatically identify the outliers in the time series data by the quartile method, and replace the identified outliers with the mean value of the original data; Remove the duplicate values in the time series data and only keep one; Perform Min-Max normalization processing on the data, and scale the data to the range of [0,1]. The formula is: Where: x is a sample value in the sequence data, Min is the minimum value in the sequence data, Max is the maximum value in the sequence data, and x′ is the output value after normalization; Use the sliding window method to construct a data matrix and corresponding labels for input to the CNN-LSTM neural network model.

3. The particle anomaly monitoring system based on CNN-LSTM according to claim 1, characterized in that: The feature extraction layer in the model construction module further includes performing a non-linear transformation on the extracted features using the ReLU activation function, and its formula is: ReLU(y l(i,j) ) = max{0, y l(i,j)} Where: l is the number of layers of the neural network, (i, j) is the position identifier of the neuron, and y l(i,j) is the output value after the convolution operation.

4. The particle anomaly monitoring system based on CNN-LSTM according to claim 1, characterized in that: The CNN-LSTM neural network model in the model construction module specifically includes a first convolutional layer, a first activation layer, a first pooling layer, a second convolutional layer, a second activation layer, a second pooling layer, an LSTM layer, and a fully connected layer connected in sequence from the input end to the output end.

5. The particle anomaly monitoring system based on CNN-LSTM according to claim 4, wherein: The convolutional kernel size of the first convolutional layer is kernel_size = 3, and the padding parameter is padding = 1; the first activation layer uses the ReLU activation function; the first pooling layer uses a pooling kernel size of pool_size = 2; the convolutional kernel size of the second convolutional layer is kernel_size = 3, the stride is stride = 1, and the padding parameter is padding = 1; the second activation layer uses the ReLU activation function; the second pooling layer uses a pooling kernel size of pool_size = 2; the number of hidden units in the LSTM layer is hidden_size = 64, and the dropout rate is dropout = 0.2; the fully connected layer is used to map the output of the LSTM layer to the target prediction space to obtain an output vector.

6. The particle anomaly monitoring system based on CNN-LSTM according to claim 4, characterized in that: The training process of the CNN-LSTM neural network model includes: P1: Input the preprocessed data into the first convolutional layer through the input layer for convolutional feature extraction, and output after being processed by the first activation layer and the first pooling layer; P2: Transmit the output of step P1 to the second convolutional layer for further feature extraction, and form a feature vector after being processed by the second activation layer and the second pooling layer; P3: Input the feature vector obtained in step P2 into the LSTM layer, and the LSTM layer captures the long-term dependence relationship of the data; P4: Input the output of the LSTM layer into the fully connected layer for mapping to obtain an output vector; P5: Map the output vector obtained in step P4 to the abnormal cause prediction result through the output layer; P6: Compare the actual value with the prediction result to evaluate the prediction performance of the CNN-LSTM model. The evaluation criteria include accuracy, precision, recall, and F1 score; among them, the calculation formulas for accuracy, precision, recall, and F1 score are respectively: Where: TP represents the number of samples predicted as positive and actually positive; FP represents the number of samples predicted as positive but actually negative; FN represents the number of samples predicted as negative but actually positive; TN represents the number of samples predicted as negative and actually negative; accuracy represents accuracy; precision represents precision; recall represents recall, and F1 represents the F1 score.

7. A particle anomaly monitoring method based on CNN-LSTM, which is used to operate the system described in any one of claims 1-6, specifically includes the following steps: S1: Use the sensors of the data acquisition module to collect the time series data of the temperature, humidity, and differential pressure environmental parameters in the clean room in real time; S2: Use the target detection module to detect the monitoring video in the clean room, collect the moments when people enter and leave the clean room and the time when people stay in the clean room, and convert them into structured data; S3: Use the data processing module to merge the data obtained in the above steps S1 and S2 to form an integrated human-environment dataset, and perform data preprocessing, specifically including interpolating missing values, treating outliers by the quartile method, removing duplicate values, and Min-Max normalization to normalize the data to the range of [0,1]; S4: Input the human-environment dataset preprocessed in step S3 into the CNN-LSTM neural network model constructed by the model construction module for feature extraction and sequence prediction. First, perform local feature extraction through the CNN network, then capture the long-term dependence relationship in the data through the LSTM network, and finally obtain the prediction output through the mapping of the fully connected layer; S5: Use the anomaly prediction module to process the feature vector output by the CNN-LSTM neural network model through the softmax function to obtain the probability distribution, select the category with the highest probability as the anomaly cause prediction result, and simultaneously output the future particle concentration trend in the clean room; S6: Use the data storage module to store the collected data and prediction results in the database, and upload them to the cloud server for online visualization management.

Images