Air quality anomaly detection method based on random derivative-free multi-error optimization

Abstract

The application discloses an air quality anomaly detection method based on random non-derivative multi-error optimization, which uses two intelligent clustering technologies and three density and distance-based combination calculation strategies to construct and train an SDF-MEOA model, and then calculates the reconstruction error of the automatic encoder based on the air quality detection dataset by using the trained SDF-MEOA, so that air quality anomaly detection is realized. The application uses two intelligent clustering technologies, namely, density peak clustering DPC and self-organizing mapping SOM, to obtain unsupervised clustering of the air quality dataset; based on the air quality normal dataset, a random non-derivative method is used to optimize the loss function of the automatic encoder, so that the SDF-MEOA model is constructed and trained; finally, based on the air quality detection dataset, the SDF-MEOA is applied to calculate the reconstruction error of the automatic encoder, so that air quality anomaly detection is realized. The application can well solve the above problems of the traditional automatic encoder.

Description

Technical Field

[0001] This invention relates to air quality anomaly detection technology, density peak clustering (DPC), and self-organizing map (SOM), specifically to an air quality anomaly detection method based on a stochastic derivative-free multi-error-optimized autoencoder (SDF-MEOA). Background Technology

[0002] Existing research has identified various methods for detecting outliers, such as clustering-based techniques, nearest neighbor-based methods, and deep learning-based models.

[0003] Clustering-based anomaly detection techniques group data based on similarity or similar patterns, considering data not belonging to any category as outliers. Common clustering methods for detecting outliers and noise points include: Density-Based Spatial Clustering of Applications with Noise (DBSCAN), which can be used for noise-robust datasets, finding all dense regions of sample points and treating these regions as clusters; Ordering Points To Identify the Clustering Structure (OPTICS), which can find clusters of arbitrary shapes in large-scale data and has good robustness to noise points; and Cluster-Based Local Outlier Factor (CBLOF), which uses the idea of ​​local outlier factor detection to detect outliers based on clustering. The core objective of clustering-based methods is to identify the structure of clusters; therefore, these methods may miss outlier data.

[0004] Nearest neighbor-based anomaly detection methods fall into two categories: distance-based and density-based methods. Distance-based methods detect anomalies by calculating the distance between anomalous and normal data. Density-based methods compare the estimated density of each data point with its nearest neighbors, identifying data with lower estimated densities as outliers. The Local Outlier Factor (LOF) method calculates the relative isolation of given data, but its performance is very poor on dispersed datasets. Furthermore, the algorithm's performance degrades when the density of anomalous data points is close to the density of their neighborhoods and boundary data. To further improve the efficiency of this method, researchers have proposed improvements such as Connectivity-Based Outlier Factor (COF), Influence-Based Local Outlier Factor (INFLO), Label-Driven Outlier Factor (LDOF), and Local Correlation-based Intensity (LOCI).

[0005] Deep learning models are currently an effective technique in the field of anomaly detection, and deep learning-based models have been applied in supervised, semi-supervised, and unsupervised modes. In supervised mode, the model is trained using normal data, and the trained model is used for anomaly detection. The key problem in this mode is obtaining accurate labels for inliers and anomalous data classes in each domain. In semi-supervised mode for anomaly detection, class labels are only available for inliers. Unsupervised mode is widely applicable because it can handle unlabeled datasets. In this mode, deep learning-based networks attempt to reconstruct the input at the output, measuring the reconstruction error to rank outliers across all data in the dataset. Many deep learning techniques, such as Adaptive Resonance Theory (ART), Generative Adversarial Networks (GANs), and Restricted Boltzmann Machines (RBMs), have been applied to anomaly detection. Besides the networks mentioned above, Recurrent Neural Networks (RNNs) are also very popular. The core method of unsupervised deep learning models is the autoencoder. Figure 1The basic architecture of an autoencoder is described. The mechanism of an autoencoder mainly consists of three layers: encoder, compression (hiding), and decoder. In the encoder stage, the autoencoder compresses the data to find useful information (compression layer); the decoder works in the opposite way to the encoder, reconstructing the input features as closely as possible. Minimizing the reconstruction error is the main goal of the autoencoder. Autoencoders can be applied to various purposes, including anomaly detection, image denoising, and data compression (dimensionality reduction). In anomaly detection using autoencoders, anomalous data has a higher reconstruction error compared to normal data. Later, researchers made a series of improvements to autoencoders, developing models such as Variational Autoencoder (VAE), Gaussian Mixture Variational Autoencoder (GMVAE), Randomized Neighbor Network (RandNet), Bayesian Autoencoder (BAE), and Graph Autoencoders (GAE), which are widely used for anomaly detection.

[0006] When using traditional autoencoders for air quality anomaly detection, the reconstruction process is based on the entire air quality detection dataset, including both normal and abnormal datasets. This approach suffers from the drawback of overestimating the reconstruction error of the normal dataset and underestimating the error of the abnormal dataset, causing traditional autoencoders to miss genuine air quality anomaly data. Furthermore, traditional autoencoders use a single loss function—mean squared error (MSE), root mean squared error (RMSE), or mean absolute error (MAE)—which is unsuitable for scenarios with multiple types of air quality anomalies (including low-density data close to dense areas, low-density data far from dense areas, and data very close to dense areas). Therefore, this invention proposes an air quality anomaly detection method based on SDF-MEOA, which effectively addresses the aforementioned problems of traditional autoencoders. Summary of the Invention

[0007] The problem addressed by this invention is to propose an air quality anomaly detection method based on a stochastic derivative-free multi-error-optimized autoencoder (SDF-MEOA). This method uses two intelligent clustering techniques and three combined computational strategies based on density and distance to divide the initial air quality detection dataset into a normal air quality dataset and an air quality anomaly dataset. Based on the normal air quality dataset, a stochastic derivative-free method is used to optimize the loss function of the autoencoder, construct and train the SDF-MEOA model, and then, based on the air quality detection dataset, the reconstruction error of the autoencoder is calculated using the trained SDF-MEOA model to achieve air quality anomaly detection. First, this invention uses two intelligent clustering techniques—Density Peak Clustering (DPC) and Self-Organizing Map (SOM)—to obtain unsupervised clustering of the air quality dataset. Second, it employs three different density- and distance-based combined calculation strategies to derive the Probable Outlier Score (POS), selecting three categories of air quality anomalies: low density but close to dense areas, low density and far from dense areas, and very close to dense areas. This divides the air quality detection dataset into normal air quality datasets and abnormal air quality datasets. Next, based on the normal air quality dataset, a stochastic derivative-free method is used to optimize the loss function of the autoencoder, constructing and training the SDF-MEOA model. Finally, based on the air quality detection dataset, SDF-MEOA is applied to calculate the reconstruction error of the autoencoder, achieving air quality anomaly detection. The specific steps are as follows:

[0008] (1) Data preparation: An air quality monitoring dataset is established based on the air quality monitoring data collected from the air quality monitoring stations, and the dataset is stored and preprocessed.

[0009] (2) Data Preprocessing: To ensure the accuracy of analysis and modeling, the selected air quality anomaly data were first cleaned, specifically including the handling of missing and outlier values. For missing values ​​with a short time span, linear or quadratic interpolation was used to fill the missing values; for missing values ​​with a long time span, data from the same time period within a nearby date was used to fill the missing values; data with obvious anomalies were replaced or deleted. Some data types were converted, and numerical information was date-converted to facilitate subsequent processing.

[0010] (3) Intelligent clustering using DPC and SOM techniques, specifically including:

[0011] (a) Initial definition: Air quality monitoring station d collects air quality monitoring dataset X = {x1, x2, ..., x...} over a continuous period of time. n In}, x i ∈R d Let R represent the i-th data item. d There are n data points in the dataset. The cluster to which the air quality monitoring dataset belongs after clustering is represented by C. j The normal air quality dataset is represented as X. normal ∈X, the air quality anomaly dataset is represented as X abnormal ∈X.

[0012] (b) Calculate x for each data point i Local density (x) i ), and x i distance(x) from dense area i These two features are used to progressively expand each density peak data point, identifying potential outliers from each cluster, and are calculated as follows:

[0013]

[0014] (1) In the formula, d ie It is x i To C j Divide x i All other data x e The distance, where D is the discriminant function. It depends on cluster C j The cutoff distance between the mean and standard deviation of the data.

[0015]

[0016] (2) In the formula, for the i-th data, its density threshold is set to d. ρ Let δ(x) be the sum of all other data within a region centered on this data point and with radius r. i ).

[0017] (4) The POS was calculated using three strategies, and the air quality monitoring dataset was divided, specifically including:

[0018] (a) In the first strategy, in order to find air quality anomalies that are locally low in density but close to dense areas, the x value of each data point in the air quality monitoring dataset is calculated. i POS (1) The calculation formula is as follows:

[0019] POS (1) (x i) = density(x i )*distance(x i (3)

[0020] Find the POS of each data point in the cluster. (1) Then, the data is sorted in ascending order, and the first N data items are selected as cluster C. j Possible outlier data in the data, denoted as probable_outlier_point. (1) The selection of N follows the "3σ principle" of the normality test, which considers data points exceeding 3σ as outliers. Here, σ represents the standard deviation of the first dimension of data (PM2.5 observations) in the air quality monitoring dataset X. The selection method for N is the same in the second and third strategies.

[0021] (b) In the second strategy, in order to identify data with low density and very far from dense areas, the x value for each data point in the air quality monitoring dataset is calculated. i POS (2) The calculation formula is as follows:

[0022]

[0023] Find the POS of each data point in the cluster. (2) Then, the data is sorted in ascending order, and the first N data points are selected as cluster C. j Possible outlier data in the data, denoted as probable_outlier_point. (2) .

[0024] (c) In the third strategy, in order to find air quality anomaly data that are very close to dense areas, the Gaussian distribution function of the characteristic distance is used to calculate x for each data point in the air quality monitoring dataset. i POS (3) The calculation formula is as follows:

[0025] POS (3) =density(x i )*f(distance(x i (5)

[0026] The Gaussian distribution function of the feature distance is:

[0027]

[0028] Where, σ d and μ represent the local density distance(x) and μ, respectively. i The standard deviation and mean of the data. Find the POS of each data point in the cluster. (3)Then, the data is sorted in ascending order, and the first N data items are selected as cluster C. j Possible outlier data in the data, denoted as probable_outlier_point. (3) .

[0029] (d) Add the list probable_outlier_point (1) probable_outlier_point (2) and probable_outlier_point (3) The selected data are combined and represented as:

[0030] probable_outlier_point=probable_outlier_point (1) ∪probable_outlier_point (2) ∪probable_outlier_point (3) (7)

[0031] (e) Subtract the air quality anomaly dataset from the air quality monitoring dataset, i.e., X normal =XX abnormal This yields a normal air quality dataset, which is then used to divide the air quality monitoring dataset.

[0032] (5) Training the SDF-MEOA model based on a normal air quality dataset, specifically including:

[0033] (a) Objective function using a combination of Mean Absolute Error (MAE) and Root Mean Square Error (RMSE) The specific formulas for calculating MAE, RMSE, and the objective function are as follows:

[0034]

[0035]

[0036]

[0037] Where, x i This represents the i-th multidimensional data vector in the input. Let n represent the i-th output vector (the reconstructed input), where n1 is the size of the normal air quality dataset. Assume the objective function is... It is bounded.

[0038] (b) Given an initial iteration point Let be the mean vector of the normal air quality dataset, Δ be the radius of the trust region, and Δ0 = Δ min >0, eta2>eta1>0, γ2≥1>γ1>0, θ>0, p>0, k>0, Δ max The trust region defines a region around the current optimization iteration where the model is considered to adequately represent the objective function.

[0039] (c) Solving the function stochastic gradient (g) i ) k When k = 1, f is the value of f. MAE When k=2, it is f RMSE , (g i ) k This represents the stochastic gradient to the i-th data point corresponding to the k-th sub-cost function.

[0040] (d) For k = 1, 2, solve the following problems:

[0041]

[0042] Obtain the search direction (d) i )1,(d i )2, (d i ) k d represents the i-th search direction corresponding to the k-th objective function. i Indicates the direction of the linear combination search for the i-th data.

[0043] (e) Let Where α k ≥0, k=1,2,…,m, and

[0044] (f) Calculation:

[0045]

[0046] in,

[0047]

[0048]

[0049] (g) If ρ k If ≥η1, then update the iteration point; otherwise, do not update.

[0050] (h) Correct the trust region radius using the following formula:

[0051]

[0052] (i) Increment the iteration count by 1 and return to step (d) until the trust region radius converges.

[0053] (j) The rectified linear unit (ReLU) and hyperbolic tangent (tanh) are used as activation functions for the SDF-MEOA model.

[0054] ReLU:f(x i )=max{0,x i} (16)

[0055]

[0056] Where x represents the input vector.

[0057] (6) Applying the SDF-MEOA model to calculate the reconstruction error of the automatic encoder to achieve air quality anomaly detection, specifically including:

[0058] (a) Based on the air quality monitoring dataset, the SDF-MEOA model is applied to calculate and output the reconstruction error for each data point.

[0059] (b) Based on the calculated reconstruction error, the data are sorted in descending order. The first N data points are identified as air quality anomalies. Here, N is selected according to the "3σ rule" of the normality test, that is, data points exceeding 3σ are considered outliers, where σ represents the standard deviation of the reconstruction error. Attached Figure Description

[0060] Figure 1 This is a basic structural diagram of an automatic encoder.

[0061] Figure 2 This is a framework diagram of an air quality anomaly detection method based on a random, derivative-free, multi-error optimized automatic encoder. Detailed Implementation

[0062] (1) Data preparation: An air quality monitoring dataset is established based on the air quality monitoring data collected from the air quality monitoring stations, and the dataset is stored and preprocessed.

[0063] (2) Data Preprocessing: To ensure the accuracy of analysis and modeling, the selected air quality anomaly data were first cleaned, specifically including the handling of missing and outlier values. For missing values ​​with a short time span, linear or quadratic interpolation was used to fill the missing values; for missing values ​​with a long time span, data from the same time period within a nearby date was used to fill the missing values; data with obvious anomalies were replaced or deleted. Some data types were converted, and numerical information was date-converted to facilitate subsequent processing.

[0064] (3) Intelligent clustering using DPC and SOM techniques, specifically including:

[0065] (a) Initial definition: Air quality monitoring station d collects air quality monitoring dataset X = {x1, x2, ..., x...} over a continuous period of time. n In}, x i ∈R d Let R represent the i-th data item. d There are n data points in the dataset. The cluster to which the air quality monitoring dataset belongs after clustering is represented by C. j The normal air quality dataset is represented as X. normal ∈X, the air quality anomaly dataset is represented as X abnormal ∈X.

[0066] (b) Calculate x for each data point i Local density (x) i ), and x i distance(x) from dense area i These two features are used to progressively expand each density peak data point, identifying potential outliers from each cluster, and are calculated as follows:

[0067]

[0068] (1) In the formula, d ie It is x i To C j Divide x i All other data x e The distance, where D is the discriminant function. It depends on cluster C j The cutoff distance between the mean and standard deviation of the data.

[0069]

[0070] (2) In the formula, for the i-th data, its density threshold is set to d. ρ Let δ(x) be the sum of all other data within a region centered on this data point and with radius r. i ).

[0071] (4) The POS was calculated using three strategies, and the air quality monitoring dataset was divided, specifically including:

[0072] (a) In the first strategy, in order to find air quality anomalies that are locally low in density but close to dense areas, the x value of each data point in the air quality monitoring dataset is calculated. i POS (1) The calculation formula is as follows:

[0073] POS (1) (x i ) = density(x i )*distance(x i (3)

[0074] Find the POS of each data point in the cluster. (1) Then, the data is sorted in ascending order, and the first N data items are selected as cluster C. j Possible outlier data in the data, denoted as probable_outlier_point. (1) The selection of N follows the "3σ principle" of the normality test, which considers data points exceeding 3σ as outliers. Here, σ represents the standard deviation of the first dimension of data (PM2.5 observations) in the air quality monitoring dataset X. The selection method for N is the same in the second and third strategies.

[0075] (b) In the second strategy, in order to identify data with low density and very far from dense areas, the x value for each data point in the air quality monitoring dataset is calculated. i POS (2) The calculation formula is as follows:

[0076]

[0077] Find the POS of each data point in the cluster. (2) Then, the data is sorted in ascending order, and the first N data points are selected as cluster C. j Possible outlier data in the data, denoted as probable_outlier_point. (2) .

[0078] (c) In the third strategy, in order to find air quality anomaly data that are very close to dense areas, the Gaussian distribution function of the characteristic distance is used to calculate x for each data point in the air quality monitoring dataset. i POS (3) The calculation formula is as follows:

[0079] POS (3) =density(x i )*f(distance(c i (5)

[0080] The Gaussian distribution function of the feature distance is:

[0081]

[0082] Where, σ d and μ represent the local density distance(x) and μ, respectively.i The standard deviation and mean of the data. Find the POS of each data point in the cluster. (3) Then, the data is sorted in ascending order, and the first N data items are selected as cluster C. j Possible outlier data in the data, denoted as probable_outlier_point. (3) .

[0083] (d) Add the list probable_outlier_point (1) probable_outlier_point (2) and probable_outlier_point (3) The selected data are combined and represented as:

[0084] probable_outlier_point=probable_outlier_point (1) ∪probable_outlier_point (2) ∪probable_outlier_point (3) (7)

[0085] (e) Subtract the air quality anomaly dataset from the air quality monitoring dataset, i.e., X normal =XX abnormal This yields a normal air quality dataset, which is then used to divide the air quality monitoring dataset.

[0086] (5) Training the SDF-MEOA model based on a normal air quality dataset, specifically including:

[0087] (a) Objective function considering both Mean Absolute Error (MAE) and Root Mean Square Error (RMSE) The specific formulas for calculating MAE, RMSE, and the objective function are as follows:

[0088]

[0089]

[0090]

[0091] Where, x i This represents the i-th multidimensional data vector in the input. Let n represent the i-th output vector (the reconstructed input), where n1 is the size of the normal air quality dataset. Assume the objective function is... It is bounded.

[0092] (b) Given an initial iteration point Let be the mean vector of the normal air quality dataset, Δ be the radius of the trust region, and Δ0 = Δ min >0, eta2>eta1>0, γ2≥1>γ1>0, θ>0, p>0, k>0, Δ max The trust region defines a region around the current optimization iteration where the model is considered to adequately represent the objective function.

[0093] (c) Solving the function stochastic gradient (g) i ) k When k = 1, f is the value of f. MAE When k=2, it is f RMSE , (g i ) k This represents the stochastic gradient to the i-th data point corresponding to the k-th sub-cost function.

[0094] (d) For k = 1, 2, solve the following problems:

[0095]

[0096] Obtain the search direction (d) i )1,(d i )2, (d i ) k d represents the i-th search direction corresponding to the k-th objective function. i Indicates the direction of the linear combination search for the i-th data.

[0097] (e) Let Where α k ≥0, k=1,2,…,m, and

[0098] (f) Calculation:

[0099]

[0100] in,

[0101]

[0102]

[0103] (g) If ρ k If ≥η1, then update the iteration point; otherwise, do not update.

[0104] (h) Correct the trust region radius using the following formula:

[0105]

[0106] (i) Increment the iteration count by 1 and return to step (d) until the trust region radius converges.

[0107] (j) The rectified linear unit (ReLU) and hyperbolic tangent (tanh) are used as activation functions for the SDF-MEOA model.

[0108] ReLU:f(x i )=max{0,x i} (16)

[0109]

[0110] Where x represents the input vector.

[0111] (6) Applying the SDF-MEOA model to calculate the reconstruction error of the automatic encoder to achieve air quality anomaly detection, specifically including:

[0112] (a) Based on the air quality monitoring dataset, the SDF-MEOA model is applied to calculate and output the reconstruction error for each data point.

[0113] (b) Based on the calculated reconstruction error, the data are sorted in descending order. The first N data points are identified as air quality anomalies. Here, N is selected according to the "3σ rule" of the normality test, that is, data points exceeding 3σ are considered outliers, where σ represents the standard deviation of the reconstruction error.

Claims

1. An air quality anomaly detection method based on random derivative-free multi-error optimization, characterized in that: Includes the following steps: Step (1): Collect air quality monitoring data from air quality monitoring stations, store and pre-process the data to form an air quality monitoring dataset; Step (2): Based on the air quality monitoring dataset, intelligent clustering is performed using DPC and SOM techniques, allowing... This is a clustering result dataset obtained after applying two intelligent clustering techniques to an air quality monitoring dataset. Air quality anomaly detection is a binary classification problem. Step (3) uses three different density and distance-based combined calculation strategies to obtain POS, and selects three types of air quality abnormal data: low density but close to dense area, low density and far from dense area, and very close to dense area. The air quality detection dataset is divided into normal air quality dataset and abnormal air quality dataset. Step (4): Based on the normal air quality dataset, a random derivative-free method is used to optimize the loss function of the autoencoder, and the SDF-MEOA model is constructed and trained. Step (5): Based on the air quality detection dataset, apply the trained SDF-MEOA model to calculate the reconstruction error of the autoencoder, perform air quality anomaly detection, and obtain the final air quality anomaly detection data. Step (4) of training the SDF-MEOA model based on the normal air quality dataset includes: (1) Using a combined objective function of mean absolute error (MAE) and root mean square error (RMSE) The specific formulas for calculating MAE, RMSE, and the objective function are as follows: in, Indicates the input number of the first... A multidimensional data vector, Indicates the first Output vectors, Given the size of the normal air quality dataset, and assuming the objective function... It is bounded; (2) Given an initial iteration point This is the mean vector of the normal air quality dataset. For the trust region radius, , , , , Trust region defines a region surrounding the current optimization iteration where the model is considered to adequately represent the objective function. (3) Solving the function stochastic gradient , At that time , At that time , Represents the corresponding number The first sub-cost function The stochastic gradient of the data; (4) For Solve the following problems: Obtain search direction , Indicates the corresponding to the first The first objective function corresponding to the th objective function One search direction, Indicates the first Linear combination of data points for search direction; (5) Order ,in ,and ; (6) Calculate: in, (7) If If the iteration point is updated, then update the iteration point; otherwise, do not update. (8) Correct the trust region radius using the following formula. (9) Increment the iteration count by 1 and return to step (4) until the trust region radius converges; (10) Rectified linear unit ReLU and hyperbolic tangent tanh are used as activation functions for SDF-MEOA; in This represents the input vector.

2. The air quality anomaly detection method based on random derivative-free multi-error optimization according to claim 1, characterized in that, The data preprocessing in step (1) includes: (1) Handling missing and outlier values; For missing values ​​with a small time span, use linear interpolation or quadratic interpolation to fill the missing values; For missing values ​​with a long time span, use data from the same time period within the nearest date to fill the missing values; Replace or delete data with obvious anomalies; (2) Convert some data types and convert numerical information to dates.

3. The air quality anomaly detection method based on random derivative-free multi-error optimization according to claim 1, characterized in that, The intelligent clustering using DPC and SOM techniques in step (2) includes: (1) Initial definition: air quality monitoring station Air quality monitoring datasets were collected over a continuous period of time. middle, Indicates the first data, The CCP The clusters to which the air quality monitoring dataset belongs after clustering are represented as follows: The normal air quality dataset is represented as: The air quality anomaly dataset is represented as ; (2) Calculate each data point Local density ,as well as Distance from dense areas Two features are used to progressively expand each density peak data point to identify potential outliers from each cluster, calculated as follows: In equation (1), yes arrive Internal All other data except distance, It is a discriminant function. It depends on the cluster The cutoff distance between the mean and standard deviation of the data; In equation (2), for the first Data points, with a density threshold set to [value]. Centered on this data, All other data within a radius of are .

4. The air quality anomaly detection method based on random derivative-free multi-error optimization according to claim 1, characterized in that, In step (3), three strategies are used to calculate the POS and to divide the air quality monitoring dataset, including: (1) In the first strategy, in order to find air quality anomaly data with low local density but close to dense areas, the data in the air quality monitoring dataset is calculated. of The calculation formula is as follows: Find each data point in the cluster. Then, the data is sorted in ascending order, and the first N data entries are selected as the cluster. Possible outliers in the data, and represent them as follows: The selection of N follows the "3σ rule" of the normality test, which considers data points exceeding 3σ to be outliers, where σ represents the air quality monitoring dataset. The standard deviation of the first dimension of data is used, and the method of selecting N is the same in the second and third strategies; (2) In the second strategy, in order to identify data with low density and very far from dense areas, the data in the air quality monitoring dataset is calculated. of The calculation formula is as follows: Find each data point in the cluster. Then, the data is sorted in ascending order, and the first N data items are selected as the cluster. Possible outliers in the data, and represent them as follows: ; (3) In the third strategy, in order to find air quality anomaly data that are very close to dense areas, the Gaussian distribution function of the characteristic distance is used to calculate the air quality detection dataset for each data point. of The calculation formula is as follows: The Gaussian distribution function of the feature distance is: in, and Representing local density respectively The standard deviation and mean; finding the standard deviation and mean of each data point in the cluster; Then, the data is sorted in ascending order, and the first N data entries are selected as the cluster. Possible outliers in the data, and represent them as follows: ; (4) List , and The selected data are combined and represented as: (5) Subtract the air quality anomaly dataset from the air quality monitoring dataset, i.e. This yields a normal air quality dataset, which is then used to divide the air quality monitoring dataset.

5. The air quality anomaly detection method based on random derivative-free multi-error optimization according to claim 1, characterized in that, In step (5), the SDF-MEOA model is applied to calculate the reconstruction error of the automatic encoder to achieve air quality anomaly detection, including: (1) Based on the air quality monitoring dataset, the SDF-MEOA model is applied to calculate and output the reconstruction error of each data point; (2) Based on the calculated reconstruction error of the data, sort the data in descending order. The first N data are judged to be abnormal air quality data; the selection of N follows the 3σ principle of normality test.

Images