Urban environmental pollutant data evaluation method and system

Abstract

The invention discloses an urban environmental pollutant data evaluation method and system, and the method comprises the following operation steps: collecting original data through a multi-source sensor, carrying out the time synchronization, and outputting an initial urban environmental pollutant data set with a unified frequency; performing pollutant concentration abnormal value and standardization preprocessing on the initial urban environmental pollutant data set to obtain an urban environmental pollutant data set; performing parameter initialization analysis on the urban environmental pollutant data set, and outputting to obtain an initialization parameter set; performing pollutant concentration anomaly detection on the initial parameter set by using an MCUSUM method to obtain a pollutant concentration anomaly detection result; and generating a pollutant concentration anomaly detection report according to the pollutant concentration anomaly detection result.

Description

Technical Field

[0001] The present invention relates to the field of environmental identification, and in particular to a method and system for evaluating urban environmental pollutant data. Background Art

[0002] The collection and processing of urban environmental pollutant data relies on a single environmental sensor. This approach has problems such as single data dimension, poor time synchronization, and susceptibility to noise and outliers, resulting in low data quality and difficulty in accurately reflecting the true state of the urban environment. In addition, data collected by different sensors often have inconsistent dimensions and large differences in statistical characteristics, which brings great challenges to subsequent data analysis and anomaly detection.

[0003] Research has found that most environmental pollution detections collect raw data through multi-source sensors, perform time synchronization, and output an initial urban environmental pollutant data set with a unified frequency. This step integrates environmental sensor data from different geographical locations or system modules to achieve multi-dimensional data collection and high-precision time synchronization; secondly, the initial urban environmental pollutant data set is preprocessed for pollutant concentration outliers and standardization. Outliers are identified and processed by calculating the corrected Z-score, and appropriate standardization methods are selected based on the skewness of the data to ensure that data from different sensors are compared on the same dimension. This step improves the quality of the data and reduces the impact of noise and outliers on subsequent analysis.

[0004] However, the existing urban environmental pollutant data assessment methods still have certain limitations, such as insufficient robustness in outlier processing and weak adaptability of standardization methods. Summary of the invention

[0005] The purpose of the present invention is to provide a method and system for evaluating urban environmental pollutant data, which solves the above-mentioned technical problems pointed out in the prior art.

[0006] The present invention provides a method for evaluating urban environmental pollutant data, comprising the following steps:

[0007] Collect raw data through multi-source sensors, synchronize time, and output an initial urban environmental pollutant data set with a unified frequency;

[0008] Performing pollutant concentration abnormal value and standardization preprocessing on the initial urban environmental pollutant data set to obtain an urban environmental pollutant data set;

[0009] Performing parameter initialization analysis on the urban environmental pollutant data set and outputting an initialization parameter set;

[0010] The MCUSUM method is used to perform pollutant concentration anomaly detection on the initialization parameter set to obtain a pollutant concentration anomaly detection result; and a pollutant concentration anomaly detection report is generated for the pollutant concentration anomaly detection result.

[0011] Preferably, the parameters of the urban environmental pollutant data set are initialized and analyzed to obtain an initialization parameter set, and the specific operation steps are as follows:

[0012] Set an initialization window w, the size of which is 100;

[0013] Calculate the current parameter mean μw and the current parameter standard deviation σw in the urban environmental pollutant data set within the initialization window w;

[0014] The parameter mean μw is:

[0015] μ w =mean(x(t-w+1):t);

[0016] Where t is the current time point;

[0017] t-w+1 represents the index starting from the current time point and moving back w time points;

[0018] The standard deviation σw of the parameter is:

[0019] σ w =std(x(t-w+1):t);

[0020] The current parameter mean μw and the current parameter standard deviation σw are used to calculate the parameter change rate in the urban environmental pollutant data set within the initialization window w:

[0021] The relative change rate between the current parameter mean μw and the parameter mean of the previous window is rμ:

[0022]

[0023] The relative change rate between the current parameter standard deviation σw and the parameter standard deviation of the previous window is rσ:

[0024]

[0025] Preset the ε threshold, and compare the maximum value of the relative change rate of the parameter mean rμ and the relative change rate of the parameter standard deviation rσ with the preset ε threshold;

[0026] If max(rμ, rσ)<ε, and w≥100, the current parameter mean μw and the current parameter standard deviation σw remain stable within the initialization window, and the current initialization window w is used to initialize the parameters of the urban environmental pollutant data set;

[0027] Then set: μ0=μw; σ0=σw;

[0028] If it is unstable, increase the initialization window until the current parameter mean μw and the current parameter standard deviation σw remain stable within the initialization window, and obtain the initialization parameter set = {μ0, σ0, w}.

[0029] Preferably, the MCUSUM method is used to perform pollutant concentration anomaly detection on the initialization parameter set to obtain a pollutant concentration anomaly detection result. The specific operation steps are as follows:

[0030] Using the MCUSUM method to determine the observed values ​​of pollutant concentrations and the preliminary control limit h value for the observed parameters in the urban environmental pollutant data set;

[0031] Calculating a reference value for the observed pollutant concentration;

[0032] The formula for the reference value is:

[0033] In the formula, δ is the minimum offset of the expected detection data;

[0034] σ0 is the standard deviation of the initialization parameter set;

[0035] k represents the allowed offset;

[0036] The target average run length ARL0 is preset; the preliminary control limit h value is repeatedly adjusted through Monte Carlo simulation, the average run length ARL of the first warning is monitored to be close to the preset target average run length ARL0, and finally the control limit h value matching the actual pollutant concentration data is output;

[0037] The initialization cumulative sum variable of the CUSUM method is used to calculate the standardized deviation of the observed values, and the CUSUM statistic for detecting the deviation of pollutant concentration is obtained;

[0038] When the CUSUM statistic exceeds the control limit h value, an alarm is triggered, the abnormal pollutant concentration is recorded, and the abnormal pollutant concentration detection result is obtained.

[0039] Preferably, the MCUSUM method is used to determine the pollutant concentration observation value and the preliminary control limit h value for the observation parameters in the urban environmental pollutant data set. The specific operation steps are as follows:

[0040] The MCUSUM method is used to monitor the observation parameters in the urban environmental pollutant data set, and the PCA method is used to extract key features;

[0041] Using expert scoring to evaluate the importance score of the key features to obtain an importance score;

[0042] The pollutant features are screened out according to the importance scores of the key features, and the pollutant concentration observation values ​​of the pollutant features are determined;

[0043] Standardizing the urban environmental pollutant data set, calculating the covariance matrix, performing eigenvalue decomposition on the covariance matrix, and extracting eigenvalues ​​and eigenvectors;

[0044] selecting a principal component according to the eigenvalue;

[0045] Set the preliminary control limit h value, and construct a multivariate control chart with the preliminary control limit h value and the selected principal component;

[0046] Monitor the relationship between the observed value of the pollutant characteristic and the preliminary control limit h value according to the multivariate control chart; determine whether the observed value of the pollutant characteristic is greater than the preliminary control limit h value;

[0047] If yes, the pollution concentration of the pollutant characteristic exceeds the preliminary control limit h value, and the preliminary control limit h value is used as the preliminary control limit threshold;

[0048] If not, reset the preliminary control limit h value to judge with the observed value of the pollutant characteristic until the observed value of the pollutant characteristic is greater than the preliminary control limit h value, and use the preliminary control limit h value as the preliminary control limit threshold.

[0049] Preferably, the observation parameters are pollution concentration data points collected in real time.

[0050] Preferably, a target average run length ARL0 is preset; the preliminary control limit h value is repeatedly adjusted through Monte Carlo simulation, the average run length ARL at which the first warning appears is monitored to be close to the preset target average run length ARL0, and finally the control limit h value matching the actual pollutant concentration data is output. The specific operation steps are as follows:

[0051] Collecting n times of historical pollutant concentration data, extracting different simulation identification states for each historical pollutant concentration data; selecting multiple historical pollutant concentration control limit h values ​​according to the different simulation identification states to form an initial candidate set;

[0052] The initial candidate set is subjected to a random number sequence of normal distribution N(μ0,σ), and the random number sequence is monitored and collected by the CUSUM algorithm to obtain the average run length ARL that triggers the warning; the average run length ARL is simulated and calculated n times according to the random number sequence, and the historical pollutant concentration control limit h value of the average run length ARL is recorded for each simulation calculation;

[0053] Sort all recorded historical pollutant concentration control limit h values ​​in descending order; determine the matching degree between the average run length ARL corresponding to the historical pollutant concentration control limit h value and the preset target average run length ARL0 in descending order;

[0054] Preset matching degree e; determine whether the matching degree between the average running length ARL corresponding to the historical pollutant concentration control limit h value and the preset target average running length ARL0 is greater than the preset matching degree e;

[0055] If so, the average run length ARL successfully matches the preset target average run length ARL0, and the two are close to each other. The historical pollutant concentration control limit h value adjusted by the average run length ARL is determined to be the real data threshold, and the control limit h value that matches the real pollutant concentration data is output;

[0056] If not, the matching degree between the average running length ARL corresponding to the historical pollutant concentration control limit h value and the preset target average running length ARL0 is determined one by one in descending order until the historical pollutant concentration control limit h value with an average running length ARL greater than the preset matching degree e is found, and the historical pollutant concentration control limit h value is updated, and the historical pollutant concentration control limit h value is used as the control limit h value that matches the actual pollutant concentration data.

[0057] Preferably, historical pollutant concentration data is collected n times, and different simulation identification states are extracted for each historical pollutant concentration data; multiple historical pollutant concentration control limit h values ​​are selected according to the different simulation identification states to form an initial candidate set. The specific operation steps are as follows:

[0058] Determine the collection time range, collect historical pollutant concentration data n times, and obtain a historical pollutant concentration data set;

[0059] Calculating skewness and kurtosis using the mean and standard deviation;

[0060] The skew expression is:

[0061]

[0062] In the formula, n represents the number of times historical pollutant concentration data are collected;

[0063] x i is the i-th observation value;

[0064] μ is the mean of the parameters of historical pollutant concentration data;

[0065] σ is the standard deviation of the parameter of historical pollutant concentration data;

[0066] It is expressed as the cube of the standardized value of each observation relative to the mean and is used to measure the degree of skewness;

[0067] The peak expression is:

[0068]

[0069] In the formula, It is expressed as the fourth power of the normalized value of each observation relative to the mean and is used to measure the degree of kurtosis;

[0070] It is expressed as the kurtosis value of the adjusted normal distribution;

[0071] Extracting different simulation identification states from the historical pollutant concentration data set using skewness and kurtosis;

[0072] According to the different simulation identification states, the corresponding control limit h value is calculated using the mean and standard deviation of the historical pollutant concentration data set, and the expression formula is:

[0073] h = μ ± k·σ;

[0074] Where k is the multiple of the control limit h;

[0075] The control limit h values ​​calculated for the different simulation identification states are obtained to form an initial candidate set.

[0076] Accordingly, the present invention also proposes an urban environmental pollutant data evaluation system, comprising: an acquisition module; a preprocessing module; an analysis module; a detection module;

[0077] The acquisition module is used to collect raw data through multi-source sensors, perform time synchronization, and output an initial urban environmental pollutant data set with a unified frequency;

[0078] The preprocessing module is used to perform pollutant concentration abnormal value and standardization preprocessing on the initial urban environmental pollutant data set to obtain an urban environmental pollutant data set;

[0079] The analysis module is used to perform parameter initialization analysis on the urban environmental pollutant data set and output an initialization parameter set;

[0080] The detection module is used to perform pollutant concentration anomaly detection on the initialization parameter set using the MCUSUM method to obtain a pollutant concentration anomaly detection result; and generate a pollutant concentration anomaly detection report for the pollutant concentration anomaly detection result.

[0081] Compared with the prior art, the embodiments of the present invention have at least the following technical advantages:

[0082] It can be seen from the analysis of the above-mentioned urban environmental pollutant data evaluation method provided by the present invention that, in specific applications, the installed multi-source sensors are used to collect raw data at a unified frequency at the same time to obtain an initial urban environmental pollutant data set, which contains rich pollution concentration information; and the initial urban environmental pollutant data set is subjected to normalization processing of pollutant concentration abnormal values ​​using Z-score to obtain an urban environmental pollutant data set, which can smooth out abnormal indexes and noise existing in the initial urban environmental pollutant data set to obtain a final urban environmental pollutant data set with more analytical value; the parameters in the urban environmental pollutant data set are calculated to perform an initial analysis of the mean value of the parameters and the standard deviation of the parameters to determine whether the urban environmental pollutant data set is stable, thereby determining whether the time series characteristics are stable, thereby improving the accuracy and reliability of detection;

[0083] Furthermore, the MCUSUM method is used to monitor the observation parameters in the urban environmental pollutant data set, and then the PCA method is used to extract key features. By extracting key features through the PCA method, the data structure can be simplified and the dimension can be reduced to facilitate subsequent analysis and visualization; and these key features are evaluated according to the importance scores using expert scores, and pollutant features with high importance scores are screened out. It can be determined which features may have higher pollution concentrations and the observed values ​​of the pollutant features can be determined to improve monitoring efficiency; the urban environmental pollutant data set is standardized, and eigenvalues ​​and eigenvectors are extracted. The larger the eigenvalue, the more data variability the corresponding principal component can explain. Further analysis of the extracted principal components can reveal the potential relationship between different pollutants and identify common sources or mechanisms of mutual influence; setting the preliminary control limit h value and constructing a multivariate control chart with the selected principal components can simultaneously monitor the concentration changes of multiple pollutants. By judging whether the observed value of the pollutant characteristic is greater than the preliminary control limit h value, it can be determined whether the set preliminary control limit h value can meet the setting of the preliminary control limit threshold. If not, the preliminary control limit h value is reset and updated until it meets the set preliminary control limit threshold. By dynamically adjusting the preliminary control limit h value, the system can adapt to environmental changes in different time periods and ensure the continuous effectiveness of monitoring;

[0084] Furthermore, the reference value of the pollutant concentration observation value is calculated to adjust the threshold value of the preliminary control limit h value, which can effectively adjust the control limit so that the preliminary control limit h value meets the simulation requirements when the average running length ARL is calculated later; the preliminary control limit h value is repeatedly adjusted through Monte Carlo simulation to simulate different environmental conditions to optimize the monitoring performance, and the average running length ARL at which the first warning appears is monitored to be close to the preset target average running length ARL0, so as to determine whether the adjusted control limit h value meets the expected performance index, ensuring that the alarm will not be issued too early under normal circumstances, and timely warning can be given when abnormal pollutant concentration occurs; at the same time, the initialization accumulation and variable of the CUSUM method is used to calculate the standardized deviation of the observed value. Through the accumulated deviation, the CUSUM method can adapt to the changes in different degrees of environmental pollution, and the CUSUM statistic can quickly reflect these fluctuations, so as to adjust the monitoring strategy in time; when the CUSUM statistic exceeds the control limit h value, an alarm is triggered;

[0085] Furthermore, historical pollutant concentration data is collected n times, and different simulation identification states are extracted for each historical pollutant concentration data, so that the conditions that trigger warnings and the pollutant concentration changes in the historical process can be understood, and the historical pollutant concentration control limit h values ​​that trigger these warning conditions in different simulation identification states are selected to form an initial candidate set; the initial candidate set is subjected to a normally distributed random number series, so that the response time from the start of monitoring to the occurrence of a warning can be better randomly recorded, and the average run length ARL can be calculated. At the same time, the historical pollutant concentration control limit h value corresponding to the average run length ARL is also recorded, so as to judge the historical pollutant concentration control limit The average running length ARL corresponding to the h value matches the preset target average running length ARL0. If the matching degree between the two is greater than the matching degree threshold, it means that the running lengths of the two are close to each other, and the corresponding historical pollutant concentration control limit h value is the control limit h value that matches the actual pollutant concentration data; if it does not meet the requirements, the corresponding historical pollutant concentration control limit h values ​​in the initial candidate set are traversed until the control limit h value that matches the actual pollutant concentration data is selected. The control limit h value that matches the actual pollutant concentration data can more effectively monitor the changes in pollutant concentration, reduce the false triggering of alarms, and detect abnormal changes in pollutant concentrations in a timely manner. BRIEF DESCRIPTION OF THE DRAWINGS

[0086] In order to more clearly illustrate the specific implementation methods of the present invention or the technical solutions in the prior art, the drawings required for use in the specific implementation methods or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some implementation methods of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0087] Figure 1 An overall flow chart of a method for evaluating urban environmental pollutant data provided by Embodiment 1 of the present invention;

[0088] Figure 2 A flow chart of abnormal pollutant concentration detection results of a method for evaluating urban environmental pollutant data provided in Example 1 of the present invention;

[0089] Figure 3 A flowchart of determining observation values ​​and preliminary control limit h values ​​for a method for evaluating urban environmental pollutant data provided in Example 1 of the present invention;

[0090] Figure 4 A flow chart of obtaining a control limit h value for a method for evaluating urban environmental pollutant data provided in Example 1 of the present invention;

[0091] Figure 5 A flowchart of forming an initial candidate set for a method for evaluating urban environmental pollutant data provided in Example 1 of the present invention;

[0092] Figure 6 A flow chart of a system for evaluating urban environmental pollutant data provided by Embodiment 2 of the present invention;

[0093] Label: acquisition module 10; preprocessing module 20; analysis module 30; detection module 40. DETAILED DESCRIPTION

[0094] The technical solution of the present invention will be described clearly and completely below in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0095] The present invention is further described in detail below through specific embodiments in conjunction with the accompanying drawings.

[0096] Embodiment 1

[0097] like Figure 1 As shown, the present invention proposes a method for evaluating urban environmental pollutant data, comprising the following steps:

[0098] S1: Collect raw data through multi-source sensors, synchronize time, and output an initial urban environmental pollutant dataset with a unified frequency;

[0099] It should be noted that, through multiple environmental sensors (such as air quality monitors, flow monitors, gas concentration monitors, etc.), multi-dimensional raw data are collected (raw data may include environmental data (such as temperature, humidity, air pressure, etc.), pollutant concentration data (such as PM2.5, NO2, CO2 and other gas concentrations, flow data (such as pollutant gas flow rate, flow, etc.), these sensors may be distributed in different geographical locations or system modules, so there may be time deviations in the collection of raw data; therefore, when multiple environmental sensors collect raw data, the time synchronization of multiple environmental sensors is first performed to ensure that all sensor data are collected under a unified time reference and a unified frequency is achieved; through the synchronization of sensor time, the urban environmental pollutant data set finally generated not only contains rich pollutant concentration information, but also has a consistent timestamp, which lays a foundation for subsequent analysis;

[0100] S2: performing pollutant concentration abnormal value and standardization preprocessing on the initial urban environmental pollutant data set to obtain an urban environmental pollutant data set;

[0101] It should be noted that the initial urban environmental pollutant data set is processed for pollutant concentration outliers; by calculating the modified Z-score (i.e., the Z-score is used to measure the degree of deviation of a data point (i.e., a data point refers to a single observation collected in a time series; each data point usually contains one or more features (variables) to describe the characteristics of the observation; for example, in meteorological data, a data point may include date, temperature, humidity, and wind speed) from the central position (median), the modified Z-score (using MAD) is more robust, especially when there are extreme values ​​in the data; the Z-score is a standardized score used to measure the degree to which a data point deviates from the mean):

[0102]

[0103] Where MAD is the median absolute deviation (i.e., MAD is a robust measure of dispersion that is not easily affected by extreme values);

[0104] The calculation formula of MAD is:

[0105] MAD=median(|x i -median(X)|);

[0106] In the formula, MAD reduces the impact of extreme values ​​by calculating the median of the absolute deviation of each data point from the median;

[0107] If |Mi|>3.5, it is regarded as an outlier in pollutant concentration and smoothed using the LOESS method (i.e., LOESS: Local Weighted Regression, suitable for nonlinear data; by performing weighted fitting in the neighborhood around each point, outliers in pollutant concentration can be effectively smoothed):

[0108] y i =LOESS(x i , window_size=20, degree=2);

[0109] Where window_size represents the size of the window used to smooth the data value range (i.e., the larger the value, the more obvious the smoothing effect), and degree represents the degree of the polynomial used for local fitting (i.e., 2 is usually selected to represent a quadratic polynomial);

[0110] If |Mi|<=3.5, keep the original value: yi=xi;

[0111] After processing, a complete urban environmental pollutant dataset without abnormal values ​​and missing values ​​of pollutant concentration is output;

[0112] The skewness is calculated for each data point xi in the urban environmental pollutant data set (i.e., skewness measures the degree of asymmetry of the data distribution). The skewness formula is:

[0113]

[0114] Skewness measures the symmetry of the data distribution; a skewness less than 0 indicates left skewness, and a skewness greater than 0 indicates right skewness;

[0115] If |skewness|<0.5, the data distribution is relatively symmetrical and Z-score standardization is used:

[0116]

[0117] Adjust the data mean to 0 and the standard deviation to 1;

[0118] If |skewness|≥0.5, the data distribution is relatively skewed, and logarithmic transformation Z-score standardization is first performed (that is, logarithmic transformation is used to deal with skewed distribution):

[0119]

[0120] Logarithmic transformation can reduce the degree of skewness and make the data closer to a normal distribution;

[0121] If the urban environmental pollutant data set has clear upper and lower bounds [a, b], use Min-Max normalization:

[0122]

[0123] Scale the data to the interval [0,1] so that they can be compared on the same scale;

[0124] Output the final standardized urban environmental pollutant data set U = {u1, u2, ..., un};

[0125] Explanation: After the initial urban environmental pollutant data set is collected, the initial urban environmental pollutant data set may be affected by noise or pollutant concentration abnormal values. In order to improve the quality of the initial urban environmental pollutant data set, the original data is processed for pollutant concentration abnormal values ​​by using methods such as sliding average or exponential smoothing in the smoothing algorithm; by processing the pollutant concentration abnormal values, not only outliers can be removed, but also random fluctuations in the original data can be reduced; at the same time, standardization processing (such as Z-score standardization) is applied to the data of different sensors to make comparisons on the same dimension, ensuring that the statistical characteristics of the source data collected by different sensors can be effectively integrated in the subsequent analysis, thereby enhancing the stability of identifying pollutant concentration abnormalities;

[0126] By processing pollutant concentration outliers and using an adaptable standardization method, the initial urban environmental pollutant dataset is ensured to be of high quality and suitable for further analysis. This method combines robustness (through MAD) and flexibility (selecting different standardization methods based on skewness), making the final urban environmental pollutant dataset more valuable for analysis.

[0127] S3: performing parameter initialization analysis on the urban environmental pollutant data set, and outputting an initialization parameter set;

[0128] It should be noted that when the urban environmental pollutant data set is initialized and analyzed, the initialization window w is first set (i.e., the initialization window allows the calculation of the mean and standard deviation within the characteristic time range, and at the same time identifies the change trend of the data in the time series), and the size of the initialization window w is 100 (i.e., w=100, the window is the data range for determining the initialization parameters);

[0129] Calculate the current parameter mean μw and the current parameter standard deviation σw in the urban environmental pollutant data set within the initialization window w;

[0130] The parameter mean μw is:

[0131] μ w =mean(x(t-w+1):t);

[0132] Where t is the current time point (i.e., the index of the latest time point in the urban environmental pollutant dataset); t-w+1 represents the index starting from the current time point back to w time points (i.e., the range from t-w+1 to t includes the most recent w time points; for example, if t=10 and w=5, then t-w+1=6, at this time, the range 6 to 10 means from the 6th data point to the 10th data point (a total of 5 data points, a data point refers to a single observation collected in a time series), x(t) is the observation value at the current time point, x(t-1), x(t-2),…, x(t-w+1) are the observation values ​​at the previous time points; by continuously sliding the window, the dynamic changes of the time series can be captured, parameters can be calculated in different initialization windows, and potential risk factors of the data in the short or long term can be analyzed);

[0133] The standard deviation σw of the parameter is:

[0134] σ w =std(x(t-w+1):t);

[0135] The current parameter mean μw and the current parameter standard deviation σw are used to calculate the parameter change rate in the urban environmental pollutant data set within the initialization window w:

[0136] The relative change rate between the current parameter mean μw and the parameter mean of the previous window is rμ (i.e., the relative amplitude of the mean change is measured. If the rμ value is very small, it means that the mean of the current window does not change much from the mean of the previous window):

[0137]

[0138] The relative rate of change between the current parameter standard deviation σw and the parameter standard deviation of the previous window is rσ (i.e., the relative magnitude of the change in the standard deviation; similarly, if the rσ value is very small, it means that the volatility of the data has not changed significantly):

[0139]

[0140] Preset the ε threshold, and compare the maximum value of the relative change rate of the parameter mean rμ and the relative change rate of the parameter standard deviation rσ with the preset ε threshold;

[0141] If max(rμ, rσ)<ε, and w≥100, the current parameter mean μw and the current parameter standard deviation σw remain stable within the initialization window, and the current initialization window w is used to initialize the parameters of the urban environmental pollutant data set (that is, by presetting the ε threshold, ε=0.001, it is used to determine whether the changes in the mean and standard deviation are within an acceptable range. If max(rμ, rσ)<ε, it can be considered that the data fluctuation is within a controllable range and the data is stable; at the same time, w≥100, a larger initialization window (at least 100) is selected to ensure that the calculated mean and standard deviation have higher reliability. A larger initialization window can more accurately reflect the overall characteristics of the data and reduce the impact of accidental fluctuations on statistical results);

[0142] Then set: μ0=μw; σ0=σw;

[0143] If it is unstable, increase the initialization window until the current parameter mean μw and the current parameter standard deviation σw remain stable within the initialization window (i.e., to deal with the unstable situation, if w increases to 1000 in the initialization window, issue a warning, and use these 1000 values ​​to initialize the parameters, store the calculated mean, standard deviation and window size in a set, and finally output the initialization parameter set), and obtain the initialization parameter set = {μ0, σ0, w} (i.e., the initialization parameters are calculated within a specific initialization window to obtain a set of statistics, including the current mean, standard deviation and sliding initialization window);

[0144] Explanation: Based on the pre-processed urban environmental pollutant data set, the parameter initialization stage is entered; the initial parameters of the urban environmental pollutant data set are determined using statistical methods (such as maximum likelihood estimation, Bayesian inference, etc.) to reflect the core characteristics of the urban environmental pollutant data set; at the same time, the urban environmental pollutant data set is subjected to stability analysis to check whether the time series characteristics of the data are stable (such as the consistency of the mean and variance); this analysis can be achieved through methods such as unit root test or variance homogeneity test to ensure that subsequent pollutant concentration anomaly detection can be carried out in a stable background, thereby improving the accuracy and reliability of detection;

[0145] S4: Use the MCUSUM method to perform pollutant concentration anomaly detection on the initialization parameter set (i.e., monitor the deviation of the mean and standard deviation calculated in the initialization parameter set from the preset control limits through the monitoring technology of the MCUSUM method, and accumulate the counts; calculate the control limits through the mean and standard deviation, and calculate the deviation from the control limits for the observed values ​​in each initialization window, thereby achieving the result of pollutant concentration anomaly detection), and obtain the pollutant concentration anomaly detection result; generate a pollutant concentration anomaly detection report for the pollutant concentration anomaly detection result.

[0146] It should be noted that the MCUSUM (Cumulative Sum Control Chart) method is used to dynamically monitor time series data; MCUSUM is a powerful statistical process control tool that identifies potential abnormal changes in pollutant concentrations in the initialization parameter set by calculating the cumulative sum, so that the changing trend of the data can be tracked in real time and any deviation from the normal range can be responded to quickly; the system sets an appropriate warning threshold (i.e., control limit), and when the cumulative sum exceeds the preset threshold, it triggers an abnormal pollutant concentration alarm. The ability of real-time monitoring makes the system more sensitive and responsive in data analysis;

[0147] Finally, the system conducts an in-depth explanation and discussion of the abnormal pollutant concentration detection results analyzed by MCUSUM. This stage is not only about generating reports, but also about visually displaying pollution data through data visualization and graphical tools (such as control charts and trend charts). During the analysis process, the possible root causes of abnormal pollutant concentrations, their impact on the system, and their potential business significance are explored. At the same time, based on the analysis results, detailed decision support reports are generated to provide effective insights and suggestions to decision makers, helping them to respond quickly and accurately when faced with abnormal pollutant concentrations.

[0148] Specifically, Figure 2 As shown, in step S4, the MCUSUM method is used to perform pollutant concentration abnormality detection on the initialization parameter set (assuming that the initialization parameter set determined by the initialization window size w' is further screened and detected) to obtain the pollutant concentration abnormality detection result. The specific operation steps are as follows:

[0149] S41: using the MCUSUM method to determine the observed values ​​of pollutant concentration and the preliminary control limit h value for the observed parameters in the urban environmental pollutant data set;

[0150] The observation parameters are pollution concentration data points collected in real time (i.e., these data points reflect the concentration of environmental pollutants at a specific time point, mainly including PM2.5, NO2, SO2, etc.);

[0151] It should be noted that the observation value is the pollution concentration observed at the time point t of the pollution concentration data point collected in the real-time data stream, such as PM2.5, NO2, SO2, etc., depending on the specific monitoring target and specific environmental pollutants; the control limit is the threshold used to determine whether to trigger an alarm, which is based on CUSUM (i.e., CUSUM is a method of monitoring the process, which detects changes in pollutant concentration or abnormal pollutant concentration by accumulating deviations from observed values ​​and reference values; MCUSUM is an improvement on the traditional CUSUM method, which is usually more flexible in calculation); the control limit is usually calculated by methods such as Monte Carlo simulation of historical pollutant concentration data, with the aim of determining the boundaries of pollutant concentration abnormal deviations; when the CUSUM statistic exceeds the control limit, the system will consider that the pollutant concentration is abnormal, thereby triggering an alarm;

[0152] S42: Calculate the reference value of the pollutant concentration observation value;

[0153] The formula for the reference value is:

[0154] Where δ is the minimum deviation of the expected detection data (i.e., the change in pollutant concentration that you hope to detect, used to represent the minimum change or deviation that you hope to identify during the detection process);

[0155] σ0 is the standard deviation of the initialization parameter set (reflecting the natural fluctuations in pollutant concentrations in the urban environment, calculated based on historical pollutant concentration data);

[0156] k represents the allowable offset (i.e., used to adjust the sensitivity of alarm triggering; a smaller k can increase the sensitivity to changes in pollutant concentrations, but may lead to an increase in false alarms; a larger k may miss some real abnormal changes in pollutant concentrations);

[0157] It should be noted that the observation value xt is the pollutant concentration data point collected in real time. The reference value (i.e., the reference value is used to calculate the mean μ0 and standard deviation σ0 of the CUSUM statistic; it can be obtained based on historical pollutant concentration data or previously set standards, indicating the expected level of observations under normal circumstances) is used to evaluate the benchmark of these observations. Then the CUSUM statistic monitors whether there is a deviation by comparing the observation value with the reference value; for example, the difference between each observation value and the reference value (e.g., xt-μ0) is used to update the CUSUM statistic to determine whether the current data is within the normal range; if the observation value deviates significantly from the reference value, it may indicate that the pollutant concentration in the urban environment is abnormal, thereby triggering an alarm mechanism.

[0158] S43: preset target average run length ARL0; repeatedly adjust the preliminary control limit h value through Monte Carlo simulation, monitor the average run length ARL of the first warning to be close to the preset target average run length ARL0, and finally output the control limit h value that matches the actual pollutant concentration data;

[0159] It should be noted that ARL0 refers to the average length of system operation (i.e. time or number of samples, such as the length of time from time t to time t2 (i.e. the average length of system operation, the time period during which the pollutant concentration does not exceed the standard limit)) under normal conditions, during which no alarm will be issued; Monte Carlo simulation: This is a statistical method. The Monte Carlo simulation method simulates different environmental conditions by randomly sampling pollutant concentration data to generate a possible sequence of observations; based on the simulation results, the value of the control limit h is repeatedly adjusted to optimize the monitoring performance; specifically, the control limit h value can be repeatedly adjusted to ensure that in real data, the system's monitoring performance can be close to the preset target average operation length ARL0;

[0160] ARL is the average length from the start of system operation to the first alarm during the actual monitoring process. By adjusting the control limit h value, it is hoped that the adjusted control limit h value will be close to the preset target average operation length ARL0 (that is, the average operation length ARL from the start of system operation to the first alarm is close to the preset target average operation length ARL0);

[0161] After multiple simulations and adjustments, a control limit h value is finally selected so that in real applications, this control limit can effectively monitor the system status and keep the average run length ARL close to the preset target average run length ARL0 (that is, in normal or actual conditions, the frequency of alarms is consistent with expectations);

[0162] Through repeated simulation and adjustment of control limits, the expected performance indicators are achieved, ensuring that alarms are not issued prematurely under normal circumstances and that timely warnings can be issued when abnormal pollutant concentrations occur;

[0163] S44: using the initialized cumulative sum variable of the CUSUM method to perform standardized deviation calculation on the observed values, and obtain a CUSUM statistic for detecting the deviation of the pollutant concentration (i.e., the CUSUM statistic is formed into a cumulative sum sequence by accumulating the deviation between each observed value and a target value (usually the reference value μ0 of the pollutant concentration));

[0164] It should be noted that before starting monitoring, two CUSUM statistics are initialized:

[0165] S+: statistic used to detect positive deviations in pollutant concentrations;

[0166] S-: statistic used to detect negative deviations in pollutant concentrations;

[0167] Usually, the two CUSUM statistics are set to 0 at the beginning, that is, S+ = 0 and S- = 0;

[0168] The CUSUM statistic detects the deviation between the observed value and the reference value. By accumulating these deviations, CUSUM can effectively identify potential abnormalities in pollutant concentrations;

[0169] Therefore, for each pollutant concentration observation xt, calculate its deviation dt from the reference value μ0, the calculation formula is:

[0170] d t =x t -μ0;

[0171] Where, the deviation dt represents the difference between the observed value xt and the reference value μ0;

[0172] The deviation dt is standardized and the standardized deviation can be expressed as:

[0173]

[0174] Where σ0 is the standard deviation calculated based on historical pollutant concentration data to ensure that the calculated deviations are comparable;

[0175] Update the CUSUM statistic based on the standardized deviation zt;

[0176] If the standardized deviation zt is positive (i.e. the observed value is higher than the reference value, the pollutant concentration is higher than the reference value), the positive CUSUM statistic is updated, and the expression is:

[0177] S + =max(0, S + +z t );

[0178] If the standardized deviation zt is negative (i.e. the observed value is lower than the reference value and the pollutant concentration is lower than the reference value), the negative CUSUM statistic is updated, and the expression is:

[0179] S - =max(0, S - -z t );

[0180] If the standardized deviation zt is large, the CUSUM statistic will increase (or decrease) rapidly, while if the deviation is small, it will decrease (i.e., the CUSUM statistic of the detection deviation), thus quickly reflecting changes in pollutant concentrations;

[0181] Explanation: The CUSUM statistic is dynamically updated based on each observation, so that the monitoring process can reflect the current state in real time, rather than relying solely on the historical mean. This dynamic characteristic can help optimize process control; if the deviation is large, the CUSUM statistic increases (or decreases) rapidly, which can reflect the drastic changes in the detection process, allowing the monitoring system to more quickly identify abnormal trends in environmental pollution; by accumulating deviations, the CUSUM method can adapt to changes in environmental pollution to varying degrees; for example, if the degree of environmental pollution becomes more volatile, the CUSUM statistic can quickly reflect these fluctuations, thereby adjusting the monitoring strategy in a timely manner (i.e., updating the CUSUM statistic after adjusting the monitoring strategy);

[0182] S45: When the CUSUM statistic exceeds the control limit h value (i.e., the CUSUM statistic is greater than the control limit h value), an alarm is triggered, the pollutant concentration abnormality is recorded, and a pollutant concentration abnormality detection result is obtained;

[0183] It should be noted that, as explained in step S44, the CUSUM statistic consists of two parts: positive CUSUM (S+) and negative CUSUM (S-); these two statistics are dynamically updated according to each observation value and the set reference value;

[0184] When either the positive CUSUM or negative CUSUM statistic exceeds the control limit h, the system will consider the pollutant concentration to be abnormal and trigger an alarm. This alarm mechanism can identify abnormal changes in pollutant concentrations in a timely manner, thereby helping the environmental monitoring system to detect and take necessary pollution control measures early;

[0185] Specifically, Figure 3 As shown, in step S41, the MCUSUM method is used to determine the pollutant concentration observation value and the preliminary control limit h value of the observation parameters in the urban environmental pollutant data set. The specific operation steps are as follows:

[0186] S411: using the MCUSUM method to monitor the observation parameters in the urban environmental pollutant data set, and then using the PCA method to extract key features;

[0187] Using expert scoring to evaluate the importance score of the key features to obtain an importance score;

[0188] The pollutant features are screened out according to the importance scores of the key features, and the pollutant concentration observation values ​​of the pollutant features are determined (i.e., the observation value is the pollution concentration observed at the time point t of the pollution concentration data point collected in the real-time data stream, such as PM2.5, NO2, SO2, etc., depending on the specific monitoring target and specific environmental pollutants (i.e., pollutant features));

[0189] It should be noted that principal component analysis (PCA) is a commonly used dimensionality reduction technique that aims to extract key features from data to simplify the data structure; the extraction of key features is the process of converting the urban environmental pollutant data set into a new feature set that is smaller but still retains important information, which can simplify the data structure and reduce the dimension for subsequent analysis and visualization;

[0190] Calculate the correlation between each key feature, use statistical methods (such as Pearson correlation coefficient) to screen out the features that are closely related to the change of pollutant concentration; and use expert scoring to evaluate the scores of these key features according to their importance; continuously evaluate and sort the features through a recursive process, remove the features that have little impact on the change of pollutant concentration, retain the most relevant pollutant features, and continuously adjust and optimize the feature set based on the monitoring results and feedback information to improve monitoring efficiency;

[0191] S412: standardizing the urban environmental pollutant data set, calculating a covariance matrix, performing eigenvalue decomposition on the covariance matrix, and extracting eigenvalues ​​and eigenvectors;

[0192] selecting a principal component according to the eigenvalue;

[0193] It should be noted that eigenvalue decomposition is the process of extracting eigenvalues ​​and eigenvectors from the covariance matrix; eigenvalues ​​(Eigenvalues) represent the amount of variance explained by the principal component, eigenvectors (Eigenvectors) represent the direction of the principal component, and each eigenvalue corresponds to an eigenvector;

[0194] The principal components are usually selected based on the size of the eigenvalue. The larger the eigenvalue, the more data variability the corresponding principal component can explain. Arrange the eigenvalues ​​in descending order and select the first few principal components with larger eigenvalues. These principal components will represent most of the data variability. By analyzing the extracted principal components, we can find the potential relationship between different pollutants and identify the common source or mutual influence mechanism. For example, assume a data set with multiple pollutant concentrations (such as PM2.5, NO2, SO2, etc.): Through PCA, calculate the covariance matrix and perform eigenvalue decomposition, and the eigenvalues ​​obtained are: [4.5, 2.0, 0.8, 0.5]; the principal component represented by the eigenvalue of 4.5 can explain 45% of the variability in the data, the principal component represented by the eigenvalue of 2.0 can explain 20% of the variability, and so on; select the principal components corresponding to the first two eigenvalues ​​(4.5 and 2.0). These two principal components will represent most (65%) of the data variability, thereby achieving dimensionality reduction.

[0195] S413: setting a preliminary control limit h value, and constructing a multivariate control chart using the preliminary control limit h value and the selected principal component;

[0196] Monitor the relationship between the observed value of the pollutant characteristic and the preliminary control limit h value according to the multivariate control chart; determine whether the observed value of the pollutant characteristic is greater than the preliminary control limit h value;

[0197] If so, the pollution concentration of the pollutant characteristic exceeds the preliminary control limit h value, and the preliminary control limit h value is used as the preliminary control limit threshold value (that is, the control limit h value is the threshold value for judging whether the pollution concentration exceeds the standard, so when the preliminary control limit h value meets the threshold requirement, the preliminary control limit h value is determined to be the preliminary control limit threshold value, that is, the preliminary control limit h value is obtained accordingly);

[0198] If not, reset the initial control limit h value to judge with the observed value of the pollutant characteristic until the observed value of the pollutant characteristic is greater than the initial control limit h value, and use the initial control limit h value as the initial control limit threshold;

[0199] It should be noted that by setting and monitoring the preliminary control limit h value and combining it with the multivariate control chart, dynamic monitoring and management of pollutant characteristics can be achieved; by constructing a multivariate control chart, the concentration changes of multiple pollutants (i.e., the observed values ​​of pollutant characteristics) can be monitored simultaneously, the relationship between them can be identified, and a more comprehensive environmental quality analysis can be provided; the key features extracted using principal component analysis can help simplify the data structure and enhance the effectiveness and accuracy of monitoring;

[0200] By dynamically adjusting the initial control limit h value, the system can adapt to environmental changes in different time periods; for example, seasonal changes, weather conditions and other factors may affect pollutant concentrations, and flexible control limits can better reflect actual conditions; when the initial control limit h value does not meet the requirements, the control limit is reset to ensure the continued effectiveness of monitoring, thereby achieving the best initial control limit h value;

[0201] Specifically, Figure 4 As shown, in step S43, the target average run length ARL0 is preset; the preliminary control limit h value is repeatedly adjusted through Monte Carlo simulation, and the average run length ARL of the first warning is monitored to be close to the preset target average run length ARL0, and finally the control limit h value matching the actual pollutant concentration data is output. The specific operation steps are as follows:

[0202] S431: collecting n historical pollutant concentration data (e.g., collecting original data of historical pollutant concentration), extracting different simulation identification states for each historical pollutant concentration data; selecting multiple historical pollutant concentration control limit h values ​​according to the different simulation identification states to form an initial candidate set;

[0203] It should be noted that historical pollutant concentration data are collected, and different simulation identification states are extracted from the historical pollutant concentration data. Different simulation identification states are represented as the conditions for triggering warnings and the pollutant concentration change process in the historical process. By selecting the historical pollutant concentration control limit h value that triggers these warning conditions in different simulation identification states (that is, the historical pollutant concentration control limit h value is the threshold value that actually triggers the warning in the historical pollutant concentration data, and the pollutant concentration threshold value that actually triggers the alarm in the historical data is used as the test data for the Monte Carlo simulation) as the test data for the Monte Carlo simulation, it can be judged whether the control limit h value in the test is close to the historical pollutant concentration control limit h value, so that the average running length ARL of the first monitoring warning (that is, the time when the monitoring alarm first appears can be evaluated, and then the reaction speed of the system to the change of pollutant concentration can be judged; the earlier the alarm is triggered, the more sensitive the system is, and the potential pollutant concentration problem can be identified in time) is close to the preset target average running length ARL0;

[0204] S432: The initial candidate set is subjected to a random number sequence of normal distribution N(μ0,σ), and the random number sequence is monitored and collected through the CUSUM algorithm to trigger the warning of the average running length ARL (that is, because the initial candidate set is composed of the historical pollutant concentration control limit h value, in the random number sequence of normal distribution, the system will monitor the warning triggered by the running length according to the threshold value of the historical pollutant concentration control limit h value, so that it can match the preset target average running length ARL0 to see if it is close, so as to adjust the historical pollutant concentration control limit h value); the average running length ARL is calculated by simulation n times according to the random number sequence, and the historical pollutant concentration control limit h value of each simulation calculation of the average running length ARL is recorded;

[0205] It should be noted that the initial candidate set is subjected to a random sequence of normal distribution N(μ0,σ) (i.e., a random sequence can follow a specific probability distribution, such as normal distribution, uniform distribution, Poisson distribution, etc.; for example, a normal distributed random sequence will cluster near the mean, with most of the data concentrated in this area, while the data far from the mean is relatively small, so that the response time from the start of monitoring to the appearance of an alarm can be better recorded randomly), and the specific parameters of the random sequence are the mean (μ0) and the standard deviation (σ); most of the data will be concentrated near the mean, and as the distance from the mean increases, the probability of data appearance will gradually decrease, which means that even a small deviation in pollutant concentration can be captured in time; during the monitoring process, the number of time steps required from the start of data collection to the first warning issued by the CUSUM algorithm (i.e., a significant deviation is detected), so that the response time from the start of monitoring to the triggering of the alarm (i.e., the average run length ARL) can be evaluated, and the deviation between the actual preset target average run length ARL0 in the subsequent simulation can be judged, thereby adjusting the control limit h value of pollutant concentration monitoring;

[0206] S433: sort all recorded historical pollutant concentration control limit h values ​​in descending order; determine the matching degree between the average run length ARL corresponding to the historical pollutant concentration control limit h value and the preset target average run length ARL0 in descending order (i.e., determine whether the average run length ARL is close to the preset target average run length ARL0; if the average run length ARL corresponding to the historical pollutant concentration control limit h value matches the preset target average run length ARL0 successfully, it is determined that the two are close to each other, and the historical pollutant concentration control limit h value is used as the control limit h value output result of the real data), and the preset matching degree e is set;

[0207] Determine whether the matching degree between the average run length ARL corresponding to the historical pollutant concentration control limit h and the preset target average run length ARL0 is greater than the preset matching degree e;

[0208] If so, the average run length ARL successfully matches the preset target average run length ARL0, and the two are close to each other. The historical pollutant concentration control limit h value adjusted by the average run length ARL is determined to be the real data threshold, and the control limit h value that matches the real pollutant concentration data is output;

[0209] If not, then determine the matching degree between the average running length ARL corresponding to the historical pollutant concentration control limit h value and the preset target average running length ARL0 one by one in descending order until the historical pollutant concentration control limit h value with the average running length ARL greater than the preset matching degree e is found, update the historical pollutant concentration control limit h value, and use the historical pollutant concentration control limit h value as the control limit h value that matches the real pollutant concentration data (that is, because n historical pollutant concentration data have been collected in step S431 to form an initial candidate set, which already includes various different historical pollutant concentration control limit h values, there is no need to adjust the historical pollutant concentration control limit h value, and directly continue to determine in descending order whether the matching degree between the average running length ARL and the preset target average running length ARL0 is greater than the preset matching degree e. If greater, directly use the historical pollutant concentration control limit h value of the average running length ARL as the threshold of the real data (that is, the control limit h value that matches the real pollutant concentration data));

[0210] It should be noted that all average run lengths ARL calculated by the CUSUM algorithm are sorted in descending order, and it is determined one by one whether the matching degree between the average run length ARL and the preset target average run length ARL0 is greater than the preset matching degree e; if the matching degree between the two is equal to or less than the preset matching degree e, it is determined one by one in descending order whether the matching degree between the average run length ARL and the preset target average run length ARL0 is greater than the preset matching degree e, until a historical pollutant concentration control limit h value close to each other is found; if the matching degree between the two is greater than the preset matching degree e, it means that the average run length ARL and the preset target average run length ARL0 are close to each other, and under the setting of the historical pollutant concentration control limit h value of the average run length ARL, the system can more effectively monitor the change of pollutant concentration, reduce the false triggering of alarms, and detect abnormal changes in pollutant concentration in time;

[0211] Specifically, Figure 5 As shown, in step S431, historical pollutant concentration data is collected n times, and different simulation identification states are extracted for each historical pollutant concentration data; multiple historical pollutant concentration control limit h values ​​are selected according to the different simulation identification states to form an initial candidate set. The specific operation steps are as follows:

[0212] S4311: Determine the collection time range, collect historical pollutant concentration data n times, and obtain a historical pollutant concentration data set;

[0213] It should be noted that in this step, the monitoring time range is first determined, and according to the time range, n historical urban air pollutant concentration data are collected, and these data form a historical pollutant concentration data set;

[0214] S4312: Calculate the skewness and kurtosis using the mean and standard deviation (i.e., the mean and standard deviation have been calculated in the above step S3, and the mean and standard deviation of the historical pollutant concentration data can be obtained using the same calculation formula method, so no redundant calculation is performed);

[0215] The skew expression is:

[0216]

[0217] In the formula, n represents the number of times historical pollutant concentration data are collected (i.e., the number of historical pollutant concentration data collected, i.e., the number of samples);

[0218] x i is the i-th observation value;

[0219] μ is the mean of the parameters of historical pollutant concentration data;

[0220] σ is the standard deviation of the parameter of historical pollutant concentration data;

[0221] It is expressed as the cube of the standardized value of each observation relative to the mean and is used to measure the degree of skewness;

[0222] The peak expression is:

[0223]

[0224] In the formula, It is expressed as the fourth power of the normalized value of each observation relative to the mean and is used to measure the degree of kurtosis;

[0225] It is expressed as the kurtosis value of the adjusted normal distribution;

[0226] It should be noted that in the monitoring system, the mean and standard deviation as basic statistics can provide the central tendency and dispersion degree of the data, while the skewness and kurtosis further describe the form of data distribution; using the mean and standard deviation to calculate the skewness and kurtosis can form a more comprehensive monitoring and analysis framework;

[0227] Skewness is used to measure the symmetry of data; by monitoring skewness, deviations in data distribution can be discovered in a timely manner; for example, in the production process, positive skewness may indicate an increase in product defects, so measures can be taken in advance for quality control; kurtosis is used to measure the sharpness of data, and the kurtosis of the normal distribution is adjusted to 0; kurtosis analysis can help identify risks, and extreme warnings may be generated in the case of high kurtosis, so timely management and supervision measures need to be taken at this time, and skewness and kurtosis can also be used to extract different simulation identification states in the historical pollutant concentration data set according to their differences (i.e., through the distribution deviation of data analysis by skewness, warnings due to deviation delays caused by time or data can be extracted; or warnings due to excessive data extracted based on the extreme peak of kurtosis);

[0228] S4313: extracting different simulation identification states from the historical pollutant concentration data set using skewness and kurtosis (i.e., for example, by analyzing the distribution deviation of the data through skewness, warnings due to deviation delays caused by time or data can be identified; or warnings due to excessively high data extracted based on the extreme peak value of the kurtosis);

[0229] It should be noted that skewness and kurtosis are used to extract different simulated identification states from historical pollutant concentration data, that is, based on these statistical features, potential pollution warning signals are extracted through changes in time series, and then alarms are triggered based on abnormal behavior of pollutant concentration data;

[0230] S4314: According to the different simulation identification states, the corresponding control limit h value is calculated using the mean and standard deviation of the historical pollutant concentration data set, and the expression formula is:

[0231] h = μ ± k·σ;

[0232] Wherein, k is a multiple of the control limit h (i.e., the k value has the same meaning as the reference value k in the above step S42, and both are deviation values ​​(i.e., threshold values) for adjusting the control limit h);

[0233] Obtaining control limit h values ​​calculated for the different simulation identification states to form an initial candidate set;

[0234] It should be noted that by calculating the control limit, the stability of the process can be monitored in a timely manner. If the pollutant concentration data point exceeds the control limit, an alarm will be triggered, indicating potential abnormal pollutant concentration or quality problems; the setting of the control limit allows the value of k to be flexibly adjusted (that is, by adjusting the value of k, the sensitivity of the alarm can be controlled; a larger k value will make the control limit wider, thereby reducing the possibility of false alarms, but may also lead to missed alarms; a smaller k value will make the control limit narrower, increasing the sensitivity to abnormal pollutant concentrations, but may lead to more false alarms), according to the volatility of historical pollutant concentration data and actual needs, to ensure the sensitivity and accuracy of the alarm system; the control limit provides an early warning mechanism that can detect pollution problems in a timely manner and take measures, thereby reducing environmental pollution risks and protecting public health and ecosystem safety;

[0235] Finally, through these steps, the control limit h values ​​calculated under different conditions will be aggregated into an initial candidate set and used as a reference for the actual pollutant concentration thresholds in the subsequent monitoring system. Through these monitoring thresholds, abnormal conditions of pollutant concentrations in the urban environment can be effectively identified and handled.

[0236] Embodiment 2

[0237] like Figure 6 As shown, accordingly, the present invention also proposes an urban environmental pollutant data evaluation system, including: an acquisition module 10; a pre-processing module 20; an analysis module 30; a detection module 40;

[0238] The acquisition module 10 is used to collect raw data through multi-source sensors, perform time synchronization, and output an initial urban environmental pollutant data set with a unified frequency;

[0239] The preprocessing module 20 is used to perform pollutant concentration abnormal value and standardization preprocessing on the initial urban environmental pollutant data set to obtain an urban environmental pollutant data set;

[0240] The analysis module 30 is used to perform parameter initialization analysis on the urban environmental pollutant data set and output an initialization parameter set;

[0241] The detection module 40 is used to perform pollutant concentration anomaly detection on the initialization parameter set using the MCUSUM method to obtain a pollutant concentration anomaly detection result; and generate a pollutant concentration anomaly detection report for the pollutant concentration anomaly detection result.

[0242] In summary, it can be seen from the urban environmental pollutant data evaluation method and system proposed in the example of the present invention that the original data are collected at a uniform frequency by installing multi-source sensors at the same time to obtain an initial urban environmental pollutant data set, which contains rich pollution concentration information; and the initial urban environmental pollutant data set is subjected to normalization processing of pollutant concentration abnormal values ​​by using Z-score to obtain an urban environmental pollutant data set, which can smooth out abnormal indexes and noise existing in the initial urban environmental pollutant data set to obtain a final urban environmental pollutant data set with more analytical value; the parameters in the urban environmental pollutant data set are calculated to perform an initial analysis of the mean value and the standard deviation of the parameters, to determine whether the urban environmental pollutant data set is stable, thereby determining whether the time series characteristics are stable, thereby improving the accuracy and reliability of detection;

[0243] Furthermore, the MCUSUM method is used to monitor the observation parameters in the urban environmental pollutant data set, and then the PCA method is used to extract key features. By extracting key features through the PCA method, the data structure can be simplified and the dimension can be reduced to facilitate subsequent analysis and visualization; and these key features are evaluated according to the importance scores using expert scores, and pollutant features with high importance scores are screened out. It can be determined which features may have higher pollution concentrations and the observed values ​​of the pollutant features can be determined to improve monitoring efficiency; the urban environmental pollutant data set is standardized, and eigenvalues ​​and eigenvectors are extracted. The larger the eigenvalue, the more data variability the corresponding principal component can explain. Further analysis of the extracted principal components can reveal the potential relationship between different pollutants and identify common sources or mechanisms of mutual influence; setting the preliminary control limit h value and constructing a multivariate control chart with the selected principal components can simultaneously monitor the concentration changes of multiple pollutants. By judging whether the observed value of the pollutant characteristic is greater than the preliminary control limit h value, it can be determined whether the set preliminary control limit h value can meet the setting of the preliminary control limit threshold. If not, the preliminary control limit h value is reset and updated until it meets the set preliminary control limit threshold. By dynamically adjusting the preliminary control limit h value, the system can adapt to environmental changes in different time periods and ensure the continuous effectiveness of monitoring;

[0244] Furthermore, the reference value of the pollutant concentration observation value is calculated to adjust the threshold value of the preliminary control limit h value, which can effectively adjust the control limit so that the preliminary control limit h value meets the simulation requirements when the average running length ARL is calculated later; the preliminary control limit h value is repeatedly adjusted through Monte Carlo simulation to simulate different environmental conditions to optimize the monitoring performance, and the average running length ARL at which the first warning appears is monitored to be close to the preset target average running length ARL0, so as to determine whether the adjusted control limit h value meets the expected performance index, ensuring that the alarm will not be issued too early under normal circumstances, and timely warning can be given when abnormal pollutant concentration occurs; at the same time, the initialization accumulation and variable of the CUSUM method is used to calculate the standardized deviation of the observed value. Through the accumulated deviation, the CUSUM method can adapt to the changes in different degrees of environmental pollution, and the CUSUM statistic can quickly reflect these fluctuations, so as to adjust the monitoring strategy in time; when the CUSUM statistic exceeds the control limit h value, an alarm is triggered;

[0245] Furthermore, historical pollutant concentration data is collected n times, and different simulation identification states are extracted for each historical pollutant concentration data, so that the conditions that trigger warnings and the pollutant concentration changes in the historical process can be understood, and the historical pollutant concentration control limit h values ​​that trigger these warning conditions in different simulation identification states are selected to form an initial candidate set; the initial candidate set is subjected to a normally distributed random number series, so that the response time from the start of monitoring to the occurrence of a warning can be better randomly recorded, and the average run length ARL can be calculated. At the same time, the historical pollutant concentration control limit h value corresponding to the average run length ARL is also recorded, so as to judge the historical pollutant concentration control limit The average running length ARL corresponding to the h value matches the preset target average running length ARL0. If the matching degree between the two is greater than the matching degree threshold, it means that the running lengths of the two are close to each other, and the corresponding historical pollutant concentration control limit h value is the control limit h value that matches the actual pollutant concentration data; if it does not meet the requirements, the corresponding historical pollutant concentration control limit h values ​​in the initial candidate set are traversed until the control limit h value that matches the actual pollutant concentration data is selected. The control limit h value that matches the actual pollutant concentration data can more effectively monitor the changes in pollutant concentration, reduce the false triggering of alarms, and detect abnormal changes in pollutant concentrations in a timely manner.

[0246] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. A person skilled in the art may modify the technical solutions described in the above embodiments, or replace part or all of the technical features therein with equivalents. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for evaluating urban environmental pollutant data, characterized in that: The steps are as follows: Collect raw data through multi-source sensors, synchronize time, and output an initial urban environmental pollutant data set with a unified frequency; Performing pollutant concentration abnormal value and standardization preprocessing on the initial urban environmental pollutant data set to obtain an urban environmental pollutant data set; Performing parameter initialization analysis on the urban environmental pollutant data set and outputting an initialization parameter set; Using the MCUSUM method to perform pollutant concentration anomaly detection on the initialization parameter set to obtain a pollutant concentration anomaly detection result; A pollutant concentration abnormality detection report is generated based on the pollutant concentration abnormality detection result.

2. According to claim 1, a method for evaluating urban environmental pollutant data is characterized in that: Perform parameter initialization analysis on the urban environmental pollutant data set and output an initialization parameter set. The specific operation steps are as follows: Set an initialization window w, the size of which is 100; Calculate the current parameter mean μw and the current parameter standard deviation σw in the urban environmental pollutant data set within the initialization window w; The parameter mean μw is: μ w =mean(x(t-w+1):t); Where t is the current time point; t-w+1 represents the index starting from the current time point and moving back w time points; The standard deviation σw of the parameter is: σ w =std(x(t-w+1):t); The current parameter mean μw and the current parameter standard deviation σw are used to calculate the parameter change rate in the urban environmental pollutant data set within the initialization window w: The relative change rate between the current parameter mean μw and the parameter mean of the previous window is rμ: The relative change rate between the current parameter standard deviation σw and the parameter standard deviation of the previous window is rσ: Preset the ε threshold, and compare the maximum value of the relative change rate of the parameter mean rμ and the relative change rate of the parameter standard deviation rσ with the preset ε threshold; If max(rμ, rσ)<ε, and w≥100, the current parameter mean μw and the current parameter standard deviation σw remain stable within the initialization window, and the current initialization window w is used to initialize the parameters of the urban environmental pollutant data set; Then set: μ0=μw; σ0=σw; If it is unstable, increase the initialization window until the current parameter mean μw and the current parameter standard deviation σw remain stable within the initialization window, and obtain the initialization parameter set = {μ0, σ0, w}.

3. The urban environmental pollutant data evaluation method according to claim 2, characterized in that: The MCUSUM method is used to perform pollutant concentration anomaly detection on the initialization parameter set to obtain pollutant concentration anomaly detection results. The specific operation steps are as follows: Using the MCUSUM method to determine the observed values ​​of pollutant concentrations and the preliminary control limit h value for the observed parameters in the urban environmental pollutant data set; Calculating a reference value for the observed pollutant concentration; The formula for the reference value is: In the formula, δ is the minimum offset of the expected detection data; σ0 is the standard deviation of the initialization parameter set; k represents the allowed offset; The preset target average run length ARL0; The preliminary control limit h value is repeatedly adjusted through Monte Carlo simulation, and the average run length ARL of the first warning is monitored to be close to the preset target average run length ARL0, and finally the control limit h value matching the actual pollutant concentration data is output; The initialization cumulative sum variable of the CUSUM method is used to calculate the standardized deviation of the observed values, and the CUSUM statistic for detecting the deviation of pollutant concentration is obtained; When the CUSUM statistic exceeds the control limit h value, an alarm is triggered, the abnormal pollutant concentration is recorded, and the abnormal pollutant concentration detection result is obtained.

4. The urban environmental pollutant data evaluation method according to claim 3 is characterized in that: The MCUSUM method is used to determine the pollutant concentration observation value and the preliminary control limit h value for the observation parameters in the urban environmental pollutant data set. The specific operation steps are as follows: The MCUSUM method is used to monitor the observation parameters in the urban environmental pollutant data set, and the PCA method is used to extract key features; Using expert scoring to evaluate the importance score of the key features to obtain an importance score; The pollutant features are screened out according to the importance scores of the key features, and the pollutant concentration observation values ​​of the pollutant features are determined; Standardizing the urban environmental pollutant data set, calculating the covariance matrix, performing eigenvalue decomposition on the covariance matrix, and extracting eigenvalues ​​and eigenvectors; selecting a principal component according to the eigenvalue; Set the preliminary control limit h value, and construct a multivariate control chart with the preliminary control limit h value and the selected principal component; Monitor the relationship between the observed value of the pollutant characteristic and the preliminary control limit h value according to the multivariate control chart; determine whether the observed value of the pollutant characteristic is greater than the preliminary control limit h value; If yes, the pollution concentration of the pollutant characteristic exceeds the preliminary control limit h value, and the preliminary control limit h value is used as the preliminary control limit threshold; If not, reset the preliminary control limit h value to judge with the observed value of the pollutant characteristic until the observed value of the pollutant characteristic is greater than the preliminary control limit h value, and use the preliminary control limit h value as the preliminary control limit threshold.

5. A method for evaluating urban environmental pollutant data according to claim 4, characterized in that: The observation parameters are pollution concentration data points collected in real time.

6. A method for evaluating urban environmental pollutant data according to claim 5, characterized in that: The target average run length ARL0 is preset; the preliminary control limit h value is repeatedly adjusted through Monte Carlo simulation, and the average run length ARL at which the first warning appears is monitored to be close to the preset target average run length ARL0, and finally the control limit h value matching the actual pollutant concentration data is output. The specific operation steps are as follows: Collecting n times of historical pollutant concentration data, extracting different simulation identification states for each historical pollutant concentration data; selecting multiple historical pollutant concentration control limit h values ​​according to the different simulation identification states to form an initial candidate set; The initial candidate set is subjected to a random number sequence of normal distribution N(μ0,σ), and the random number sequence is monitored and collected by the CUSUM algorithm to obtain the average run length ARL that triggers the warning; the average run length ARL is simulated and calculated n times according to the random number sequence, and the historical pollutant concentration control limit h value of the average run length ARL is recorded for each simulation calculation; Sort all recorded historical pollutant concentration control limit h values ​​in descending order; determine the matching degree between the average run length ARL corresponding to the historical pollutant concentration control limit h value and the preset target average run length ARL0 in descending order; Preset matching degree e; Determine whether the matching degree between the average run length ARL corresponding to the historical pollutant concentration control limit h and the preset target average run length ARL0 is greater than the preset matching degree e; If so, the average run length ARL successfully matches the preset target average run length ARL0, and the two are close to each other. The historical pollutant concentration control limit h value adjusted by the average run length ARL is determined to be the real data threshold, and the control limit h value that matches the real pollutant concentration data is output; If not, the matching degree between the average running length ARL corresponding to the historical pollutant concentration control limit h value and the preset target average running length ARL0 is determined one by one in descending order until the historical pollutant concentration control limit h value with an average running length ARL greater than the preset matching degree e is found, and the historical pollutant concentration control limit h value is updated, and the historical pollutant concentration control limit h value is used as the control limit h value that matches the actual pollutant concentration data.

7. A method for evaluating urban environmental pollutant data according to claim 6, characterized in that: Collect n times of historical pollutant concentration data, extract different simulation identification states for each historical pollutant concentration data; select multiple historical pollutant concentration control limit h values ​​according to the different simulation identification states to form an initial candidate set. The specific operation steps are as follows: Determine the collection time range, collect historical pollutant concentration data n times, and obtain a historical pollutant concentration data set; Calculating skewness and kurtosis using the mean and standard deviation; The skew expression is: In the formula, n represents the number of times historical pollutant concentration data are collected; x i is the i-th observation value; μ is the mean of the parameters of historical pollutant concentration data; σ is the standard deviation of the parameter of historical pollutant concentration data; It is expressed as the cube of the standardized value of each observation relative to the mean and is used to measure the degree of skewness; The peak expression is: In the formula, It is expressed as the fourth power of the normalized value of each observation relative to the mean and is used to measure the degree of kurtosis; It is expressed as the kurtosis value of the adjusted normal distribution; Extracting different simulation identification states from the historical pollutant concentration data set using skewness and kurtosis; According to the different simulation identification states, the corresponding control limit h value is calculated using the mean and standard deviation of the historical pollutant concentration data set, and the expression formula is: h = μ ± k·σ; Where k is the multiple of the control limit h; The control limit h values ​​calculated for the different simulation identification states are obtained to form an initial candidate set.

8. An urban environmental pollutant data evaluation system, characterized in that: include: Get module; Preprocessing module; Analysis module; Detection module; The acquisition module is used to collect raw data through multi-source sensors, perform time synchronization, and output an initial urban environmental pollutant data set with a unified frequency; The preprocessing module is used to perform pollutant concentration abnormal value and standardization preprocessing on the initial urban environmental pollutant data set to obtain an urban environmental pollutant data set; The analysis module is used to perform parameter initialization analysis on the urban environmental pollutant data set and output an initialization parameter set; The detection module is used to perform pollutant concentration anomaly detection on the initialization parameter set using the MCUSUM method to obtain a pollutant concentration anomaly detection result; and generate a pollutant concentration anomaly detection report for the pollutant concentration anomaly detection result.

Images