Environmental monitoring data abnormal value identification method, system and device and storage medium

Abstract

The invention belongs to the technical field of atmospheric environment monitoring quality control, and relates to an environment monitoring data abnormal value identification method, system and device and a storage medium, and the identification method comprises the steps: 1) data collection; 2) data preprocessing; 3) stationary processing: converting the real-time data from a non-stationary time sequence into a stationary time sequence; 4) constructing a sliding window: setting the width # imgabs0 # and the step length # imgabs1 # of the sliding window; 5) calculating statistical parameters in the sliding windows: for each sliding window, calculating a window mean value # imgabs3 # and a standard deviation # imgabs4 # of difference estimated values of # imgabs2 # real-time data contained in the sliding window; and 6) optimizing the Grobust step test: for each real-time data # imgabs5 # contained in the sliding window, calculating and standardizing the deviation between the real-time data # imgabs5 # and the window mean value # imgabs6 # to obtain a test statistic # imgabs7 #, and when the calculated T value exceeds 95% confidence interval, considering that the real-time data # imgabs8 # belongs to an abnormal value. The method effectively solves the problem of insufficient real-time performance and accuracy in the prior art.

Description

Technical Field

[0001] The invention belongs to the technical field of atmospheric environment monitoring quality control, and relates to a method for identifying outliers in environmental monitoring data, and in particular to a method for detecting outliers in environmental monitoring time series data by using an ARIMA-Grubbs test method based on a sliding window. Background Art

[0002] Environmental monitoring is an important means of ecological protection and pollution control. Its core lies in real-time collection of environmental parameters such as atmosphere and water bodies through sensors to support environmental quality assessment and management decisions. However, monitoring data is usually affected by many factors such as sensor equipment stability, external interference, and changes in climate conditions, which may lead to the emergence of outliers. If these outliers are not effectively processed, they will not only reduce the accuracy of monitoring results, but also mislead subsequent analysis and prediction models based on these data. In serious cases, they may even affect the scientific nature of environmental management and policy formulation.

[0003] Existing technologies have significant deficiencies in detecting outliers in environmental monitoring data. Traditional statistical methods such as the Grubbs test and the Z test identify outliers by detecting the deviation between the data and the sample center value. They rely on the normal distribution and large sample size assumptions, while environmental monitoring data usually exhibit non-stationarity and complex distributions, making it difficult to accurately detect outliers. At the same time, these methods have high computational costs and poor real-time performance, making it difficult to meet the needs of high-frequency dynamic data. Although machine learning methods have strong nonlinear processing capabilities, they rely on a large amount of labeled data, have high computational complexity, are not adaptable to real-time scenarios, and are prone to ignoring the dynamic characteristics of time series data, leading to false detections or missed detections.

[0004] These deficiencies can lead to inaccurate outlier detection, affect the credibility of monitoring data, and mislead pollution assessment and decision-making. At the same time, inefficient processing increases system complexity and maintenance costs.

[0005] Therefore, there is an urgent need for a real-time, efficient and dynamic time series outlier detection method to improve the reliability and practicality of environmental monitoring data. Summary of the invention

[0006] In order to overcome the defects of the prior art, the present invention combines the sliding window, ARIMA model and optimized Grubbs test to propose an efficient and accurate method for identifying outliers in environmental monitoring data, which effectively solves the problems of insufficient real-time performance and accuracy in the prior art.

[0007] In order to achieve the above object, the present invention provides the following technical solutions: A method for identifying abnormal values ​​in environmental monitoring data, characterized in that it comprises the following steps: 1) Data collection: obtaining real-time data with time stamps from environmental monitoring equipment; 2) Data preprocessing: preprocess the acquired real-time data; 3) Stabilization processing: Based on the ARIMA model, the long-term trend and periodic fluctuations in the pre-processed real-time data are removed through differential operation to obtain the differential estimate of the real-time data, thereby converting the real-time data from a non-stationary time series to a stationary time series; 4) Sliding window construction: setting the width of the sliding window and step length , according to the time sequence, the differential estimate of the real-time data of the stationary time series is divided into multiple overlapping sliding windows, each of which contains The difference estimate of real-time data, with a step size of Slide forward; 5) Calculation of statistical parameters within the sliding window: For each sliding window, calculate the The window mean of the difference estimates of real-time data and standard deviation :

[0008]

[0009] in, is the difference estimate of the real-time data at time point ti; 6) Optimize the Grubbs test: For each real-time data contained in the sliding window , calculate its mean with the window The deviation is standardized and the test statistic is obtained :

[0010] When the calculated T value exceeds the 95% confidence interval, the real-time data is considered It is an outlier.

[0011] Preferably, the stabilization process in step 3) is specifically as follows: 3.1) Determine the number of differences d of the ARIMA model: Perform a stationarity analysis on the preprocessed real-time data to check whether the preprocessed real-time data is a stationary time series. If it is a non-stationary time series, a difference operation is required. Each difference will eliminate the trend item in the real-time data. After d differences, the preprocessed real-time data becomes a stationary time series. The formula for the dth difference is:

[0012] In the formula, Indicates the current time point Real-time data The d-th difference value of ; Indicates the current time point Real-time data The d-1th difference value of ; Indicates the current time point The previous time point Real-time data The d-1th difference value of ; 3.2) Determine the model order of the ARIMA model: Determine the order of the autoregressive part in the ARIMA model by calculating the autocorrelation function and partial autocorrelation function of real-time data with timestamps and the order of the moving average part ; 3.3) ARIMA model building: The established ARIMA model is an autoregressive moving average model, and its mathematical form is:

[0013] In the formula, Indicates the current time point obtained based on the ARIMA model Real-time data The d-th difference value of ; Indicates the previous time point Real-time data The random error of is the autoregressive coefficient, describing the current time point Real-time data With the past Time point Real-time data , ,… the relationship between; is the sliding average coefficient, describing the current time point Real-time data With the past Time point Real-time data , ,… The random error the relationship between; 3.4) Solve the autoregressive coefficients of the ARIMA model using the Yule-Walker equation , sliding average coefficient and The mean ; 3.5) Based on the autoregressive coefficient and , get the current time point by iteration Real-time data The difference estimate of , .

[0014] Preferably, in step 3.1), a unit root test method is used to test and determine whether the preprocessed real-time data is a stationary time series.

[0015] Preferably, in step 3.2), the model order of the ARIMA model is determined as follows: 3.2.1) Calculate the autocorrelation function and partial autocorrelation function of real-time data with time stamp; 3.2.2) By analyzing the partial autocorrelation function, find the lag order at which the partial autocorrelation function significantly drops to zero , specifically, when the partial autocorrelation function is at lag When it drops sharply to zero or becomes insignificant, the order of the autoregressive part is considered to be ; 3.2.3) By analyzing the autocorrelation function, find the lag order at which the autocorrelation function significantly drops to zero , specifically, when the autocorrelation function is at the lag When it drops sharply to zero or becomes insignificant, the order of the moving average part is considered to be .

[0016] Preferably, in step 1), the real-time data with a timestamp obtained is the concentration ratio of PM2.5 to PM10.

[0017] Preferably, the step 1) further includes uniformly processing the acquired real-time data with timestamps, including time series formatting, unit conversion and abnormal character cleaning to ensure data consistency.

[0018] Preferably, in step 2), the preprocessing of the acquired real-time data includes: supplementing the missing values ​​in the acquired real-time data by using a linear interpolation method.

[0019] In addition, the present invention also provides an environmental monitoring data abnormal value identification system, which is characterized by comprising: A data acquisition module, which is used to obtain real-time data with time stamps from environmental monitoring equipment; A data preprocessing module, which is used to preprocess the acquired real-time data; The stabilization processing module is used to remove the long-term trend and periodic fluctuations in the pre-processed real-time data through differential operation based on the ARIMA model to obtain the differential estimation value of the real-time data, thereby converting the real-time data from a non-stationary time series to a stationary time series; Sliding window construction module, which is used to set the width of the sliding window and step length , according to the time sequence, the differential estimate of the real-time data of the stationary time series is divided into multiple overlapping sliding windows, each of which contains The difference estimate of real-time data, with a step size of Slide forward; The statistical parameter calculation module in the sliding window is used to calculate the statistical parameters contained in the sliding window for each sliding window. The window mean of the difference estimates of real-time data and standard deviation :

[0020]

[0021] in, is the difference estimate of the real-time data at time point ti; Optimize the Grubbs test module, for each real-time data contained in the sliding window , which is used to calculate its and the window mean The deviation is standardized and the test statistic is obtained :

[0022] When the calculated T value exceeds the 95% confidence interval, the real-time data is considered It is an outlier.

[0023] Moreover, the present invention also provides an environmental monitoring data abnormal value identification device, which is characterized by comprising: one or more processors; A memory for storing one or more programs; When the one or more programs are executed by the one or more processors, the one or more processors implement the method for identifying abnormal values ​​in environmental monitoring data as described above.

[0024] Finally, the present invention also provides a computer-readable storage medium having a computer program stored thereon, characterized in that when the program is executed by a processor, the steps of the method for identifying abnormal values ​​of environmental monitoring data as described above are implemented.

[0025] Compared with the prior art, the method, system, device and storage medium for identifying abnormal values ​​in environmental monitoring data of the present invention have one or more of the following beneficial technical effects: 1. Aiming at the problem that the non-stationarity of environmental monitoring data leads to the failure of traditional statistical methods to detect, the present invention introduces the ARIMA model, removes the long-term trend and periodic fluctuations in the data through differential operation, and transforms the non-stationary time series into a stationary time series, providing a more stable basis for outlier detection. Compared with the existing static assumptions, the ARIMA model is more flexible and can adapt to the dynamic change characteristics of environmental monitoring data.

[0026] 2. The present invention adopts a sliding window mechanism, which divides the stationary time series into local windows of fixed width, containing only the current data point and its previous data, and realizes real-time update of the detection range in a dynamic sliding manner. Compared with the traditional global sample-dependent detection method, this mechanism not only significantly improves the real-time performance of detection, but also reduces the computational cost, and is particularly suitable for high-frequency, continuous acquisition of environmental monitoring scenarios.

[0027] 3. The present invention optimizes the traditional Grubbs test and proposes a fast judgment criterion based on standard deviation, replacing the complex confidence interval calculation with an empirically simplified method, which greatly reduces the computational complexity while ensuring that the detection accuracy is not affected. This optimized design is particularly suitable for resource-constrained monitoring equipment, enabling it to perform efficient detection under low power consumption conditions.

[0028] 4. The present invention organically combines the sliding window with the optimized Grubbs test to significantly enhance the local outlier detection capability, especially in high-noise, complex-fluctuation environmental monitoring scenarios, and can accurately identify short-term burst outliers and reduce false detections and missed detections.

[0029] 5. While improving data quality and credibility, the present invention reduces computing and operation and maintenance costs, and provides a scientific, reliable and efficient outlier detection and correction method for the field of environmental monitoring, which has broad application prospects and promotion value. BRIEF DESCRIPTION OF THE DRAWINGS

[0030] Figure 1 It is a flow chart of the method for identifying abnormal values ​​of environmental monitoring data of the present invention.

[0031] Figure 2 It is a schematic diagram of the structure of the environmental monitoring data abnormal value identification system of the present invention. DETAILED DESCRIPTION

[0032] Before describing in detail any embodiment of the present invention, it should be understood that the present invention is not limited in its application to the construction and arrangement details of the components described below or illustrated in the following figures. The present invention can have other embodiments and can be practiced or carried out in various ways. In addition, it should be understood that the words and terms used here are for descriptive purposes and should not be considered restrictive. The use of "including" or "having" and its variations herein is intended to cover the items and their equivalents and additional items displayed below. Unless otherwise specified or limited, the terms "install", "connect", "support" and "couple" and their variations are widely used and cover direct installation and indirect installation, connection, support and connection. In addition, "connect" and "couple" are not limited to physical or mechanical connections or connections.

[0033] Furthermore, in the disclosure of the present invention, the term "one" should be understood as "at least one" or "one or more", that is, in one embodiment, the number of an element may be one, while in another embodiment, the number of the element may be multiple, and the term "one" should not be understood as a limitation on the quantity.

[0034] In view of the defects of the prior art, the present invention combines the sliding window, ARIMA model and optimized Grubbs test to propose an efficient and accurate method for detecting outliers in environmental monitoring data, which effectively solves the problems of insufficient real-time performance and accuracy in the prior art.

[0035] Figure 1 The flowchart of the method for identifying abnormal values ​​of environmental monitoring data of the present invention is shown. Figure 1 As shown, the method for identifying abnormal values ​​of environmental monitoring data of the present invention comprises the following steps: 1. Data collection.

[0036] In the present invention, real-time data with time stamps can be obtained from the environmental monitoring equipment, including hourly values ​​of PM2.5 and PM10. At the same time, in the present invention, after obtaining the hourly values ​​of PM2.5 and PM10, the concentration ratio of PM2.5 to PM10 can be obtained through them, and the obtained concentration ratio of PM2.5 to PM10 is used as the basis for subsequent processing.

[0037] Preferably, the acquired real-time data with timestamps can be processed uniformly, including time series formatting, unit conversion, and abnormal character cleaning, to ensure data consistency. These uniform processing methods are commonly used in the art and are not described in detail here for simplicity.

[0038] 2. Data preprocessing.

[0039] In the present invention, the acquired real-time data needs to be preprocessed, mainly for missing values ​​in the real-time data, using a linear interpolation method to supplement the missing values. Specifically, the linear interpolation algorithm is used to calculate the supplementary value according to the linear relationship between the adjacent data before and after the missing point, thereby ensuring the integrity of the time series data.

[0040] 3. Stabilization processing.

[0041] The real-time data after pre-processing comes from the monitoring equipment, which may contain long-term trends and periodic fluctuations, and therefore may be a non-stationary time series. Therefore, it is necessary to perform a stationarity analysis on the real-time data after pre-processing.

[0042] In the present invention, the ADF (unit root test) method can be used to test and determine whether the pre-processed real-time data is a stationary time series. If the pre-processed real-time data is a non-stationary time series, a differential operation is required. Each differential operation will eliminate the trend item in the data. After multiple differential operations, the real-time data is converted from a non-stationary time series to a stationary time series.

[0043] At present, when performing differentiation in the prior art, each real-time data is differentiated using the following formula:

[0044] In the formula, Indicates the current time point Real-time data The d-th difference value of ; Indicates the current time point Real-time data The d-1th difference value of ; Indicates the current time point The previous time point Real-time data The d-1th difference value of .

[0045] For example, for the current time point Real-time data , after a difference, we set ,at this time, , where Indicates the current time point Real-time data The first difference value of Indicates the current time point Real-time data ; Indicates the current time point The previous time point Real-time data .

[0046] Use the ADF (unit root test) method to test whether the first difference value of all real-time data is a stationary time series. If it is not a stationary time series, increase the number of differences so that ,at this time, , where Indicates the current time point Real-time data The second difference value of Indicates the current time point Real-time data The first difference value of Indicates the current time point The previous time point Real-time data The first difference value of .

[0047] Afterwards, the ADF (unit root test) method is used to test whether the second difference value of all real-time data is a stationary time series. If it is not a stationary time series, the number of differences is increased and the difference is continued until the dth difference value is a stationary time series.

[0048] However, the above differential method belongs to static assumptions and cannot adapt to the dynamic change characteristics of environmental monitoring data. To this end, in the present invention, the ARIMA model is introduced. Based on the ARIMA model, the long-term trend and periodic fluctuations in the pre-processed real-time data are removed by differential operation to obtain the differential estimate of the real-time data, thereby converting the real-time data from a non-stationary time series to a stationary time series, as follows: 1. Determine the number of differences d for the ARIMA model.

[0049] In the present invention, first, the difference method described above is used to determine the number of differences d of the ARIMA model. That is, the pre-processed real-time data is subjected to a stationary analysis to determine whether the pre-processed real-time data is a stationary time series. If it is a non-stationary time series, a difference operation is required. Each difference operation will eliminate the trend item in the real-time data. The pre-processed real-time data becomes a stationary time series through d-time differences. The formula for the d-th difference is:

[0050] In the formula, Indicates the current time point Real-time data The d-th difference value of ; Indicates the current time point Real-time data The d-1th difference value of ; Indicates the current time point The previous time point Real-time data The d-1th difference value of .

[0051] 2. Determine the model order of the ARIMA model.

[0052] In the present invention, the order of the autoregressive part (AR) in the ARIMA model is determined by calculating the ACF (autocorrelation function) and PACF (partial autocorrelation function) of real-time data with timestamps. and the order of the moving average part (MA) , which are specifically: 1) Calculate the autocorrelation function (ACF) and partial autocorrelation function (PACF) of real-time data with timestamps.

[0053] 2) By analyzing the partial autocorrelation function PACF, find the lag order at which the partial autocorrelation function PACF drops significantly to zero Specifically, when the partial autocorrelation function PACF is lagged When it drops sharply to zero or becomes insignificant, the order of the autoregressive part AR is considered to be .

[0054] 3) By analyzing the autocorrelation function ACF, find the lag order at which the autocorrelation function ACF drops significantly to zero Specifically, when the autocorrelation function ACF is at lag When it drops sharply to zero or becomes insignificant, the order of the moving average part MA is considered to be .

[0055] For example, if the partial autocorrelation function PACF is truncated after lag 1 while the autocorrelation function ACF decays slowly, then the AR(1) model is selected; if the ACF is truncated after lag 1 while the PACF decays slowly, then the MA(1) model is selected; if both the ACF and the PACF are truncated after a certain lag order, then the MA(1) model is selected. After truncation, select Model.

[0056] 3. ARIMA model building.

[0057] The established ARIMA model is an autoregressive moving average model, and its mathematical form is: .

[0058] In the formula, Indicates the current time point obtained based on the ARIMA model Real-time data The d-th difference value of ; , Represents the past obtained based on the ARIMA model Time point Real-time data , ,… The d-th difference value of ; is the autoregressive coefficient, describing the current time point Real-time data With the past Time point Real-time data , ,… the relationship between; is the sliding average coefficient, describing the current time point Real-time data With the past Time point Real-time data , ,… The random error the relationship between; Indicates the previous time point Real-time data of random errors.

[0059] 4. Solve the autoregressive coefficient of the ARIMA model through the Yule-Walker equation , sliding average coefficient and The mean .

[0060] Through steps 1-3, d, p, and q are determined. In this way, the autoregressive coefficients of the ARIMA model can be solved by the Yule-Walker equation. , sliding average coefficient and The mean .

[0061] The Yule-Walker equation method is a commonly used method for solving the ARIMA model. It is divided into two steps: first, it is used to estimate the parameters of the AR model. , by calculating the sample autocorrelation function, establishing and solving the Yule-Walker linear equations to obtain the parameters of the AR model, and estimate the prediction error variance; secondly, since the MA part does not directly affect the autocorrelation function, the maximum likelihood estimation method is used to determine the parameters of the MA part The whole process is automatically completed by statistical software packages (such as the ARIMA function in the Python statsmodels.tsa.arima.model module) in practical applications. Through Ask for expectation.

[0062] 5. Based on the autoregressive coefficient and , get the current time point by iteration Real-time data The difference estimate of , .

[0063] Among them, due to , is a random error with mean 0, so The overall mean is 0, so the current time point can be obtained by the above formula Real-time data The difference estimate of .

[0064] In order to solve the problem that the non-stationarity of environmental monitoring data leads to the failure of traditional statistical methods to detect errors, this paper introduces the ARIMA model, removes the long-term trend and periodic fluctuations in the data through differential operations, and transforms the non-stationary time series into a stationary time series, providing a more stable basis for outlier detection. Compared with the existing static assumptions, the ARIMA model is more flexible and can adapt to the dynamic changes of environmental monitoring data.

[0065] 4. Sliding window construction.

[0066] In the present invention, a sliding window mechanism is adopted to divide the stationary time series into local windows with fixed width, which only contain the data at the current time point t and its previous data, and realize the real-time update of the detection range in a dynamic sliding manner. Specifically, the width of the sliding window is set to and step length For example, W is set to be all data within 24 hours, and S is all data within 1 hour each time. Thus, according to the time sequence, the differential estimate of the real-time data of the stationary time series is divided into multiple overlapping sliding windows, each of which contains The difference estimate of real-time data (i.e., the difference estimate of the real-time data at the current time point t and the real-time data at the previous W-1 time points) is calculated with a step size of Slide forward.

[0067] Therefore, each sliding window provides a local data set for outlier detection. Compared with the traditional global sample-dependent detection method, this mechanism not only significantly improves the real-time performance of detection, but also reduces the computational cost. It is especially suitable for environmental monitoring scenarios with high frequency and continuous collection.

[0068] 5. Calculation of statistical parameters within the sliding window.

[0069] For each sliding window, calculate the number of The window mean of the difference estimates of real-time data and standard deviation :

[0070]

[0071] in, is the differential estimate of the real-time data at time point ti. Since the value range of i is from 0 to W, It is the difference estimate of the real-time data at the current time point t and the real-time data at the previous W-1 time points.

[0072] 6. Optimize the Grubbs test.

[0073] For each real-time data contained in the sliding window , calculate its mean with the window The deviation is standardized and the test statistic is obtained : .

[0074] When the calculated T value exceeds the 95% confidence interval, the real-time data is considered A large number of empirical results show that when the difference between a data point and the sample median exceeds 5 times the standard deviation, it generally exceeds the 95% confidence interval. This result can be used to simplify the Grubbs test.

[0075] The present invention optimizes the traditional Grubbs test and proposes a fast judgment criterion based on standard deviation, replacing the complex confidence interval calculation with an empirically simplified method, which greatly reduces the computational complexity while ensuring that the detection accuracy is not affected. This optimized design is particularly suitable for resource-constrained monitoring equipment, enabling it to perform efficient detection under low power consumption conditions.

[0076] Moreover, the present invention significantly enhances the local outlier detection capability by organically combining the sliding window with the optimized Grubbs test, especially in high-noise, complex and volatile environmental monitoring scenarios, and can accurately identify short-term sudden outliers and reduce false detections and missed detections. Therefore, while improving data quality and credibility, the present invention reduces computing and operation and maintenance costs, and provides a scientific, reliable and efficient outlier detection and correction method for the field of environmental monitoring, which has broad application prospects and promotion value.

[0077] Figure 2 The schematic diagram of the system for identifying abnormal values ​​of environmental monitoring data of the present invention is shown. Figure 2 As shown, the environmental monitoring data abnormal value identification system of the present invention includes: 1. Data acquisition module.

[0078] The data acquisition module is used to obtain real-time data with time stamps from environmental monitoring equipment.

[0079] 2. Data preprocessing module.

[0080] The data preprocessing module is used to preprocess the acquired real-time data.

[0081] 3. Stabilization processing module.

[0082] The stabilization processing module is used to remove the long-term trend and periodic fluctuations in the pre-processed real-time data through differential operation based on the ARIMA model to obtain the differential estimation value of the real-time data, thereby converting the real-time data from a non-stationary time series into a stationary time series.

[0083] 4. Sliding window building module.

[0084] The sliding window construction module is used to set the width of the sliding window and step length , according to the time sequence, the differential estimate of the real-time data of the stationary time series is divided into multiple overlapping sliding windows, each of which contains The difference estimate of real-time data, with a step size of Slide forward.

[0085] 5. Statistical parameter calculation module within the sliding window.

[0086] For each sliding window, the statistical parameter calculation module in the sliding window is used to calculate the The window mean of the difference estimates of real-time data and standard deviation :

[0087]

[0088] in, It is the difference estimate of the real-time data at time point ti.

[0089] 6. Optimize the Grubbs test module.

[0090] For each real-time data contained in the sliding window , the optimized Grubbs test module is used to calculate its difference with the window mean The deviation is standardized and the test statistic is obtained :

[0091] When the calculated T value exceeds the 95% confidence interval, the real-time data is considered It is an outlier.

[0092] Moreover, the present invention also relates to an environmental monitoring data outlier identification device, which includes: one or more processors; a memory for storing one or more programs; when the one or more programs are executed by the one or more processors, the one or more processors implement the environmental monitoring data outlier identification method as described above.

[0093] Finally, the present invention also provides a computer-readable storage medium on which a computer program is stored, and when the program is executed by a processor, the steps of the method for identifying abnormal values ​​of environmental monitoring data as described above are implemented.

[0094] Finally, it should be noted that the above embodiments are only used to illustrate the technical solution of the present invention, rather than to limit the scope of protection of the present invention. Those skilled in the art can modify or replace the technical solution of the present invention according to the idea of ​​the present invention without departing from the essence and scope of the technical solution of the present invention.

Claims

1. A method for identifying abnormal values ​​in environmental monitoring data, characterized in that: The following steps are involved: 1) Data collection: obtaining real-time data with time stamps from environmental monitoring equipment; 2) Data preprocessing: preprocess the acquired real-time data; 3) Stabilization processing: Based on the ARIMA model, the long-term trend and periodic fluctuations in the pre-processed real-time data are removed through differential operation to obtain the differential estimate of the real-time data, thereby converting the real-time data from a non-stationary time series to a stationary time series; 4) Sliding window construction: setting the width of the sliding window and step length , according to the time sequence, the differential estimate of the real-time data of the stationary time series is divided into multiple overlapping sliding windows, each of which contains The difference estimate of real-time data, with a step size of Slide forward; 5) Calculation of statistical parameters within the sliding window: For each sliding window, calculate the The window mean of the difference estimates of real-time data and standard deviation : ; ; in, is the difference estimate of the real-time data at time point ti; 6) Optimize the Grubbs test: For each real-time data contained in the sliding window , calculate its mean with the window The deviation is standardized and the test statistic is obtained : ; When the calculated T value exceeds the 95% confidence interval, the real-time data is considered It is an outlier.

2. The method for identifying abnormal values ​​of environmental monitoring data according to claim 1, characterized in that: The stabilization process of step 3) is specifically as follows: 3.1) Determine the number of differences d of the ARIMA model: Perform a stationarity analysis on the preprocessed real-time data to check whether the preprocessed real-time data is a stationary time series. If it is a non-stationary time series, a difference operation is required. Each difference will eliminate the trend item in the real-time data. After d differences, the preprocessed real-time data becomes a stationary time series. The formula for the dth difference is: ; In the formula, Indicates the current time point Real-time data The d-th difference value of ; Indicates the current time point Real-time data The d-1th difference value of ; Indicates the current time point The previous time point Real-time data The d-1th difference value of ; 3.2) Determine the model order of the ARIMA model: Determine the order of the autoregressive part in the ARIMA model by calculating the autocorrelation function and partial autocorrelation function of real-time data with timestamps and the order of the moving average part ; 3.3) ARIMA model building: The established ARIMA model is an autoregressive moving average model, and its mathematical form is: ; In the formula, Indicates the current time point obtained based on the ARIMA model Real-time data The d-th difference value of ; Indicates the previous time point Real-time data The random error of is the autoregressive coefficient, describing the current time point Real-time data With the past Time point Real-time data , ,… the relationship between; is the sliding average coefficient, describing the current time point Real-time data With the past Time point Real-time data , ,… The random error the relationship between; 3.4) Solve the autoregressive coefficients of the ARIMA model using the Yule-Walker equation , sliding average coefficient and The mean ; 3.5) Based on the autoregressive coefficient and , get the current time point by iteration Real-time data The difference estimate of , 。 3. The method for identifying abnormal values ​​of environmental monitoring data according to claim 2, characterized in that: In the step 3.1), the unit root test method is used to test and determine whether the preprocessed real-time data is a stationary time series.

4. The method for identifying abnormal values ​​of environmental monitoring data according to claim 3, characterized in that: In step 3.2), the model order of the ARIMA model is determined as follows: 3.2.1) Calculate the autocorrelation function and partial autocorrelation function of real-time data with time stamp; 3.2.2) By analyzing the partial autocorrelation function, find the lag order at which the partial autocorrelation function significantly drops to zero , specifically, when the partial autocorrelation function is at lag When it drops sharply to zero or becomes insignificant, the order of the autoregressive part is considered to be ; 3.2.3) By analyzing the autocorrelation function, find the lag order at which the autocorrelation function significantly drops to zero , specifically, when the autocorrelation function is at the lag When it drops sharply to zero or becomes insignificant, the order of the moving average part is considered to be .

5. The method for identifying abnormal values ​​in environmental monitoring data according to any one of claims 1 to 4, characterized in that: In the step 1), the real-time data with a timestamp obtained is the concentration ratio of PM2.5 to PM10.

6. The method for identifying abnormal values ​​of environmental monitoring data according to claim 5, characterized in that: The step 1) also includes unified processing of the acquired real-time data with timestamps, including time series formatting, unit conversion and abnormal character cleaning to ensure data consistency.

7. The method for identifying abnormal values ​​in environmental monitoring data according to claim 6, characterized in that: In the step 2), the acquired real-time data is preprocessed by using a linear interpolation method to supplement the missing values ​​in the acquired real-time data.

8. A system for identifying abnormal values ​​in environmental monitoring data, characterized in that: include: A data acquisition module, which is used to obtain real-time data with time stamps from environmental monitoring equipment; A data preprocessing module, which is used to preprocess the acquired real-time data; The stabilization processing module is used to remove the long-term trend and periodic fluctuations in the pre-processed real-time data through differential operation based on the ARIMA model to obtain the differential estimation value of the real-time data, thereby converting the real-time data from a non-stationary time series to a stationary time series; Sliding window construction module, which is used to set the width of the sliding window and step length , according to the time sequence, the differential estimate of the real-time data of the stationary time series is divided into multiple overlapping sliding windows, each of which contains The difference estimate of real-time data, with a step size of Slide forward; The statistical parameter calculation module in the sliding window is used to calculate the statistical parameters contained in the sliding window for each sliding window. The window mean of the difference estimates of real-time data and standard deviation : ; ; in, is the difference estimate of the real-time data at time point ti; Optimize the Grubbs test module, for each real-time data contained in the sliding window , which is used to calculate its and the window mean The deviation is standardized and the test statistic is obtained : ; When the calculated T value exceeds the 95% confidence interval, the real-time data is considered It is an outlier.

9. A device for identifying abnormal values ​​of environmental monitoring data, characterized in that: include: one or more processors; A memory for storing one or more programs; When the one or more programs are executed by the one or more processors, the one or more processors implement the method for identifying abnormal values ​​in environmental monitoring data as described in any one of claims 1 to 7.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the steps of the method for identifying abnormal values ​​of environmental monitoring data as described in any one of claims 1 to 7 are implemented.

Images