A method for retrospective analysis of pollutant monitoring data from fixed atmospheric environmental monitoring stations

Abstract

This invention belongs to the field of atmospheric environmental monitoring and relates to a method for backtracking pollutant monitoring data from fixed atmospheric environmental monitoring stations. The method constructs a BPCA-LSTM model combining Bayesian Principal Component Analysis (BPCA) and Long Short-Term Memory (LSTM) networks. Based on the range of pollutant concentration monitoring data to be backtracked from M stations, a usable complete dataset is selected from the historical pollutant concentration monitoring data of all N stations. The selected dataset undergoes data cleaning and dimensionality reduction to learn the time-series data mapping relationship between N-M stations and M station. Using this model, the missing data from N-M stations corresponding to the period of data loss at M station is used to backtrack the missing data from M station. This invention's method is simple and universal, providing a reliable data backtracking scheme for atmospheric environmental monitoring, and has significant practical implications and broad application prospects.

Description

Technical Field

[0001] This invention belongs to the field of atmospheric environment monitoring and relates to a method for retrospective analysis of pollutant monitoring data from fixed atmospheric environment monitoring stations. Background Technology

[0002] With the continued advancement of urbanization and the shift of the economy towards a post-industrial stage, green travel is playing an increasingly important role in modern urban governance and is a crucial pathway to achieving urban ecological civilization. The promotion and implementation of green travel relies on the assessment and forecasting of air quality. Complete pollutant monitoring data from fixed atmospheric environmental monitoring stations are essential for air quality assessment and forecasting, and also a vital data foundation for capturing the causes, evolution, and dissipation patterns of concentrations. The lack of such data not only affects the comprehensive assessment of the urban atmospheric environment but also restricts the accurate location of pollution sources and the formulation of air pollution control measures.

[0003] In actual monitoring, data gaps at monitoring stations are unavoidable due to various reasons such as equipment failure, maintenance, and additions / removals. Solutions to these gaps in air pollutant data can be broadly categorized into two types. One type utilizes pollution diffusion models such as AERMOD, CALINE 4, and ADMS to simulate pollution diffusion. These models rely on complete pollution emission inventories, are computationally time-consuming, and cannot guarantee the accuracy of the simulation results. The other type focuses on data supplementation through spatiotemporal interpolation methods, Bayesian methods, and deep learning. This research primarily concentrates on the street-level, analyzing pollution conditions in areas based on relatively dense data sampling. Currently, there is a lack of an effective method for backtracking data at the city-level, posing a challenge to a comprehensive assessment of pollution conditions at the meso-scale. Summary of the Invention

[0004] To address the aforementioned issues, this invention provides a method for backtracking pollutant monitoring data from fixed atmospheric environmental monitoring stations. This method utilizes a combination of statistical and deep learning approaches to learn the concentration time-series data mapping relationship between data from NM monitoring stations (excluding the M monitoring station with missing data) and data from the M monitoring station. Then, it uses data from the NM monitoring stations corresponding to the period when the M station data is missing to backtrack and complete the missing data from the M station.

[0005] Technical solution of the present invention:

[0006] A method for retrospectively analyzing pollutant monitoring data from fixed atmospheric environmental monitoring stations includes the following steps:

[0007] S1. Construct a BPCA-LSTM network model, including a Bayesian Principal Component Analysis (BPCA) part and a Long Short-Term Memory (LSTM) network part; at the same time, based on the range of pollutant concentration monitoring data that needs to be traced back for M stations, select and form a usable complete dataset from the historical pollutant concentration monitoring data of all N stations.

[0008] S2. Perform data cleaning on the selected dataset, including pollutant concentration standardization, outlier removal and missing point imputation, and apply the Bayesian principal component analysis to the cleaned data for dimensionality reduction.

[0009] S3. Set the initial data sliding window and BPCA-LSTM network model hyperparameters, and use the long short-term memory network part to construct the time-series pollutant concentration data mapping relationship between NM sites and each of the M sites;

[0010] S4. Optimize and adjust the sliding window duration and the hyperparameters of the BPCA-LSTM network model;

[0011] S5. Using historical data from NM stations corresponding to the period when data from station M is missing as input, the missing pollution concentration monitoring data from station M is traced back through the constructed BPCA-LSTM network model.

[0012] This invention utilizes Bayesian Principal Component Analysis (BPCA) and Long Short-Term Memory (LSTM) technologies to retrospectively analyze pollutant monitoring data from fixed atmospheric environmental monitoring stations. By employing a BPCA-LSTM model, it learns the time-series concentration data mapping relationship between monitoring stations with missing data and those with complete data, thereby filling in the missing data. This method effectively solves the problem of missing monitoring data caused by equipment failure, maintenance, or additions / removals, filling a gap in urban-scale monitoring data retrospective methods. It provides a reliable solution for governments and environmental protection departments, supports more scientific air quality assessment and pollution control, promotes green travel development and ecological civilization construction, and has significant practical implications and broad application prospects.

[0013] Beneficial effects of this invention:

[0014] Achieving accurate data backtracking: By combining statistical and deep learning methods, we can effectively capture the temporal relationship patterns of air pollutant concentrations and accurately backtrack missing data.

[0015] Filling a technological gap: It fills the technological gap in retrospective analysis of atmospheric environmental monitoring data at the urban scale, which has significant practical implications and application prospects.

[0016] It has wide applicability: it is suitable for backtracking monitoring data of different types of pollutants, has strong versatility, and can be used in a variety of urban environmental monitoring application scenarios.

[0017] Improve work efficiency: Use data cleaning and dimensionality reduction techniques (such as BPCA) to preprocess data, reduce data dimensionality and noise, and optimize the sliding window and network model hyperparameters through surrogate models to reduce computation time, ensuring that data backtracking is completed within a limited time and improving efficiency.

[0018] Supporting scientific decision-making: Providing reliable pollutant monitoring data retrospective results, offering air quality assessment and atmospheric pollution control basis for governments and environmental protection departments, supporting scientific decision-making, promoting the development of green travel, and contributing to the construction of urban ecological civilization. Attached Figure Description

[0019] Figure 1 This is the main process of a method for retrospective analysis of pollutant monitoring data from fixed atmospheric environment monitoring stations according to the present invention;

[0020] Figure 2 This is a schematic diagram of the weight matrix used for data dimensionality reduction via BPCA in step 2 of the present invention.

[0021] Figure 3 This invention utilizes complete period data to construct mapping relationships and traces back the overall process of missing period data;

[0022] Figure 4 This is a schematic diagram of the bidirectional LSTM structure used in step 3 of the present invention;

[0023] Figure 5 This is a scatter plot of the true values ​​and predicted values ​​on the training set of the optimal model obtained based on the present invention in the embodiment.

[0024] Figure 6 The example shows the site data backtracking results obtained based on the present invention. Detailed Implementation

[0025] The technical solutions provided in this application will be further described below with reference to specific embodiments and accompanying drawings. The advantages and features of this application will become clearer from the following description.

[0026] like Figure 1 As shown, a method for retrospective analysis of pollutant monitoring data from fixed atmospheric environmental monitoring stations includes the following steps:

[0027] S1. Construct a BPCA-LSTM network model, including a Bayesian Principal Component Analysis (BPCA) part and a Long Short-Term Memory (LSTM) network part; at the same time, based on the range of pollutant concentration monitoring data that needs to be traced back for M stations, select and form a usable complete dataset from the historical pollutant concentration monitoring data of all N stations.

[0028] S2. Perform data cleaning on the selected dataset, including pollutant concentration standardization, outlier removal and missing point imputation, and apply the Bayesian principal component analysis to the cleaned data for dimensionality reduction.

[0029] S3. Set the initial data sliding window and BPCA-LSTM network model hyperparameters, and use the long short-term memory network part to construct the time-series pollutant concentration data mapping relationship between NM sites and each of the M sites;

[0030] S4. Optimize and adjust the sliding window duration and the hyperparameters of the BPCA-LSTM network model;

[0031] S5. Using historical data from NM stations corresponding to the period when data from station M is missing as input, the missing pollution concentration monitoring data from station M is traced back through the constructed BPCA-LSTM network model.

[0032] Furthermore, in step S1, the data selected from the historical data of all N stations are generally standardized to be the pollutant concentration monitoring data of N stations 2 to 3 years after the missing pollutant concentration data of the M stations whose data needs to be traced back. This ensures sufficient training and testing data in step S3, and that the relationship pattern of concentration time series data among the N stations covered by the selected data can effectively characterize the concentration time series data mapping relationship between the NM stations and the M station within the missing data interval of the M station. Data from the NM stations with a data completeness of less than 90% needs to be discarded. Data completeness is defined by the following formula:

[0033]

[0034] Where Ω represents the completeness of the data, and Q represents the total amount of data that should be monitored in the dataset. a This represents the total amount of data in the dataset that was not monitored.

[0035] Further, step S2 involves data cleaning and dimensionality reduction of the selected dataset. Data cleaning includes pollutant concentration standardization, outlier removal, and missing value imputation, as detailed below:

[0036] (a3.1) For the standardization of pollutant concentration measurements, if the stored data uses concentration values ​​under standard conditions (273.15K, 101.325kPa), then it should be converted to concentration values ​​under actual conditions. Considering the small difference in atmospheric pressure, the simplified conversion formula is as follows:

[0037]

[0038] Among them, C r C represents the concentration value under actual conditions. s T represents the concentration value under standard conditions. r This indicates the ambient temperature under actual conditions, and it uses the Kelvin temperature scale.

[0039] (a3.2) Outlier removal involves removing outlier detection records (e.g., data of 9999 and -9999) and outliers. Outliers are defined as values ​​less than the lower limit or greater than the upper limit. The formulas for calculating the upper and lower limits are as follows:

[0040]

[0041] Where U and L represent the upper and lower limits, respectively; Q3 and Q1 represent the upper and lower quartiles, respectively; and IQR represents the interquartile range, i.e., the distance between Q3 and Q1.

[0042] (a3.3) For missing data completion, missing data in the original data is completed with the average concentration of pollutants within the selected data range. For data removed due to outliers, if the original data was an abnormal detection record, it is completed with the average concentration mentioned above. If the data is an outlier, if it is less than the lower limit, it is completed with the minimum value of the data after removing the outlier. If it is greater than the upper limit, it is completed with the maximum value of the data after removing the outlier.

[0043] Next, Bayesian principal component analysis was used to reduce the dimensionality of the cleaned data. The dimensionality reduction process was performed using the Expectation-Maximization (EM) algorithm. For the actual concentration vector at a given time... D is the feature dimension, where x is the vector representing the actual concentrations of all stations except those with missing data at that moment. The goal of dimensionality reduction is to find a low-dimensional representation. and the corresponding weight matrix (like Figure 2 (as shown on the right) and a bias vector Make:

[0044] x=Wz+μ+ε (4)

[0045] Where ε is the noise term. The following prior distribution assumptions hold regarding x, z, and W: Let I represent a Gaussian distribution, where I is the identity matrix and σ is the distribution. 2 It is the variance of observation noise:

[0046]

[0047] The set of parameters to be estimated under the current assumptions is θ = {W, μ, σ}. 2 ,α}, where μ is estimated using the following formula:

[0048]

[0049] Parameters W, σ 2 α is updated iteratively using the EM algorithm. The EM algorithm estimates the model parameters by alternately updating the E-step (expectation step) and the M-step (maximization step). The specific steps of the EM algorithm are described below:

[0050] (b3.1) Initialize parameters W and σ 2 α;

[0051] (b3.2) In step E, the expected value and covariance of the posterior distribution of the latent variable z are calculated using the following formula:

[0052] Where M = W T W+σ 2 I (10)

[0053]

[0054] (b3.3) In step M, parameters W and σ are updated according to the expected maximization. 2 α, and its update formula is as follows:

[0055]

[0056] Where Tr(·) represents the trace of the matrix;

[0057] (b3.4) Repeat steps E and M until the model parameters converge, i.e., the iterative change of the parameters is less than the set threshold.

[0058] Furthermore, step S3 constructs a time-series pollutant concentration data mapping relationship between the NM stations and each of the M stations. This is used to backtrack the data of the M stations using the data from the remaining NM stations during periods when the M station's data is missing. Specifically, for the current time T, a time window [Tn, T+n] is selected, where n is a hyperparameter for adjusting the window size. The pollutant concentration data within the time window of the NM stations are used as input (Z). T-n ,…,ZT ,…,Z T+n The output is the concentration value of the station to be traced back at the current time T (Y). T ), and under selected network hyperparameters, a mapping relationship between the two is constructed using a Long Short-Term Memory (LSTM) network (e.g. Figure 3 (As shown). This mapping relationship considers the lag in the impact of pollution diffusion, using concentration information from other sites around the unknown data site over a time window from the past to the future relative to the retrospective time to infer the concentration value of the unknown data site at the retrospective time. The network model adopts a bidirectional structure, that is, constructing a bidirectional LSTM (Bi-LSTM network), as shown... Figure 4 As shown, the concentration relationship patterns of time series data in two directions within a time window are captured and memorized: from past to future (forward time series) and from future to past (reverse time series). The training and validation sets are divided in a 5:1 ratio.

[0059] Further, step S4 optimizes the hyperparameters using a differentiated optimization strategy. The adjusted network hyperparameters include the time window size, learning rate, regularization parameter, batch size, network layers, number of neurons, and loss function. This method uses mean squared error as the loss function, dynamically adjusts the learning rate using the Adadelta optimizer, and employs L2 regularization. Other hyperparameters include the time window size, batch size, network layers, and number of neurons. The hyperparameter space is defined as x, and a Gaussian process is used. The specific steps for hyperparameter optimization using a surrogate model are as follows:

[0060] (a4.1) In the initialization phase, define the hyperparameter space x, and randomly sample the initial hyperparameter point set A = {a1, a2, ..., a...}. n Based on the initial hyperparameter point set, a Bi-LSTM network model is trained, and the validation set loss y = {y1, y2, ..., y} corresponding to each set of hyperparameters is calculated. n Define a Gaussian process f(a):

[0061]

[0062] Where μ(a) represents the mean function, i.e. k(a,a′) is the kernel function, characterizing the correlation between a and a′ in the hyperparameter space. Here, the radial basis function (RBF) is used, with the initial sample set. Train the Gaussian process model f(a);

[0063] (a4.2) Calculate the predicted distribution, for candidate point a, the predicted mean μ(a) and the predicted variance σ. 2The formula for calculating (a) is as follows:

[0064] μ(a)=k(a,A)(K(A,A)+σ 2 I) -1 y (16)

[0065] σ 2 (a)=k(a,a)-k(a,A)(K(A,A)+σ 2 I) -1 k(A,a) (17)

[0066] Where A is the matrix of evaluated points, y is the corresponding objective function value vector, K(A,A) is the covariance matrix of the kernel function on the evaluated points, k(a,A) is the covariance vector between the candidate point and the evaluated points, and k(a,a) is the covariance between the candidate point and itself.

[0067] (a4.3) Select and optimize the acquisition function, using the desired improvement (EI) as the acquisition function. Its calculation formula is as follows:

[0068]

[0069] Among them, a + The current optimal point is given by ξ, a non-negative exploration parameter used to control the trade-off between exploration and exploitation in the algorithm. Φ(·) and φ(·) are the cumulative distribution function and probability of the standard normal distribution, respectively. The algorithm finds the optimal point on the surrogate model that makes the acquisition function α... EI (a) The largest point a * , that is:

[0070]

[0071] (a4.4) Update the model for hyperparameter a * The training model yields the validation set loss y. * Add the new sampling points to the set of evaluated points to form a set {(a1,y1),(a2,y2),…,(a... n ,y n ),(a * ,y * The Gaussian process model was retrained using the updated dataset.

[0072] (a4.5) Check the stopping condition. Check if the stopping condition is met (maximum number of iterations, time limit, or objective function value reaches the expected value). If it is, stop. Otherwise, repeat (a4.2) to (a4.4).

[0073] Furthermore, in step S5, the BPCA-LSTM model is constructed and compiled using the PyTorch deep learning framework. The model is trained under the hyperparameter optimization framework, and the iteratively optimal hyperparameter settings and corresponding network model are obtained under a preset time limit. For the period when the data of station M is missing, the model is used as input to backtrack the pollutant concentration data of station M by taking the pollutant concentration time series window data of the other NM stations as input.

[0074] Example

[0075] Taking the missing NO2 data of the Jing'an Yan'an West Road Transportation Station in Shanghai in 2018 as an example, based on the data backtracking method proposed in this invention, using complete full-site data from 2019 to 2021, a time-series mapping relationship between the NO2 concentration data of 11 national monitoring stations and the Jing'an Yan'an West Road Transportation Station is constructed through training a network model. Then, using the data of the 11 national monitoring stations in 2018 as input, the trained network model is used to complete the backtracking of the missing data of the Jing'an Yan'an West Road Transportation Station in 2018. The specific implementation steps are as follows:

[0076] S1. Select data from the historical data period 2 to 3 years after the missing data period to form training and validation datasets, to ensure sufficient training and testing data in step S3. The selected data should cover the relationship pattern of concentration time series data among all stations, which can effectively characterize the mapping relationship of NO2 concentration data between 11 national control stations and Jing'an Yan'an West Road Transportation Station within the data missing period. The data includes the following (as shown in Table 1):

[0077] (1) Training set: Standard condition data of 11 national monitoring stations and Jing'an Yan'an West Road Traffic Station NO2 in the second half of 2019, 2020 and 2021;

[0078] (2) Validation set: Standard condition data of 11 national monitoring stations and Jing'an Yan'an West Road Traffic Station NO2 in the first half of 2019;

[0079] The above data should discard site data with a data completeness of less than 90%. Data completeness is defined by the following formula:

[0080]

[0081] Where Ω represents the completeness of the data, and Q represents the total amount of data that should be monitored in the dataset. a This represents the total amount of data in the dataset that was not monitored.

[0082] Table 1. Training and validation datasets (partial list)

[0083]

[0084]

[0085] S2. Perform data cleaning and dimensionality reduction on the selected dataset. Data cleaning includes pollutant concentration standardization, outlier removal, and missing value imputation, as detailed below:

[0086] (a3.1) For the standardization of pollutant concentration measurements, if the stored data uses concentration values ​​under standard conditions (273.15K, 101.325kPa), then it should be converted to concentration values ​​under actual conditions. Considering the small difference in atmospheric pressure, the simplified conversion formula is as follows:

[0087]

[0088] Among them, C r C represents the concentration value under actual conditions. s T represents the concentration value under standard conditions. r This represents the ambient temperature under actual conditions, using the Kelvin temperature scale. Conversion examples are shown in Table 2 below.

[0089] Table 2 Standardization of Pollutant Concentration Measurement

[0090]

[0091]

[0092] (a3.2) For outlier removal, in addition to removing outlier detection records (e.g., data of 9999 and -9999), outliers are also removed. Outliers are defined as values ​​less than the lower limit or greater than the upper limit. The formulas for calculating the upper and lower limits are as follows:

[0093]

[0094] Where U and L represent the upper and lower limits, respectively; Q3 and Q1 represent the upper and lower quartiles, respectively; and IQR represents the interquartile range, i.e., the distance between Q3 and Q1.

[0095] (a3.3) For missing data completion, missing data in the original data is completed with the average concentration of pollutants within the selected data range. For data removed due to outliers, if the original data was an abnormal detection record, it is completed with the average concentration mentioned above. If the data is an outlier, if it is less than the lower limit, it is completed with the minimum value of the data after removing the outlier. If it is greater than the upper limit, it is completed with the maximum value of the data after removing the outlier.

[0096] Next, Bayesian principal component analysis was used to reduce the dimensionality of the cleaned data. The dimensionality reduction process was performed using the Expectation-Maximization (EM) algorithm. For the actual concentration vector at a given time... D is the feature dimension, where x is the vector representing the actual concentrations of all stations except those with missing data at that moment. The goal of dimensionality reduction is to find a low-dimensional representation. and the corresponding weight matrix and a bias vector Make:

[0097] x=Wz+μ+ε (4)

[0098] Where ε is the noise term. The following prior distribution assumptions hold regarding x, z, and W: Let I represent a Gaussian distribution, where I is the identity matrix and σ is the distribution. 2 It is the variance of observation noise:

[0099]

[0100]

[0101] The set of parameters to be estimated under the current assumptions is θ = {W, μ, σ}. 2 ,α}, where μ is estimated using the following formula:

[0102]

[0103] Parameters W, σ 2 α is updated iteratively using the EM algorithm. The EM algorithm estimates the model parameters by alternately updating the E-step (expectation step) and the M-step (maximization step). The specific steps of the EM algorithm are described below:

[0104] (b3.1) Initialize parameters Q and σ 2 α;

[0105] (b3.2) In step E, the expected value and covariance of the posterior distribution of the latent variable z are calculated using the following formula:

[0106] Where M = W T W+σ 2 I (10)

[0107]

[0108] (b3.3) In step M, parameters Q and σ are updated according to the expected maximization. 2 α, and its update formula is as follows:

[0109]

[0110] Where Tr(·) represents the trace of the matrix;

[0111] (b3.4) Repeat steps E and M until the model parameters converge, i.e., the iterative change of the parameters is less than the set threshold.

[0112] S3. Construct a time-series data mapping relationship between NO2 concentrations from 11 national monitoring stations and the Jing'an Yan'an West Road traffic station. This is used to backtrack traffic station data during periods when traffic station data is missing, utilizing data from the 11 national monitoring stations. An initial time window [T-2, T+2] is selected, and a bidirectional Long Short-Term Memory (Bi-LSTM) network is trained under the initial network hyperparameters to construct the mapping relationship between the two. This mapping relationship considers the lag effect of pollution diffusion, using concentration information from other stations around the unknown data station over a past to future time window relative to the backtracking time to infer the concentration value of the unknown data station at the backtracking time.

[0113] S4. Optimize hyperparameters using a differentiated optimization strategy. The network hyperparameters adjusted include the time window size, learning rate, regularization parameter, batch size, network layers, number of neurons, and loss function. This method uses mean squared error as the loss function, dynamically adjusts the learning rate using the Adadelta optimizer, and employs L2 regularization. Other hyperparameters include the time window size, batch size, network layers, and number of neurons. The hyperparameter space is defined as follows: Gaussian process is used. The specific steps for hyperparameter optimization using a surrogate model are as follows:

[0114] (a4.1) Initialization phase, define the hyperparameter space Randomly sample the initial hyperparameter point set A = {a1, a2, ..., a...} n Based on the initial hyperparameter point set, a Bi-LSTM network model is trained, and the validation set loss y = {y1, y2, ..., y} corresponding to each set of hyperparameters is calculated. n Define a Gaussian process f(a):

[0115]

[0116] Here, μ(a) represents the mean function, i.e., E[f(a)], and k(a,a′) is the kernel function, characterizing the correlation between a and a′ in the hyperparameter space. Radial basis functions (RBF) are used here, with the initial sample set... Train the Gaussian process model f(a);

[0117] (a4.2) Calculate the predicted distribution, for candidate point a, the predicted mean μ(a) and the predicted variance σ. 2 The formula for calculating (a) is as follows:

[0118] μ(a)= k(a,A)(K(A,A)+σ 2 I) -1 y (16)

[0119] σ 2 (a)=k(a,a)-k(a,A)(K(A,A)+σ 2 I) -1 k(A,a) (17)

[0120] Where A is the matrix of evaluated points, y is the corresponding objective function value vector, K(A,A) is the covariance matrix of the kernel function on the evaluated points, k(a,A) is the covariance vector between the candidate point and the evaluated points, and k(a,a) is the covariance between the candidate point and itself.

[0121] (a4.3) Select and optimize the acquisition function, using the desired improvement (EI) as the acquisition function. Its calculation formula is as follows:

[0122]

[0123] Among them, a + The current optimal point is given by ξ, a non-negative exploration parameter used to control the trade-off between exploration and exploitation in the algorithm. Φ(·) and φ(·) are the cumulative distribution function and probability of the standard normal distribution, respectively. The algorithm finds the optimal point on the surrogate model that makes the acquisition function α... EI (a) The largest point a * , that is:

[0124]

[0125] (a4.4) Update the model for hyperparameter a * The training model yields the validation set loss y. * Add the new sampling points to the set of evaluated points to form a set {(a1,y1),(a2,y2),…,(a... n ,y n ),(a * ,y * The Gaussian process model was retrained using the updated dataset.

[0126] (a4.5) Check the stopping condition. Check if the stopping condition is met (maximum number of iterations, time limit, or objective function value reaches the expected value). If it is, stop. Otherwise, repeat (a4.2) to (a4.4).

[0127] In this example, the time window was ultimately determined to be [T-6, T+6], the batch size to be 32, the maximum number of iterations to be 200, and the network parameters are shown in Table 3 below:

[0128] Table 3 Network Parameters

[0129]

[0130]

[0131] S5. Using the PyTorch deep learning framework, construct and compile the BPCA-LSTM model. Train the model within the hyperparameter optimization framework, obtaining the iteratively optimal hyperparameter settings and corresponding network model under a preset time limit. The true vs. predicted scatter plot on the validation set of the optimal model is shown below. Figure 5 As shown. For the missing 2018 NO2 data for the Jing'an Yan'an West Road traffic station, the model was applied using time-series NO2 pollutant concentration data from 11 national monitoring stations as input to back-track the hourly NO2 data series for the Jing'an Yan'an West Road traffic station in 2018. The results are as follows. Figure 6 As shown.

[0132] The above description is merely a description of preferred embodiments of this application and is not intended to limit the scope of this application in any way. Any changes or modifications made by those skilled in the art based on the above-disclosed technical content should be considered as equivalent and valid embodiments and fall within the scope of protection of the technical solution of this application.

Claims

1. A method for retrospectively analyzing pollutant monitoring data from fixed atmospheric environmental monitoring stations, characterized in that, Includes the following steps: S1. Construct a BPCA-LSTM network model, including a Bayesian principal component analysis part and a long short-term memory network part; at the same time, based on the range of pollutant concentration monitoring data that needs to be traced back at M stations, select and form a usable complete dataset from the historical pollutant concentration monitoring data of all N stations. S2. Perform data cleaning on the selected dataset, including pollutant concentration standardization, outlier removal and missing point imputation, and apply the Bayesian principal component analysis to the cleaned data for dimensionality reduction. Among them, Bayesian principal component analysis was applied to reduce the dimensionality of the cleaned data, and the dimensionality reduction process was completed using the expectation-maximization algorithm; for the actual concentration vector at a certain moment... , It is the feature dimension. Given a vector representing the actual concentrations of all stations except those with missing data at that moment, the goal of dimensionality reduction is to find a low-dimensional representation. and the corresponding weight matrix and a bias vector , so that: in, It is a noise term; regarding , , Given the following prior distribution assumptions, where Indicates a Gaussian distribution. It is the identity matrix. It is the variance of observation noise: The set of parameters to be estimated under the current assumptions ,in The following formula is used for estimation: parameter , , The EM algorithm iteratively updates the model parameters by alternately updating the E step (expectation step) and the M step (maximization step). S3. Set the initial data sliding window and BPCA-LSTM network model hyperparameters, and use the long short-term memory network part to construct the time series pollutant concentration data mapping relationship between NM sites and each of the M sites, so as to use the data of the remaining NM sites to backtrack the data of the M sites during the period when the data of the M sites is missing. S4. Optimize and adjust the sliding window duration and the hyperparameters of the BPCA-LSTM network model; S5. Using historical data from NM stations corresponding to the period when data from station M is missing as input, the missing pollution concentration monitoring data from station M is traced back through the constructed BPCA-LSTM network model.

2. The method for retrospective analysis of pollutant monitoring data from fixed atmospheric environment monitoring stations as described in claim 1, characterized in that, In step S1, the data selected from the historical data of all N stations are uniformly the pollutant concentration monitoring data of N stations that are adjacent to the missing pollutant concentration data interval of the M stations that need to trace back the data. The relationship pattern of the concentration time series data between the N stations covered by the selected data effectively characterizes the concentration time series data mapping relationship between the NM stations and the M station within the missing data interval of the M station. Data completeness is defined by the following formula: in, Indicates the completeness of the data. This represents the total amount of data that should be monitored in the dataset. This represents the total amount of data in the dataset that was not monitored.

3. The method for retrospective analysis of pollutant monitoring data from fixed atmospheric environment monitoring stations as described in claim 1, characterized in that, In step S2, data cleaning includes pollutant concentration standardization, outlier removal, and missing data completion, as detailed below: (a3.1) For the standardization of pollutant concentration measurement, if the stored data uses concentration values ​​under standard conditions, it should be converted to concentration values ​​under actual conditions using the following conversion formula: in, This represents the concentration value under actual conditions. This represents the concentration value under standard conditions. This indicates the ambient temperature under actual conditions, and it uses the Kelvin temperature scale. (a3.2) Outlier removal refers to the removal of outlier records and data anomalies. Outliers are defined as values ​​that are less than the lower limit or greater than the upper limit. The formulas for calculating the upper and lower limits are as follows: in, and These represent the upper and lower limits, respectively. and These represent the upper quartile and the lower quartile, respectively. Indicates the interquartile range, i.e. and The distance between them; (a3.3) For missing data completion, missing data in the original data is completed with the average concentration of pollutants within the selected data range. For data removed due to outliers, if the original data was an abnormal detection record, it is completed with the average concentration mentioned above. If the data is an outlier, if it is less than the lower limit, it is completed with the minimum value of the data after removing the outlier. If it is greater than the upper limit, it is completed with the maximum value of the data after removing the outlier.

4. The method for retrospective analysis of pollutant monitoring data from fixed atmospheric environment monitoring stations as described in claim 1, characterized in that, The EM algorithm: Specific steps: (b3.1) Initialization parameters , , ; (b3.2) Calculate the latent variables in step E. The expected value and covariance of the posterior distribution are calculated using the following formulas: (b3.3) In step M, the parameters are updated according to the expected maximization. , , Its update formula is as follows: in, Represents the trace of a matrix; (b3.4) Repeat steps E and M until the model parameters converge, i.e., the iterative change of the parameters is less than the set threshold.

5. The method for retrospective analysis of pollutant monitoring data from fixed atmospheric environment monitoring stations as described in claim 1, characterized in that, In step S3: A time-series pollutant concentration data mapping relationship was constructed between NM stations and each of the M stations, so that data from the remaining NM stations can be used to backtrack the data of the M station during periods when the M station data is missing.

6. The method for retrospective analysis of pollutant monitoring data from fixed atmospheric environment monitoring stations as described in claim 5, characterized in that, In step S3: Specifically, For the current time T, a time window [Tn, T+n] is selected, where n is a hyperparameter for adjusting the window size. The pollutant concentration data within the time window of NM stations are used as input, and the concentration value of the station to be traced back at the current time T is used as output. Under the selected network hyperparameters, a long short-term memory network is used to construct the mapping relationship between the two. Using the concentration information of other stations around the unknown data station in a time window from the past to the future relative to the trace back time, the concentration value of the unknown data station at the trace back time is inferred.

7. The method for retrospective analysis of pollutant monitoring data from fixed atmospheric environment monitoring stations as described in claim 6, characterized in that, In step S3: Long Short-Term Memory (LSTM) network is a bidirectional network model that uses a bidirectional LSTM to capture and memorize the concentration patterns of time series data in both the past-to-future and future-to-past directions within a time window.

8. The method for retrospective analysis of pollutant monitoring data from fixed atmospheric environment monitoring stations as described in claim 1, characterized in that, Step S4 employs a differentiated hyperparameter optimization strategy, adjusting network hyperparameters including time window size, learning rate, regularization parameter, batch size, network layers, number of neurons, and loss function. This method uses mean squared error as the loss function, dynamically adjusts the learning rate using the Adadelta optimizer, and employs L2 regularization. Other hyperparameters include time window size, batch size, network layers, and number of neurons, and their hyperparameter space is defined as follows: The hyperparameter optimization is performed using a Gaussian process as a surrogate model. The specific steps are as follows: (a4.1) Initialization phase: Define hyperparameter space Randomly sample the initial hyperparameter point set. A Bi-LSTM network model is trained based on the initial hyperparameter point set, and the validation set loss of the training model corresponding to each set of hyperparameters is calculated. Define a Gaussian process : in, Represents the mean function, i.e. , It is a kernel function, representing the hyperparameter space. and The correlation is determined using radial basis functions and an initial sample set. Training Gaussian process model ; (a4.2) Calculate the predicted distribution for candidate points. Predicted mean and prediction variance The calculation formula is as follows: in, It is the matrix of evaluated points. It is the corresponding objective function value vector. It is the covariance matrix of the kernel function at the evaluated points. It is the covariance vector between candidate points and evaluated points. It is the covariance between the candidate point and itself; (a4.3) Select and optimize the acquisition function, using the desired improvement (EI) as the acquisition function. Its calculation formula is as follows: in, It is the best option at present. It is a non-negative exploration parameter used to control the trade-off between exploration and exploitation in the algorithm. and These are the cumulative distribution function and probability of the standard normal distribution, respectively. Find the acquisition function on the surrogate model. The largest point , that is: (a4.4) Update the model, for hyperparameters The training model obtains the validation set loss. Add new sampling points to the set of already evaluated points to form a collection. The Gaussian process model was retrained using the updated dataset. (a4.5) Check the stopping condition. If the stopping condition is met, stop; otherwise, repeat (a4.2) to (a4.4).

9. The method for retrospective analysis of pollutant monitoring data from fixed atmospheric environment monitoring stations as described in claim 2, characterized in that, In step S1, the selected data is: Data from sites with a data integrity rate of less than 90% in the NM sites was discarded.

Images