Medium and long term model forecasting method for air quality based on grid monitoring

Abstract

The invention relates to the technical field of data processing, in particular to an air quality medium and long term model forecasting method based on grid monitoring. A gridding interpolation technology is used to carry out tuning on a small grid unit, and the method mainly comprises the steps of expanding limited monitoring station data interpolation to the whole grid space, improving the spatial resolution, comparing forecast results through real-time monitoring data, and continuously carrying out model parameter optimization by using the grid unit, thereby improving the forecast precision. Compared with the prior art, the forecasting time is prolonged, medium-and-long-term air quality forecasting can be carried out, the forecasting of an existing model is mostly short-term forecasting which is generally about 3-5 days, the forecasting model can be used for forecasting the future 15 days, the forecasting time is mainly influenced by an initial meteorological field, GFS data are adopted at present, medium-and-long-term forecasting can be carried out, and the forecasting efficiency is greatly improved. And the timeliness of forecasting is effectively improved.

Description

Technical Field

[0001] The present invention relates to the field of data processing technology, and specifically to a medium- and long-term model forecasting method for air quality based on grid monitoring. The method is applied to the fields of meteorology, atmospheric science, atmospheric chemistry, aerodynamics and geographic information science (GIS) to provide scientific support for solving air pollution problems. Background Art

[0002] As the living environment of human beings continues to deteriorate, air quality has become an important issue in the field of public health and environmental protection. Therefore, developing a method that can accurately predict air quality is of great significance to improving public health and environmental quality.

[0003] Commonly used air quality forecasting methods are mainly based on statistical models and machine learning models. These methods mainly use historical air quality data for statistical analysis and then make predictions, but due to the lack of actual monitoring for continuous correction and continuous machine learning, the accuracy of the predictions is greatly limited. Summary of the invention

[0004] The present invention provides a method for a medium- and long-term air quality forecast model based on grid monitoring, which can be used for long-term air quality forecasting. The model uses meteorological field data GFS, biological sources, and anthropogenic sources, combined with sciences such as aerodynamics and atmospheric science, to study the complex relationship between the air quality index, and uses a deep learning algorithm to perform bidirectional coupling of meteorological and air quality data to establish a high-precision, stable and reliable forecast model, and combines the real-time monitoring data of national control stations, provincial control stations, and micro stations to establish a grid to correct the parameters of the forecast model in real time, so that the forecast model is more and more accurate.

[0005] The present invention solves the technical problem by adopting the following technical solution: S1. Data preprocessing: (1) Data acquisition: Acquisition of GFS data for initial meteorological fields, biological sources, anthropogenic sources, and air quality monitoring data; (2) Data cleaning: interpolation of missing values ​​and correction or deletion of outliers; Missing values ​​are handled using linear interpolation: in, and is two adjacent monitoring times, and is the corresponding value, is the time point that needs to be interpolated; Minimum-maximum data normalization method: compare and merge data of different scales and ranges to perform data normalization; in, It is the value between 0 and 1 after the current value is normalized. and is the minimum and maximum value in the current set of data, is the original value; S2. Model establishment: Model data preparation: load meteorological fields, biological sources, and human sources into a Pandas DataFrame and clean it. Extract meteorological fields, biological sources, human sources, and real-time monitoring feature data according to daily average data, analyze the relevant time features between several types of data in different time periods, and use recurrent neural networks (RNNs) to extract the time series features of the data: Input layer: Serialize the output of CNN into (batch_size, time_steps, features); RNN layer: Use LSTM or GRU units to process sequence data; Repeated stacking: Stack multiple RNN layers to extract the time series features of meteorological fields, biological sources, human sources, and real-time monitoring; Based on the longitude and latitude of different stations, the spatial correlation features between meteorological fields, biological sources, human sources, and real-time monitoring feature data are extracted, and the convolutional neural network CNN is used to extract the spatial features of the data: Input layer: receives normalized data tensors with a shape of (batch_size, height, width, channels); Convolution layer: uses multiple convolution layers to extract spatial features; Pooling layer: uses a pooling layer after the convolution layer to reduce the dimension of the data; Repeated combination: repeats the combination of convolution layer and pooling layer, gradually increasing the number of filters to extract the spatial features of meteorological fields, biological sources, anthropogenic sources, and real-time monitoring; Meteorological fields, biological sources, human sources, and real-time monitoring feature data are used to establish a matrix array, and the fully connected network Dense is used to integrate the features extracted by CNN and RNN for final classification or regression. The specific implementation is as follows: Flatten layer: flattens the output of the RNN layer; Fully connected layer: uses multiple fully connected layers to integrate features; Output layer: selects the output layer according to the task type; The model parameters are corrected according to the real-time monitoring data. The parameter correction formula is: = +α· Loss in, is the updated parameter, is the previous parameter, α is the learning rate, Loss is the gradient of the loss function with respect to the parameters; S3. Model training: (1) Model training: Use preprocessed data to train the model. During the training process, the model gradually adjusts parameters according to the given loss function and optimization algorithm to minimize the gap between the predicted value and the actual label; (2) Model evaluation: Use the mean absolute error (MAE) method combined with air quality monitoring data to evaluate accuracy; (3) Grid optimization: Establish an interpolation grid based on the measured air quality data, perform comparative optimization according to the grid cells, and continuously adjust the parameters; (4) Model validation: After model training and tuning, the model is compared with the actual observations.

[0006] Beneficial effects of the invention: 1. Use grid interpolation technology to perform optimization on small grid units, mainly to expand the data interpolation of limited monitoring stations (national control, provincial air conditioning, micro stations) to the entire grid space, improve spatial resolution, compare forecast results with real-time monitoring data, and use grid units to continuously optimize model parameters, thereby improving forecast accuracy.

[0007] 2. It is superior to previous forecasts in that it can improve the forecast time and make medium- and long-term air quality forecasts. The forecasts of existing models are mostly short-term forecasts, generally about 3-5 days. This forecast model can predict the forecast for the next 15 days. The forecast time is mainly affected by the initial meteorological field. At present, we use GFS data to make medium- and long-term forecasts, which effectively improves the timeliness of the forecast.

[0008] 3. Combine multiple related data, consider multi-dimensional influencing factors, improve forecast accuracy, combine meteorological observation data, air quality monitoring data, GFS data, biological sources, anthropogenic sources and other data that affect forecast accuracy, and analyze them through different dimensions, both as model driving data and as verification data, providing richer information input for the forecast model, effectively ensuring the accuracy and comprehensiveness of the forecast.

[0009] 4. Use intelligent means to realize the visualization of forecast results. Combining Python with GIS technology, with the help of powerful Pandas, NumPy, and Matplotlib libraries, data processing, spatial analysis, and visualization can be realized, and spatial distribution maps and change trends can be dynamically generated. BRIEF DESCRIPTION OF THE DRAWINGS

[0010] Figure 1 This is a schematic diagram of the forecast results for the next 15 days; Figure 2 This is a comparison chart of the AQI forecast and actual measurement for the next 15 days; Figure 3 A comparison chart of NO2 reports and actual measurements for the next 15 days; Figure 4 For PM 10 Schematic diagram of pollution forecast results; Figure 5 For PM 2.5 Schematic diagram of pollution forecast results; Figure 6 This is a schematic diagram of SO2 pollution forecast results; Figure 7 This is a schematic diagram of O3 pollution forecast results; Figure 8 Provide grid tuning diagrams for national, provincial and micro-station monitoring; Fig. 9 Schematic diagram of model composition; Fig.10 This is a flow chart of the forecasting method. DETAILED DESCRIPTION

[0011] Further explanation with reference to the embodiments 1. Data preprocessing: Collect the initial meteorological field (such as GFS data), biological sources, anthropogenic sources, and air quality monitoring data of Lanzhou City in the past five years and clean the data, remove abnormal data and missing values, and normalize the data.

[0012] (1) Data acquisition: Acquire data for initial meteorological fields (such as GFS data), biogenic sources, anthropogenic sources, and air quality monitoring. These data are acquired in real time from meteorological stations and air quality monitoring stations. Biogenic and anthropogenic sources are mainly acquired through the pollution source emission inventories of previous years.

[0013] (2) Data cleaning: The original data obtained may contain missing values, outliers, and other problems, which require data cleaning. Missing values ​​need to be interpolated, outliers need to be corrected or deleted, and data needs to be smoothed to reduce the impact of errors.

[0014] The GFS meteorological field data, emission data from biological and anthropogenic sources, and real-time monitoring data from national control stations, provincial control stations, and micro stations need to be integrated into a unified format, and the influence of outliers and missing values ​​must be eliminated. Missing values ​​should be processed using linear interpolation.

[0015] Point monitoring abnormalities and missing data processing Regarding the above-mentioned "Lanlian Hotel" location on 2024 / 6 / 20 First, we remove the outliers and set them as " / " missing values. Then, for 2024 / 6 / 16 and June 20, 2024 The missing values ​​are interpolated using linear interpolation. The specific process is as follows.

[0016] l Linear interpolation method: in, and is two adjacent monitoring times, and is the corresponding value, is the time point that needs to be interpolated.

[0017] = = = 32 μg / m³ = = = 33 μg / m³ (3) Minimum-maximum data normalization method: In order to compare and merge data of different scales and ranges, data normalization is often required. The normalization algorithm is used to scale the data to between 0 and 1.

[0018] l Minimum-maximum data normalization method: in, It is the value between 0 and 1 after the current value is normalized. and is the minimum and maximum value in the current set of data, is the original value.

[0019] According to the above data, the six indicators of the site are normalized as follows: ① Original data: = [27,27,31,26,30,44,33,22,27,23,26] = [40,51,42,38,56,74,33,38,55,46,62] = [0.4,0.5,0.3,0.4,0.3,0.5,0.3,0.4,0.3,0.4,0.4] = [25,27,27,29,13,25,16,23,31,24,31] = [11,31,23,15,14,27,10,9,16,18,24] = [176,199,207,205,190,191,112,171,199,157,197] ② Find the maximum and minimum values ​​of each indicator: = 44 = 22 = 74 = 33 = 0.5 = 0.3 = 31 = 13 = 31 = 9 = 207 = 112 ③ According to the minimum-maximum data normalization method, the normalized index array is obtained: = [0.259,0.259,0.324,0.185,0.222,1,0.259,0.111,0.259,0.148,0.185] = [0.061,0.242,0.121,0.061,0.303,1.0,0,0.061,0.272,0.242,0.379] = [0.5,1.0,0.0,0.5,0.0,1.0,0.0,0.5,0.0,0.5,0.5] = [0.462,0.539,0.539,0.615,0,0.462,0.308,0.462,0.692,0.461,0.692] = [0.375,1,0.562,0.438,0.375,0.688,0.25,0.5,0.562,0.625,0.625] = [0.52,0.68,0.76,0.72,0.64,0.65,0.24,0.59,0.68,0.55,0.67] The calculation method is implemented in Python as follows: def min_max normalization(data): min_val = min(data) max_val = max(data) normalized_data=[(x-min_val) / (max_val-min_val)for x indata] return normalized_data 2. Model establishment: Based on the collected meteorological fields (such as GFS data), biological sources, anthropogenic sources, and historical air quality data, a high-precision, stable and reliable deep learning model is established. The model is implemented using Python and TensorFlow libraries, combining convolutional neural networks (CNN), recurrent neural networks (RNN), and fully connected networks (Dense) to effectively capture the spatiotemporal correlation between data.

[0020] Specific implementation process: ① Model data preparation, prepare and preprocess data. Load meteorological fields, biological sources, and human sources into a Pandas DataFrame and clean them. Some code implementation is as follows: import pandas as pd import numpy as np # df is the DataFrame containing all the data # The data includes meteorological data, biological sources, anthropogenic sources and historical air quality data # Select relevant feature columns features = df[['temperature', 'humidity', 'wind_speed', 'pressure', 'biomass', 'anthropogenic']] # Standardize data from sklearn.preprocessing import StandardScaler scaler = StandardScaler() scaled_features = scaler.fit_transform(features) # Processing time series data, you need to convert it into a supervised learning problem def create_dataset(dataset, look_back=1): X, Y = [], [] for i in range(len(dataset) - look_back): a = dataset[i:(i + look_back), :] X.append(a) Y.append(dataset[i + look_back, 0]) return np.array(X), np.array(Y) # Set the time window look_back = 1 X, Y = create_dataset(scaled_features, look_back) # Divide the dataset from sklearn.model_selection import train_test_split X_train, X_test, y_train, y_test = train_test_split(X, Y, test_size=0.2, random_state=42) ②According to the daily average data, extract the meteorological field, biological source, human source, and real-time monitoring feature data, analyze the relevant time characteristics between several types of data in different time periods, and use the recurrent neural network (RNN) to extract the time series characteristics of the data. The following is the detailed design of the recurrent neural network (RNN): l Input layer: Serialize the output of CNN into a shape of (batch_size, time_steps, features).

[0021] l RNN layer: Use LSTM or GRU units to process sequence data.

[0022] l Repetition: Multiple RNN layers can be stacked to extract temporal features of meteorological fields, biological sources, anthropogenic sources, and real-time monitoring.

[0023] ③ Based on the longitude and latitude of different stations, the spatial correlation features between meteorological fields, biological sources, anthropogenic sources, and real-time monitoring feature data are extracted, and the spatial features of the data are extracted using a convolutional neural network (CNN). The following is the detailed design of CNN: l Input layer: receives the normalized data tensor with a shape of (batch_size, height, width, channels).

[0024] l Convolutional layer: Use multiple convolutional layers to extract spatial features.

[0025] Pooling layer: Pooling layer is used after convolutional layer to reduce the dimensionality of data.

[0026] l Repetition: Repeat the combination of convolutional layers and pooling layers, gradually increasing the number of filters to extract the spatial features of meteorological fields, biological sources, anthropogenic sources, and real-time monitoring.

[0027] ④ Establish a matrix array of meteorological field, biological source, human source, and real-time monitoring feature data, use a fully connected network (Dense) to integrate the features extracted by CNN and RNN, and perform the final classification or regression. The specific implementation is as follows: l Flatten layer: Flattens the output of the RNN layer.

[0028] l Fully connected layer: Use multiple fully connected layers to integrate features.

[0029] l Output layer: Select the output layer according to the task type.

[0030] ⑤ Some code implementations of the convolutional neural network (CNN), recurrent neural network (RNN) and fully connected network (Dense) layers are as follows: import tensorflow as tf from tensorflow.keras.models import Sequential from tensorflow.keras.layers import LSTM, Dense, Conv2D,MaxPooling2D, Flatten # Define the model model = Sequential() # The shape of X_train is (samples, time_steps, features) # Add a dimension to fit the Conv2D layer X_train_reshaped = X_train.reshape((X_train.shape[0], X_train.shape[1], X_train.shape[2], 1)) # CNN layers model.add(Conv2D(filters=64, kernel_size=(2, 1), activation='relu',input_shape=(look_back, X_train.shape[2], 1))) model.add(MaxPooling2D(pool_size=(2, 1))) model.add(Flatten()) # RNN Layer model.add(LSTM(50, activation='relu', return_sequences=True)) model.add(LSTM(50, activation='relu')) # Fully connected layer model.add(Dense(100, activation='relu')) model.add(Dense(1)) The model parameters are corrected according to the real-time monitoring data. The parameter correction formula is: = +α· Loss in, is the updated parameter, is the previous parameter, α is the learning rate, Loss is the gradient of the loss function with respect to the parameters.

[0031] #Evaluate the model test_data is the validation sample data Loss_old, accuracy = model.evaluate(test_data) 3. Model training: Based on feature vectors, deep learning algorithms are used to establish forecast models for initial meteorological fields (such as GFS data), biological sources, anthropogenic sources, and air quality monitoring data, and they are trained and optimized through a large number of experiments.

[0032] (1) Model training: Use preprocessed data to train the model. During the training process, the model gradually adjusts parameters according to the given loss function and optimization algorithm to minimize the gap between the predicted value and the actual label.

[0033] (2) Model evaluation: The mean absolute error (MAE) method was used in combination with air quality monitoring data to evaluate the accuracy.

[0034] (3) Grid tuning: In order to improve the accuracy of the model, the model parameters need to be tuned. The interpolation grid is established through the measured air quality data, and the comparison and tuning are carried out according to the grid units, and the parameters are continuously adjusted.

[0035] (4) Model validation: After model training and tuning, the model is compared with actual observations to evaluate its performance and accuracy.

[0036] Some code implementations are as follows: # Compile the model model.compile(optimizer='adam', loss='mean_squared_error') Model creation, compilation, and training from keras.models import Sequential from keras.layers import LSTM, Dense # Create LSTM model model = Sequential() model.add(LSTM(32, input_shape=(timesteps, features))) model.add(Dense(1, activation='sigmoid')) # Compile the model model.compile(optimizer='adam',loss='binary_crossentropy',metrics=['accuracy']) # Train the model model.fit(train_data,train_labels,epochs=10,batch_size=64,validation_data=(val_data, val_labels)) 4. Forecast output: The model can predict the air quality indicators in the next 15 days: , CO and other pollutants. These prediction results can be used for urban air quality management, health risk assessment, pollutant control strategy formulation and other applications.

[0037] 5. Experimental results: In order to verify the prediction effect of the forecast model of the present invention, a large number of experiments were conducted. The experimental data include meteorological fields (such as GFS data), biological sources, anthropogenic sources, and real-time air quality monitoring data, totaling 642,540 data. Among them, 602,540 data are used to train the model, 600,000 data are used to verify the model, and 42,540 data are used to test the model.

[0038] The experimental results show that the forecast model of the present invention can accurately predict the change trend and trend degree of the air quality index in the next 15 days. The mean absolute error (MAE) on the test set is 10, and the mean relative error (MRE) is 5%, with high prediction accuracy. At the same time, the forecast model of the present invention has good generalization ability.

[0039] Air quality forecast: Air quality forecast can be made based on a variety of meteorological and environmental parameters, such as meteorological observation data, air quality monitoring data, GFS data, biological sources, anthropogenic sources and other data.

[0040] Accuracy: The accuracy of air quality forecasts can be improved through the use of reasonable mathematical models and algorithms, combined with a large amount of observational data and simulation data.

[0041] Air quality management and decision support: The patented application of this model method can provide valuable information and support for air quality management and decision-making. The forecast results can be used to formulate and optimize environmental policies, plan air pollution prevention and control measures, adjust industrial layout, etc., to improve and protect air quality.

[0042] Risk assessment and emergency response: Methods based on air quality forecast models can help conduct air pollution risk assessments and provide timely warnings and responses to air quality emergencies.

[0043] Data analysis and trend forecasting: By analyzing historical air quality data and real-time monitoring data, this model method can identify and analyze the changing trends of air quality and make future trend forecasts.

[0044] Possible application scenarios: Urban air quality management: This model method can be applied to urban air quality management systems to help cities monitor and predict the concentration and distribution of air pollutants. It can help urban decision makers develop air quality improvement strategies, optimize traffic management, reduce emission sources, etc. to improve urban air quality.

[0045] Industrial enterprise environmental monitoring: Large industrial enterprises usually monitor and evaluate the impact of their emission sources. This model method can be applied to the environmental monitoring system of industrial enterprises to provide accurate air quality forecasts and pollutant dispersion simulations, helping enterprises to comply with regulations and reduce environmental impacts.

[0046] Architecture and urban planning: In the process of architecture and urban planning, it is very important to consider the impact of air quality on the health and comfort of residents. This model method can be applied to architecture and urban planning software to evaluate and optimize building design, greening layout, road planning, etc. to improve the air quality of indoor and urban environments.

[0047] Glossary GFS: Global Forecast System (GFS) is a global numerical weather forecast model of the National Center for Environmental Prediction (NCEP) of the United States. GFS forecast data is widely used in numerical weather forecasting and is one of the important global meteorological data.

[0048] Biogenic sources: Biogenic sources refer to air pollutants produced by biological processes in nature. These pollutants mainly come from the natural metabolic processes of plants, microorganisms and animals.

[0049] Anthropogenic sources: Anthropogenic sources refer to air pollution emissions caused by human activities. These emissions include pollutants generated by industrial production, transportation, energy consumption, agricultural activities, urbanization, and waste disposal.

[0050] GIS: Abbreviation for Geographic Information System. It is a specific information system used to capture, store, inspect, manipulate, analyze and display all types of data related to geographic location. GIS technology can combine various spatial information (such as maps, satellite images, aerial photos) with attribute information (such as demographic data, transportation network information, land use) to help users better understand and interpret spatial data.

[0051] Medium-Range Forecast: Medium-Range Forecast of air quality usually refers to the forecast of meteorological conditions for the next 3 to 14 days. This time range is between short-term forecast (usually 24 hours to 3 days) and long-term forecast (such as seasonal or annual climate forecast).

Claims

1. A medium- and long-term air quality model forecasting method based on grid monitoring, characterized by The steps include: S1. Data preprocessing: (1) Data acquisition: Acquisition of GFS data for initial meteorological fields, biological sources, anthropogenic sources, and air quality monitoring data; (2) Data cleaning: interpolation of missing values ​​and correction or deletion of outliers; Missing values ​​are handled using linear interpolation: in, and is two adjacent monitoring times, and is the corresponding value, is the time point that needs to be interpolated; Minimum-maximum data normalization method: compare and merge data of different scales and ranges to perform data normalization; in, It is the value between 0 and 1 after the current value is normalized. and is the minimum and maximum value in the current set of data, is the original value; S2. Model establishment: Model data preparation: load meteorological fields, biological sources, and human sources into a Pandas DataFrame and clean it. Extract meteorological fields, biological sources, human sources, and real-time monitoring feature data according to daily average data, analyze the relevant time features between several types of data in different time periods, and use recurrent neural networks (RNNs) to extract the time series features of the data: Input layer: Serialize the output of CNN into (batch_size, time_steps, features); RNN layer: Use LSTM or GRU units to process sequence data; Repeated stacking: Stack multiple RNN layers to extract the time series features of meteorological fields, biological sources, human sources, and real-time monitoring; Based on the longitude and latitude of different stations, the spatial correlation features between meteorological fields, biological sources, human sources, and real-time monitoring feature data are extracted, and the convolutional neural network CNN is used to extract the spatial features of the data: Input layer: receives normalized data tensors with a shape of (batch_size, height, width, channels); Convolution layer: uses multiple convolution layers to extract spatial features; Pooling layer: uses a pooling layer after the convolution layer to reduce the dimension of the data; Repeated combination: repeats the combination of convolution layer and pooling layer, gradually increasing the number of filters to extract the spatial features of meteorological fields, biological sources, anthropogenic sources, and real-time monitoring; Meteorological fields, biological sources, human sources, and real-time monitoring feature data are used to establish a matrix array, and the fully connected network Dense is used to integrate the features extracted by CNN and RNN for final classification or regression. The specific implementation is as follows: Flatten layer: flattens the output of the RNN layer; Fully connected layer: uses multiple fully connected layers to integrate features; Output layer: selects the output layer according to the task type; The model parameters are corrected according to the real-time monitoring data. The parameter correction formula is: = + α· Loss in, is the updated parameter, is the previous parameter, α is the learning rate, Loss is the gradient of the loss function with respect to the parameters; S3. Model training: (1) Model training: Use preprocessed data to train the model. During the training process, the model gradually adjusts parameters according to the given loss function and optimization algorithm to minimize the gap between the predicted value and the actual label; (2) Model evaluation: Use the mean absolute error (MAE) method combined with air quality monitoring data to evaluate accuracy; (3) Grid optimization: Establish an interpolation grid based on the measured air quality data, perform comparative optimization according to the grid cells, and continuously adjust the parameters; (4) Model validation: After model training and tuning, the model is compared with the actual observations.

Images