Air quality multivariate prediction method based on Attention-LSTM

Abstract

The invention relates to an Attention-LSTM-based air quality multivariate prediction method. The method comprises the steps of obtaining historical data of an air quality AQI and performing preprocessing; performing feature analysis on the data in the data set, and constructing an LSTM network model according to the analyzed features; the LSTM network model is improved; the improved LSTM network model is trained; and inputting to-be-predicted air quality data into the trained model to obtain an air quality predicted value, evaluating future air quality, and selecting a corresponding early warning mechanism. According to the method, error analysis is carried out on the air quality predicted value # imgabs0 # and the actual air quality value y, the accuracy of the model can be evaluated, if the error is small, it shows that the model has high prediction capacity, and if the error is large, the model needs to be further optimized, model parameters are adjusted in a targeted mode, and the accuracy of the model is improved. When factors such as different geographic positions and different weather conditions are considered, the weight needs to be adjusted.

Description

Technical Field

[0001] The present invention relates to the technical field of air quality forecasting, and in particular to an air quality multivariate prediction method based on Attention-LSTM. Background Art

[0002] Traditional air quality forecasts are mainly based on empirical formulas and statistical models, which can only roughly forecast air quality. However, they do not take into account factors such as complex atmospheric chemical reaction processes and pollutant diffusion laws. The forecast accuracy is very limited, and it is impossible to predict the source of pollution and the spread of pollution.

[0003] In recent years, some scholars have also used deep learning to conduct research on air quality detection. This method mainly constructs and trains deep neural networks, which can accurately predict air quality in the future. Compared with traditional monitoring methods, this method has the advantages of high accuracy, good real-time performance, and wide coverage. At present, scholars use deep learning methods to predict air quality, mainly using models such as convolutional neural networks and recurrent neural networks, combined with multiple data sources such as government monitoring stations, satellite remote sensing, and meteorological data to capture the spatial and temporal characteristics of air quality. However, these methods have some defects, including data quality and availability issues, such as missing data and noise, overfitting risks caused by model complexity, and poor model interpretability. In addition, air quality is affected by many external factors, and a single model may not be able to fully capture these complex relationships. At the same time, deep learning models have high requirements for computing resources, limited real-time prediction capabilities, and feature differences in different regions make model transfer learning challenging. These problems suggest that researchers need to comprehensively consider data diversity, model optimization, and external environmental changes when predicting air quality. Summary of the invention

[0004] In order to solve the problems of low calculation accuracy and complex calculation in the prior art, the purpose of the present invention is to provide an air quality multivariate prediction method based on Attention-LSTM.

[0005] To achieve the above object, the present invention adopts the following technical solution: a multivariate prediction method of air quality based on Attention-LSTM, comprising:

[0006] (1) Obtain historical data of air quality AQI and perform preprocessing on it. The preprocessed historical data form a data set, and the data set is divided into a training set, a validation set, and a test set;

[0007] (2) Perform feature analysis on the data in the dataset to obtain the analyzed features, including seasonality, trend, and periodicity, and build an LSTM network model based on the analyzed features;

[0008] (3) Improving the LSTM network model to obtain an improved LSTM network model;

[0009] (4) Use the training set to train the improved LSTM network model to obtain a trained model;

[0010] (5) Input the air quality data to be predicted into the trained model to obtain the air quality prediction value, evaluate the future air quality, and select the corresponding early warning mechanism.

[0011] In step (1), the preprocessing includes filling missing values ​​with a moving average method, processing outliers, normalization and standardization, and dividing the data set into a training set, a validation set, and a test set in a ratio of 8:1:1.

[0012] The step (3) specifically refers to: adding an attention mechanism module on the top of the LSTM network model to obtain an improved LSTM network model, wherein the improved LSTM network model includes an input layer, an LSTM layer, an Attention layer, a weighted summation module, a merging layer and an output layer;

[0013] The input layer is used to receive input sequence data;

[0014] The LSTM layer is used to process input sequence data and capture time dependency; the LSTM layer is composed of multiple LSTM units, each of which includes an input gate, a forget gate, and an output gate, wherein:

[0015] The input gate is used to determine how much of the current input information is written into the cell state;

[0016] The forget gate is used to decide how much information to discard from the LSTM cell state;

[0017] The output gate is used to determine how much information of the current cell state is output to the next time step;

[0018] The Attention layer is used to calculate the similarity between the output of each LSTM unit and the current query Query; convert the similarity into weights through the Softmax function to ensure that the sum of all weights is 1; and output a set of attention weights to indicate the importance of each time step;

[0019] The weighted summation module uses the attention weights to perform weighted summation on the output of the LSTM layer to generate a context vector;

[0020] The merging layer merges the output of the LSTM layer with the context vector by concatenation or addition, and the merged vector contains the state information and context information of the LSTM;

[0021] The output layer passes the merged result to the fully connected layer, and the fully connected layer maps the merged vector to the target output space to generate an air quality prediction value.

[0022] The step (4) specifically comprises the following steps:

[0023] (4a) Model training and optimization: During the training process, the gradient of the loss function with respect to the model weights is calculated through the back-propagation algorithm. The optimizer uses this gradient information to update the weights in the network to reduce the difference between the predicted value and the true value. The goal of the optimizer is to gradually adjust the weights to make the value of the loss function as small as possible.

[0024] (4b) Model evaluation: Use the validation set for evaluation, which is used to evaluate the performance of the model during the training process, and use the test set for final evaluation, which is used to evaluate the generalization ability of the model;

[0025] (4c) Hyperparameter adjustment: The learning rate is set to 0.01 to 0.05 and the batch size is set to 50.

[0026] In step (5), the early warning mechanism refers to:

[0027] When the air quality forecast When it is less than or equal to the second threshold, it means that the air quality will be normal in the future and no warning signal will be issued;

[0028] When the second threshold value is less than the air quality prediction value When the value is less than or equal to the first threshold, it indicates that the air quality will be in the second-level pollution state in the future, and a second-level warning signal will be issued;

[0029] When the first threshold value is less than the air quality prediction value When the air quality is in the first-level pollution state, it indicates that the air quality will be in the first-level pollution state in the future, and a first-level warning signal will be issued;

[0030] The first threshold is 150, and the second threshold is 100.

[0031] It can be seen from the above technical solutions that the beneficial effects of the present invention are: first, the LSTM network model in the present invention can effectively capture the long-term and short-term dependencies in time series data, and is suitable for processing the time series characteristics of air quality; at the same time, the Attention mechanism enables the model to dynamically focus on the features that have a greater impact on the prediction results, thereby improving the prediction accuracy; second, the present invention can simultaneously process a variety of input data, such as meteorological and traffic flow data, which enhances the comprehensiveness and adaptability of the model. Through the Attention mechanism, the impact of features on the prediction results can be more clearly understood, the interpretability of the model can be improved, and more accurate and interpretable air quality prediction results can be provided; third, the present invention can provide more accurate and interpretable air quality prediction results by combining air quality prediction values By performing an error analysis with the actual air quality value y, the accuracy of the model can be evaluated. If the error is small, it means that the model has a high predictive ability. If the error is large, the model needs to be further optimized and the model parameters need to be adjusted in a targeted manner. When considering factors such as different geographical locations and different weather conditions, the weights need to be adjusted. Fourth, the present invention applies the Attention-LSTM air quality prediction model that has undergone error analysis, that is, the improved LSTM network model, to predict and evaluate future air quality conditions, and promptly issue relevant early warning information, thereby achieving dynamic supervision of pollution sources. When it is found that certain pollution sources may affect the air quality, timely measures can be taken to restrict or control them. It can also predict possible heavy pollution weather in advance, thereby strengthening emergency response capabilities, taking corresponding measures before the arrival of heavy pollution weather to reduce pollutant emissions and reduce the degree of air pollution, reminding the public to pay attention to air quality conditions, and guiding the public to take corresponding protective measures to reduce exposure to pollutants and protect their health. BRIEF DESCRIPTION OF THE DRAWINGS

[0032] Figure 1 is a flow chart of the method of the present invention;

[0033] Figure 2 It is a schematic diagram of the structure of the Attention layer in the present invention;

[0034] Figure 3 It is a structural diagram of the LSTM network model in the present invention. DETAILED DESCRIPTION

[0035] like Figure 1 As shown in the figure, a multivariate prediction method for air quality based on Attention-LSTM includes:

[0036] (1) Obtain historical data of air quality AQI and perform preprocessing on it. The preprocessed historical data form a data set, and the data set is divided into a training set, a validation set, and a test set;

[0037] (2) Perform feature analysis on the data in the dataset to obtain the analyzed features, including seasonality, trend, and periodicity, and build an LSTM network model based on the analyzed features;

[0038] (3) Improving the LSTM network model to obtain an improved LSTM network model;

[0039] (4) Use the training set to train the improved LSTM network model to obtain a trained model;

[0040] (5) Input the air quality data to be predicted into the trained model to obtain the air quality prediction value, evaluate the future air quality, and select the corresponding early warning mechanism.

[0041] In step (1), the preprocessing includes filling missing values ​​with a moving average method, processing outliers, normalization and standardization, and dividing the data set into a training set, a validation set, and a test set in a ratio of 8:1:1.

[0042] The step (3) specifically refers to: adding an attention mechanism module on the top of the LSTM network model to obtain an improved LSTM network model, wherein the improved LSTM network model includes an input layer, an LSTM layer, an Attention layer (i.e., an attention mechanism module), a weighted summation module, a merging layer, and an output layer;

[0043] The input layer is used to receive input sequence data;

[0044] like Figure 3 As shown, the LSTM layer is used to process input sequence data and capture time dependency; the LSTM layer is composed of multiple LSTM units, each of which includes an input gate, a forget gate, and an output gate, where:

[0045] The input gate is used to determine how much of the current input information is written into the cell state;

[0046] The forget gate is used to decide how much information to discard from the LSTM cell state;

[0047] The output gate is used to determine how much information of the current cell state is output to the next time step;

[0048] like Figure 2 As shown, the Attention layer is used to calculate the similarity between the output of each LSTM unit and the current query Query; the similarity is converted into weights through the Softmax function to ensure that the sum of all weights is 1; and a set of attention weights is output to indicate the importance of each time step;

[0049] The weighted summation module uses the attention weights to perform weighted summation on the output of the LSTM layer to generate a context vector;

[0050] The merging layer merges the output of the LSTM layer with the context vector by concatenation or addition, and the merged vector contains the state information and context information of the LSTM;

[0051] The output layer passes the merged result to the fully connected layer, and the fully connected layer maps the merged vector to the target output space to generate an air quality prediction value.

[0052] The step (4) specifically comprises the following steps:

[0053] (4a) Model training and optimization: During the training process, the gradient of the loss function with respect to the model weights is calculated through the back-propagation algorithm. The optimizer uses this gradient information to update the weights in the network to reduce the difference between the predicted value and the true value. The goal of the optimizer is to gradually adjust the weights to make the value of the loss function as small as possible.

[0054] (4b) Model evaluation: Use the validation set for evaluation, which is used to evaluate the performance of the model during the training process, and use the test set for final evaluation, which is used to evaluate the generalization ability of the model;

[0055] (4c) Hyperparameter adjustment: The learning rate is set to 0.01 to 0.05 and the batch size is set to 50.

[0056] In step (5), the early warning mechanism refers to:

[0057] When the air quality forecast When it is less than or equal to the second threshold, it means that the air quality will be normal in the future and no warning signal will be issued;

[0058] When the second threshold value is less than the air quality prediction value When the value is less than or equal to the first threshold, it indicates that the air quality will be in the second-level pollution state in the future, and a second-level warning signal will be issued;

[0059] When the first threshold value is less than the air quality prediction value When the air quality is in the first-level pollution state, it indicates that the air quality will be in the first-level pollution state in the future, and a first-level warning signal will be issued;

[0060] The first threshold is 150, and the second threshold is 100.

[0061] In summary, the present invention can process multiple input data at the same time, such as meteorological and traffic flow data, enhance the comprehensiveness and adaptability of the model, and through the Attention mechanism, the influence of features on the prediction results can be more clearly understood, the interpretability of the model can be improved, and more accurate and interpretable air quality prediction results can be provided; the present invention can process air quality prediction values ​​by By performing an error analysis with the actual air quality value y, the accuracy of the model can be evaluated. If the error is small, it means that the model has a high predictive ability. If the error is large, the model needs to be further optimized and the model parameters need to be adjusted in a targeted manner. When considering factors such as different geographical locations and different weather conditions, the weights need to be adjusted. The present invention applies the Attention-LSTM air quality prediction model that has undergone error analysis, that is, the improved LSTM network model, to predict and evaluate future air quality conditions, and promptly issues relevant early warning information, thereby achieving dynamic supervision of pollution sources. When it is found that certain pollution sources may affect the air quality, timely measures can be taken to restrict or control them. It can also predict possible heavy pollution weather in advance, thereby strengthening emergency response capabilities, taking corresponding measures before the arrival of heavy pollution weather to reduce pollutant emissions and reduce the degree of air pollution, reminding the public to pay attention to air quality conditions, and guiding the public to take corresponding protective measures to reduce exposure to pollutants and protect their health.

Claims

1. A multivariate prediction method for air quality based on Attention-LSTM, characterized by: The method comprises the following steps in order: (1) Obtain historical data of air quality AQI and perform preprocessing on it. The preprocessed historical data form a data set, and the data set is divided into a training set, a validation set, and a test set; (2) Perform feature analysis on the data in the dataset to obtain the analyzed features, including seasonality, trend, and periodicity, and build an LSTM network model based on the analyzed features; (3) Improving the LSTM network model to obtain an improved LSTM network model; (4) Use the training set to train the improved LSTM network model to obtain a trained model; (5) Input the air quality data to be predicted into the trained model to obtain the air quality prediction value, evaluate the future air quality, and select the corresponding early warning mechanism.

2. The air quality multivariate prediction method based on Attention-LSTM according to claim 1 is characterized in that: In step (1), the preprocessing includes filling missing values ​​with a moving average method, processing outliers, normalization and standardization, and dividing the data set into a training set, a validation set, and a test set in a ratio of 8:1:

1.

3. The air quality multivariate prediction method based on Attention-LSTM according to claim 1 is characterized in that: The step (3) specifically refers to: adding an attention mechanism module on the top of the LSTM network model to obtain an improved LSTM network model, wherein the improved LSTM network model includes an input layer, an LSTM layer, an Attention layer, a weighted summation module, a merging layer and an output layer; The input layer is used to receive input sequence data; The LSTM layer is used to process the input sequence data and capture the time dependency; The LSTM layer consists of multiple LSTM units, each of which includes an input gate, a forget gate, and an output gate. The input gate is used to determine how much of the current input information is written into the cell state; The forget gate is used to decide how much information to discard from the LSTM cell state; The output gate is used to determine how much information of the current cell state is output to the next time step; The Attention layer is used to calculate the similarity between the output of each LSTM unit and the current query Query; convert the similarity into weights through the Softmax function to ensure that the sum of all weights is 1; and output a set of attention weights to indicate the importance of each time step; The weighted summation module uses the attention weights to perform weighted summation on the output of the LSTM layer to generate a context vector; The merging layer merges the output of the LSTM layer with the context vector by concatenation or addition, and the merged vector contains the state information and context information of the LSTM; The output layer passes the merged result to the fully connected layer, and the fully connected layer maps the merged vector to the target output space to generate an air quality prediction value.

4. The air quality multivariate prediction method based on Attention-LSTM according to claim 1, characterized in that: The step (4) specifically comprises the following steps: (4a) Model training and optimization: During the training process, the gradient of the loss function with respect to the model weights is calculated through the back-propagation algorithm. The optimizer uses this gradient information to update the weights in the network to reduce the difference between the predicted value and the true value. The goal of the optimizer is to gradually adjust the weights to make the value of the loss function as small as possible. (4b) Model evaluation: Use the validation set for evaluation, which is used to evaluate the performance of the model during the training process, and use the test set for final evaluation, which is used to evaluate the generalization ability of the model; (4c) Hyperparameter adjustment: The learning rate is set to 0.01 to 0.05 and the batch size is set to 50.

5. The air quality multivariate prediction method based on Attention-LSTM according to claim 1, characterized in that: In step (5), the early warning mechanism refers to: When the air quality forecast When it is less than or equal to the second threshold, it means that the air quality will be normal in the future and no warning signal will be issued; When the second threshold value is less than the air quality prediction value When the value is less than or equal to the first threshold, it indicates that the air quality will be in the second-level pollution state in the future, and a second-level warning signal will be issued; When the first threshold value is less than the air quality prediction value When the air quality is in the first-level pollution state, it indicates that the air quality will be in the first-level pollution state in the future, and a first-level warning signal will be issued; The first threshold is 150, and the second threshold is 100.

Images