Air quality index prediction method based on BiLSTM-WaveNet-MHSA
By adopting the BiLSTM-WaveNet-MHSA method in air quality index prediction, the problem that the prior art is difficult to effectively extract bidirectional dependence characteristics in time series data is solved, and more efficient feature extraction and prediction accuracy is achieved.
Patent Information
- Application Number
- CN202510044002.9
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-01-10
- Publication Date
- 2025-04-15
AI Technical Summary
When the existing air quality index prediction methods process time series data, it is difficult to effectively extract the two-way dependence characteristics in pollutant concentration data, resulting in insufficient prediction accuracy.
The air quality index prediction method based on BiLSTM-WaveNet-MHSA is adopted, and the improved WaveNet residual network is added through the Bilateral Long Short-term Memory Network (BiLSTM) as the basic network, and the multi-head self-attention mechanism (MHSA) is introduced to output the predicted value.
Through the expansion causal convolution of WaveNet residual network and the weight calculation of the multi-head self-attention mechanism, the time characteristics and correlation characteristics of influencing factors in the air quality data can be more effectively extracted, improving the accuracy of prediction and network efficiency.
Smart Images
Figure CN119831107A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to an air quality index prediction method based on BiLSTM-WaveNet-MHSA, and relates to deep learning related fields. Background Art
[0002] With the development of global economy and industrialization, people's living standards have been improved. At the same time, environmental issues such as air pollution have become the focus of people's attention. As the World Health Organization (WHO) points out, air pollution is the world's largest environmental health risk, which will cause many diseases, including but not limited to respiratory infections, heart disease, chronic obstructive pulmonary disease, stroke and cancer. Therefore, as people's health awareness increases, more and more smart devices (such as smart belts) are developed and equipped to report air quality conditions.
[0003] The problem of air pollution has been widely discussed in the field of Internet of Things and sensor networks. C. Zhang et al. designed an effective convolutional neural network for predicting air quality index, which mainly consists of two parts: first, a negative logarithmic order classifier is designed in the last layer of the network to improve the ordinal discrimination ability of the model. Secondly, an activation function based on a linear rectifier function is designed for the problem of air quality detection based on sky images. L. Lin et al. proposed two dynamic models based on recurrent neural networks and long short-term memory neural networks (Long Short Time Memory, LSTM) models to predict PM2.5 concentration using PM2.5 time series obtained at air quality monitoring stations and weather information obtained in the near weather. In order to more accurately capture the bidirectional dependency characteristics in time series data, Huang Guangqiu et al. constructed K-CNN-BiLSTM for overall prediction analysis.
[0004] At present, the BiLSTM model is widely used in air quality index prediction and has been widely recognized. However, it is ignored that the processing of time series data by the WaveNet residual network is also applicable to pollutant concentration data. They all have the same data structure. The WaveNet residual network uses dilated causal convolution in feature extraction to greatly improve efficiency and prediction accuracy. Therefore, the present invention proposes an air quality index prediction method based on BiLSTM-WaveNet-MHSA. Summary of the invention
[0005] The purpose of the present invention is to disclose an air quality index prediction method based on BiLSTM-WaveNet-MHSA, which adopts a bidirectional long short-term memory network BiLSTM as the basic network, adds an improved WaveNet residual network, and introduces a multi-head self-attention mechanism to output the predicted value, which can not only better process pollutant concentration data and extract features, but also improve the efficiency of the network and the accuracy of the prediction.
[0006] The air quality index prediction method based on BiLSTM-WaveNet-MHSA includes:
[0007] Step 1: Preprocess the air quality data set;
[0008] Step 2: The backbone network uses the bidirectional long short-term memory network BiLSTM as the basic network, adds the improved WaveNet residual network, and introduces the multi-head self-attention MHSA network to output the predicted value;
[0009] Step 3: Input the preprocessed air quality data into the backbone network in step 2. The data stream passes through the BiLSTM network layer and the WaveNet residual network layer to extract shallow time features and deep detail features, including the characteristics of pollutant concentration changes over time and the correlation characteristics with influencing factors such as wind speed, temperature and humidity;
[0010] Step 4: Input the shallow temporal features and deep detail features extracted in step 3 into the multi-head self-attention MHSA network, assign weights according to the importance of different features, and calculate the weight outputs of different features;
[0011] Step 5: Based on the output obtained after weighting different features in step 4, the MHSA layer output is finally obtained through the fully connected layer to complete the air quality index prediction.
[0012] The present invention is also characterized in that:
[0013] The so-called pollutant data in step 1 includes time, AQI value, pollutant concentration, wind speed, temperature, humidity, average dew point temperature, visibility and precipitation, etc. The data comes from the China National Environmental Monitoring Center. The obtained data needs to be preprocessed, including missing value processing and normalization processing. The format is changed to a data set with a time series structure that can be input into the network structure. The data set is divided into training set, test set and validation set in a ratio of 7:2:1.
[0014] The backbone network in step 2 uses the bidirectional long short-term memory network BiLSTM as the basic network, adds an improved WaveNet residual network, and introduces the multi-head self-attention MHSA network to output the predicted value.
[0015] The specific process of step 3 is as follows: the data preprocessed in step 1 is input into the backbone network in step 2. A BiLSTM unit includes a forward LSTM unit block and a reverse LSTM unit block. In each LSTM unit block, the input at the current moment is concatenated with the output value of the previous moment and input into the forget gate, input gate and output gate respectively. The forget gate performs a sigmoid operation on the concatenated values together. The value after the operation f t The cell block state C at the previous moment t-1 The input gate performs a sigmoid operation on the concatenated value to obtain i t , and then perform tanh operation to get the current unit block update state C~ t , change i t and C~ t Perform a bitwise multiplication operation, and perform a bitwise addition operation on the value obtained and the cell block state after the forget gate operation is completed to obtain the current cell block state C t The output gate performs a sigmoid operation on the concatenated value to obtain O t , C t Do the tanh operation with O t Perform bitwise multiplication to get the hidden state output value h at the current moment t . Then the hidden state output of the current forward LSTM unit block is concatenated with the hidden state output of the reverse LSTM unit block to obtain the BiLSTM unit block output at the current moment. Then the BiLSTM unit block outputs at all times of all time steps are concatenated and input into the improved WaveNet residual network layer. The improved WaveNet residual network layer is composed of causal convolution and 50 layers of dilated convolution feature extraction network superposition. Each layer is subjected to dilated convolution and tanh operation and sigmoid operation respectively. The bitwise multiplication is performed through 1×1 convolution and the value just input after the dilated convolution is added bitwise to obtain the output value. The output values of the 50 layers are added bitwise and skipped as the output value of the WaveNet residual block, and then passed through a 1×1 convolution, Relu activation function, Dropout layer, 1×1 convolution and Relu activation function convolution block to obtain the output of the WaveNet residual network layer.
[0016] Step 4 is as follows: When the WaveNet residual network layer processes the time series information in step 3, each residual network calculation will extract a deeper layer of time features, and use the output of the WaveNet residual network layer as the input of the multi-head self-attention MHSA network layer. Weights are assigned according to the importance of different features, and the weighted outputs of different features are calculated.
[0017] Step 5 is as follows: according to the output obtained after weighting different features in step 4, the MHSA layer output is finally obtained through the fully connected layer to complete the air quality index prediction.
[0018] The beneficial effects of the present invention are:
[0019] The present invention proposes an air quality index prediction method based on BiLSTM-WaveNet-MHSA. The backbone network adopts a bidirectional long short-term memory network BiLSTM as the basic network, adds an improved WaveNet residual network, and introduces a multi-head self-attention MHSA to output the predicted value. The dilated causal convolution can better process the time series data, improve the network training efficiency, and extract features that are closer to reality, thereby improving the prediction accuracy. Then, the network feature output is subjected to multi-head self-attention weighted processing, and key data is selected to be given a high weight, thereby improving the accuracy of the network model prediction and more effectively predicting the air quality index. BRIEF DESCRIPTION OF THE DRAWINGS
[0020] Figure 1 Flow chart of the air quality index prediction method based on BiLSTM-WaveNet-MHSA involved in the present invention
[0021] Figure 2 The network structure diagram of the air quality index prediction method based on BiLSTM-WaveNet-MHSA involved in the present invention
[0022] Figure 3 LSTM unit block network structure diagram of the air quality index prediction method based on BiLSTM-WaveNet-MHSA involved in the present invention
[0023] Figure 4 BiLSTM network structure diagram of the air quality index prediction method based on BiLSTM-WaveNet-MHSA involved in the present invention
[0024] Figure 5 A network structure diagram of an improved WaveNet residual network based on the air quality index prediction method based on BiLSTM-WaveNet-MHSA according to the present invention DETAILED DESCRIPTION
[0025] The present invention proposes an air quality index prediction method based on BiLSTM-WaveNet-MHSA. In order to enable those skilled in the art to better understand the technical solution in the embodiments of the present invention and to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, the technical solution in the present invention is further described in detail below in conjunction with the accompanying drawings:
[0026] The present invention first provides an air quality index prediction method based on BiLSTM-WaveNet-MHSA, such as Figure 1 As shown, the specific method is as follows:
[0027] Step 1: Preprocess the pollutant data set;
[0028] Step 1 is as follows:
[0029] The pollutant data including time, AQI value, pollutant concentration, wind speed, temperature, humidity, average dew point temperature, visibility and precipitation are preprocessed. These data come from the China National Environmental Monitoring Center. The obtained data need to be preprocessed, including missing value processing and normalization processing. The format is changed to a data set with a time series structure that can be input into the network structure. The data set is divided into training set, test set and validation set in a ratio of 7:2:1.
[0030] Step 2: The backbone network uses the bidirectional long short-term memory network BiLSTM as the basic network, adds the improved WaveNet residual network, and introduces the multi-head self-attention MHSA to output the air quality prediction value;
[0031] Step 2 is as follows:
[0032] The data is input into the BiLSTM layer to obtain the BiLSTM unit block outputs at all times of all time steps, which are then spliced and input into the improved WaveNet residual network layer. It first passes through causal convolution, then enters the 50-layer dilated convolution residual network, and passes through a convolution block to obtain the output of the WaveNet residual network layer. It is then used as the input of the multi-head self-attention MHSA network layer. Weights are assigned according to the importance of different features, and the weighted outputs of different features are calculated. Finally, the prediction value is output through the fully connected layer, as shown in the following figure: Figure 2 The network structure shown;
[0033] Step 3: Input the preprocessed air quality data into the backbone network in step 2. The data stream passes through the BiLSTM network layer and the WaveNet residual network layer to extract shallow time features and deep detail features, including the characteristics of pollutant concentration changes over time and the correlation characteristics with influencing factors such as wind speed, temperature and humidity;
[0034] Step 3 is as follows:
[0035] The preprocessed data is input into Figure 2 In the network model shown, take the current time step as an example: the data preprocessed in step 1 is input into the backbone network in step 2, such as Figure 4 As shown, a BiLSTM unit includes a forward LSTM unit block and a reverse LSTM unit block. The network structure of the LSTM unit block is as follows Figure 3 As shown in the figure, in each LSTM unit block, the current input is concatenated with the output value of the previous moment and input into the forget gate, input gate and output gate respectively. The forget gate performs a sigmoid operation on the concatenated values together, and the value after the operation is f t The cell block state C at the previous moment t-1 The input gate performs a sigmoid operation on the concatenated value to obtain i t , and then perform tanh operation to get the current unit block update state C~ t , change i t and C~ t Perform a bitwise multiplication operation, and perform a bitwise addition operation on the value obtained and the cell block state after the forget gate operation is completed to obtain the current cell block state C t The output gate performs a sigmoid operation on the concatenated value to obtain O t , C t Do the tanh operation with O t Perform bitwise multiplication to get the hidden state output value h at the current moment t Then the hidden state output of the current forward LSTM unit block is concatenated with the hidden state output of the reverse LSTM unit block to obtain the BiLSTM unit block output at the current moment. Then the BiLSTM unit block outputs at all moments of all time steps are concatenated and input into the improved WaveNet residual network layer. The improved WaveNet residual network is as follows: Figure 5 As shown in the figure, the improved WaveNet residual network layer is composed of a causal convolution and a 50-layer dilated convolution feature extraction network superposition. Each layer is subjected to dilated convolution and then performs tanh and sigmoid operations respectively. The bitwise multiplication is performed through 1×1 convolution and the value just input after the hole convolution is added bitwise to obtain the output value. The output values of the 50 layers are added bitwise and jump-connected as the output values of the WaveNet residual block, and then passed through a 1×1 convolution, Relu activation function, Dropout layer, 1×1 convolution and Relu activation function convolution block to obtain the output of the WaveNet residual network layer.
[0036] Step 4: Input the shallow temporal features and deep detail features extracted in step 3 into the multi-head self-attention MHSA network, assign weights according to the importance of different features, and calculate the weight outputs of different features;
[0037] Step 4 is as follows:
[0038] In step 3, when the WaveNet residual network layer processes the time series information, each residual network calculation will extract a deeper layer of time features. The output of the WaveNet residual network layer is used as the input of the multi-head self-attention MHSA network layer. Weights are assigned according to the importance of different features, and the weighted outputs of different features are calculated.
[0039] Step 5: Based on the output obtained after weighting different features in step 4, the MHSA layer output is finally obtained through the fully connected layer to complete the air quality index prediction.
[0040] Step 5 is as follows:
[0041] According to the output obtained after weighting different features in step 4, the air quality index prediction value is output through the fully connected layer.
Claims
1. The air quality index prediction method based on BiLSTM-WaveNet-MHSA is characterized by: This is accomplished by following these steps: Step 1: Preprocess the air quality data set; Step 2: The backbone network uses the bidirectional long short-term memory network BiLSTM as the basic network, adds the improved WaveNet residual network, and introduces the multi-head self-attention MHSA network to output the predicted value; Step 3: Input the preprocessed air quality data into the backbone network in step 2. The data stream passes through the BiLSTM network layer and the WaveNet residual network layer to extract shallow time features and deep detail features, including the characteristics of pollutant concentration changes over time and the correlation characteristics with influencing factors such as wind speed, temperature and humidity; Step 4: Input the shallow temporal features and deep detail features extracted in step 3 into the multi-head self-attention MHSA network, assign weights according to the importance of different features, and calculate the weight outputs of different features; Step 5: Based on the output obtained after weighting different features in step 4, the MHSA layer output is finally obtained through the fully connected layer to complete the air quality index prediction.