Air quality forecasting method fusing monitoring image data

Abstract

The invention discloses an air quality forecasting method fusing monitoring image data. The method comprises the following steps: constructing an LSTM single-channel deep learning network for feature extraction of historical air quality data, meteorological data and time data; a CNN-LSTM single-channel deep learning network used for monitoring image feature extraction is constructed; integrating two single-channel networks, and constructing a dual-channel integrated deep learning model for air quality forecasting; the method comprises the following steps: collecting multi-source data such as monitoring images, air quality and weather, and constructing an air quality multi-source data set; generating a sequence sample for the constructed air quality multi-source data set by using a sliding window method and a'T1: T2 'mode; training the dual-channel integrated deep learning model by using the sequence sample training set to obtain a multi-step multivariable output regression model with forecasting capability; and inputting to-be-detected sequence sample data into the regression model, and outputting a plurality of air quality index forecast results at a plurality of moments in the future. According to the method, the two single-channel neural networks are combined, a relation model among the air pollutant concentration, the monitoring image, the weather and the time is constructed, time sequence forecasting of the air quality is achieved, outdoor monitoring image data are fused, and the forecasting precision of the air quality is improved.

Description

Technical Field

[0001] The invention belongs to the field of air monitoring, and relates to an air quality forecasting method, and in particular to an air quality forecasting method integrating monitoring image data. Background Art

[0002] Air pollution is harmful to humans and the environment in many ways. Therefore, it is necessary to make an estimate of the air quality status and trend in the future, and provide pollution warning information when serious pollution may occur in the future or the pollution concentration exceeds a certain limit, so that relevant environmental protection agencies can take corresponding preventive measures in advance to prevent the occurrence of air pollution hazards. Predicting the concentration of various pollutants in the air in advance is the basis for strengthening air pollution prevention and control and achieving comprehensive environmental governance, and it has important practical significance for people's daily health and government environmental protection.

[0003] Air quality changes have the characteristics of coexistence and superposition of multiple pollutants, mutual influence of multiple media, and coupling of multiple processes. A single data source often cannot fully reflect the complex relationship between them. Multi-source data can provide information of different dimensions and angles to make up for the shortcomings of a single data source. By fusing and comprehensively analyzing multi-source data, new data relationships and mutual influences can be explored, and more comprehensive and rich information can be obtained, thereby improving the accuracy and credibility of the forecast. According to the summary and analysis of existing air quality forecast research, it can be found that the most commonly used multi-source data for single-site air quality forecasts include air pollutant data, meteorological data, and time data. This is because air quality changes are closely related to factors such as atmospheric pollutants, meteorological conditions, and human activities, which interact in a complex way in the urban environment. Therefore, it is scientific and reasonable to use these factors as forecasting factors to forecast air quality. Therefore, in the present invention, commonly used data such as air pollutant data, meteorological data, and time data will continue to be used.

[0004] In addition, with the popularization of image devices such as surveillance cameras, portable cameras and smart phones, and the rapid development of computer vision technology for image analysis, the use of real-scene images for urban air quality perception has become one of the hot research directions with potential. At present, many studies have explored the representation relationship between air quality and outdoor real-scene images, and used images to achieve qualitative and quantitative monitoring of urban environmental air quality. Unlike air quality monitoring stations, tens of thousands of fixed imaging devices (e.g., surveillance and traffic cameras) are widely deployed in various cities, which facilitates and timely collects image or video information. Moreover, these data continuously record the dynamic changes of air quality and contain rich spatiotemporal information. Therefore, these surveillance video or image data can be combined to deeply explore their intrinsic information and improve the accuracy and precision of air quality forecasts. To this end, the present invention proposes an air quality forecasting method that integrates multi-source heterogeneous data such as surveillance image data, air pollution data, meteorological data and time data. Summary of the invention

[0005] Purpose of the invention: In order to improve the accuracy of air quality forecast, an air quality forecast method integrating monitoring image data is provided, which fully considers the correlation between air quality and imaging, the temporal variation characteristics of air pollution, the influence of meteorological factors and the interaction between pollutants, combines two single-channel deep learning models to learn the spatial characteristics and temporal associations of homogeneous data and heterogeneous data respectively, and constructs a regression model of multi-source data spatiotemporal characteristics and air quality, thereby achieving high-precision, all-weather air quality forecast.

[0006] Technical solution: To achieve the above purpose, the present invention provides an air quality forecasting method integrating monitoring image data, comprising the following steps:

[0007] S1: Build a long short-term memory (LSTM) single-channel deep learning network to learn the deep features of data with the same structure, such as air pollutant concentrations, meteorological elements, and time information at multiple historical moments;

[0008] S2: Construct a CNN-LSTM single-channel deep learning network for monitoring image feature extraction. Connect the Convolutional Neural Network (CNN) with the LSTM. CNN is used as an image spatial feature extractor, and LSTM is used as an image temporal feature extractor to jointly extract the spatiotemporal information contained in the monitoring image data at multiple historical moments.

[0009] S3: The constructed two single-channel network models, LSTM and CNN-LSTM, are subjected to feature fusion, and a mapping relationship between the fused features and multiple air quality indicators at multiple moments in the future is established, forming a dual-channel integrated deep learning model with multi-step and multi-variable outputs;

[0010] S4: Collect long-term continuous surveillance camera video images, continuously collected air pollutant concentrations and meteorological data, and combine the data records at the same time one by one based on the timestamp of the data to build a multi-source air quality data set;

[0011] S5: Using the sliding window method and “T 1 :T 2 " mode, generating sequence samples for the constructed air quality multi-source dataset and dividing it into training sets;

[0012] S6: Use the sequence sample training set to train the dual-channel integrated deep learning model to obtain an air quality regression model with forecasting capabilities;

[0013] S7: Input the image sequence to be detected into the regression model obtained in step S6, and output the predicted values ​​of multiple air pollutant concentrations at multiple moments in the future.

[0014] Furthermore, the specific process of constructing the LSTM single-channel deep learning network in step S1 is:

[0015] First, T 1 The homogeneous data set consisting of historical air pollutant concentration data, meteorological element data, and time information at each time step is used as input. Then, it is transmitted to the LSTM layer to learn and extract the time series features of multi-source homogeneous data, and finally outputs the feature vector F lstm .

[0016] Furthermore, the specific process of constructing the CNN-LSTM single-channel deep learning network in step S2 is:

[0017] First, multiple surveillance images taken continuously are composed of the dimension (T 1 ,H,W,C) sequence as input, T 1 is the length of the image sequence, which is also the time dimension of the historical data to be learned. (H, W, C) is the dimension of a single surveillance image. H, W, and C are the height, width, and number of color channels of the input image. Then, the fully connected layer and output layer in the complete CNN framework are removed, and only the previous feature extractor part is retained for image spatial feature learning. Subsequently, the LSTM network layer is seamlessly connected, and the spatial features of the image sequence output by the CNN feature extractor are used as the input of the LSTM layer, stored and written into the memory unit of the LSTM, and the temporal characteristics between them are learned, thereby obtaining the spatiotemporal feature vector F of the surveillance image.cnn-lstm .

[0018] Furthermore, the network structure of the dual-channel integrated deep learning model in step S3 is:

[0019] The F output of the above two single-channel networks is lstm and F cnn-lstm The two feature vectors are fused, and a fully connected layer and an output layer are added after the fusion layer. The fused features are input into the fully connected layer to achieve mapping with the output target. The number of neurons in the output layer is consistent with the length of time to be predicted. In addition, the output layer of the model sets a sigmoid activation function to control the output value within the range of [0,1].

[0020] Furthermore, in the process of constructing the data set in step S4, the data needs to be preprocessed, and the specific process is as follows:

[0021] First, all monitoring images in the dataset are compressed to the input size required by the deep learning network. Then, the image data and air quality data are normalized to speed up the convergence of model training. For image data, the pixel value of the input image is divided by the pixel upper limit of 255 to scale it from 0-255 to 0-1; for air pollutant concentration data and meteorological data, the minimum-maximum normalization method is used to map air pollutant concentration and meteorological element indicators to between 0-1.

[0022] Furthermore, the method for generating the sequence samples in step S5 is:

[0023] The sliding window method is used to generate sequence samples from the entire data, using the "T 1 :T 2 " mode sets the time length of historical data and the time length of the target to be predicted, that is, continuous T 1 +T 2 The data collected in hours is taken as a time series sample. In this time series, the first T 1 Hours of monitoring image data, air quality data, meteorological data and time information are used as historical data for prediction. 2 The air pollutant concentration value in the next hour is taken as the target to be estimated.

[0024] Furthermore, the model training process in step S6 needs to initialize the model hyperparameters in advance, specifically including:

[0025] Loss function, gradient descent optimizer during backpropagation, learning rate used for parameter update, number of training epochs, batch size.

[0026] Beneficial effects: Compared with the prior art, the present invention increases the utilization of monitoring image data, uses a dual-channel integrated deep learning model, extracts and learns the spatiotemporal characteristics of multi-source heterogeneous data such as historical air quality, weather, time and images, and constructs a relationship model between spatiotemporal characteristics and air quality, which can achieve the prediction of future air quality. By considering the time series variation characteristics of air pollutants, the influence of meteorological factors, the correlation between pollution factors, and the image representation of air quality, this method effectively improves the forecast accuracy of all-weather air quality. BRIEF DESCRIPTION OF THE DRAWINGS

[0027] Figure 1 is a flow chart of the method of the present invention;

[0028] Figure 2 Network structure diagram for dual-channel integrated deep learning model;

[0029] Figure 3 It is a scatter plot of the prediction results of the method of the present invention on the test data set;

[0030] Figure 4 This is an example diagram of the prediction results of the test image by the method of the present invention;

[0031] Figure 5 This is the comparison result between the method of the present invention and the traditional forecasting method. DETAILED DESCRIPTION

[0032] The present invention is further explained below in conjunction with the accompanying drawings and specific embodiments. It should be understood that these embodiments are only used to illustrate the present invention and are not used to limit the scope of the present invention. After reading the present invention, various equivalent forms of modifications to the present invention by those skilled in the art all fall within the scope defined by the claims attached to this application.

[0033] The present invention provides an air quality forecasting method integrating monitoring image data, such as Figure 1 As shown, in this embodiment, the method is applied as an example, that is, based on the historical 12 hours (T 1 ) to predict the next 12 hours (T 2 ) 2.5 、PM 10 、SO 2 、NO 2 , O 3 and CO concentrations of six air pollutants, which specifically include the following steps:

[0034] Step 1: Build an LSTM single-channel deep learning network based on Python programming language and TensorFlow open source framework. The LSTM single-channel network takes multi-source homogeneous data such as six air pollutant concentration data, meteorological data and time data of 12 historical time steps as input and transmits them to the LSTM network layer to learn the time characteristics of the data. The LSTM layer contains 200 hidden neurons and outputs a 200-dimensional feature vector F after the LSTM layer. lstm .

[0035] Step 2: Using VGG16 as the CNN extractor, construct a VGG16-LSTM single-channel deep learning network, convert the surveillance image sequence continuously shot in the previous 12 hours into a multi-dimensional array form of (12,224,224,3), and input it into the VGG16-LSTM feature extractor for learning. After the image sequence is extracted with spatial features by the VGG16 structure, a feature vector with a dimension of (12,512) is output, which is then transmitted to the memory unit of the LSTM layer to extract and learn the temporal association between the spatial feature sequences. In order to match the feature output of VGG16, the number of hidden neurons in the LSTM layer is set to 512. Finally, a 512-dimensional spatiotemporal feature F is obtained. vgg16-lstm , waiting for the next step of integrated learning.

[0036] Step 3: In order to fully mine the spatiotemporal information of multi-source heterogeneous data, the method of the present invention associates and merges the temporal features of the isomorphic data extracted by the LSTM single-channel network with the spatiotemporal features of the image sequence extracted by the VGG16-LSTM single-channel network to achieve "feature-level" fusion. The fused features are represented as F c After feature fusion, the network connects two fully connected layers to form a mapping relationship from input to output. The first fully connected layer contains 200 neuron nodes, which further transforms the fused feature F c The second fully connected layer is equivalent to the output layer of the model. It has 12 neurons, corresponding to the length of time to be predicted. PM is output at each time point. 2.5 、PM 10 、SO 2 、NO 2 , O 3 Therefore, the entire dual-channel integrated air quality forecast deep learning model is as follows Figure 2 shown.

[0037] Step 4: Collect 8499 surveillance images of Shanghai Lujiazui taken by surveillance cameras every hour from 00:00 on January 1, 2021 to 23:00 on December 31, 2021, and obtain the PM2.5 value at the corresponding time of each image from the historical data published by the China National Environmental Monitoring Center. 2.5 、PM10 、SO 2 、NO 2 , O 3 The concentration data of six air pollutants, including CO, are collected and calculated from the fifth-generation meteorological reanalysis data ERA5 to obtain seven meteorological elements, including precipitation, pressure, temperature, evapotranspiration, relative humidity, wind speed and wind direction, at the corresponding time of each image. Combined with the month and time information of each image, a multi-source data set of air quality is constructed. The data needs to be preprocessed during the construction of the data set. The specific process is as follows:

[0038] Step 2-1: Compress all surveillance images in the dataset to the input size required by the VGG16 network, i.e. 224×224; divide the pixel value of each image by 255 to scale it from 0-255 pixel values ​​to 0-1;

[0039] Step 2-2: Use the minimum-maximum normalization method to normalize the air quality data and meteorological data, and convert PM 2.5 、PM 10 、SO 2 、NO 2 , O 3 The values ​​of seven meteorological elements, including CO concentration, precipitation, pressure, temperature, evapotranspiration, relative humidity, wind speed and wind direction, are mapped to a range of 0-1.

[0040] Step 5: Use the sliding window method and "T 1 :T 2 " mode, generate training sequence samples for the constructed air quality multi-source dataset, that is, for the entire time series data, take the continuous images, six air pollutant concentrations, seven meteorological elements, and time information from the 1st to the 12th hour as the first historical sequence, and take the PM from the 13th to the 24th hour as the second historical sequence. 2.5 、PM 10 、SO 2 、NO 2 , O 3 The six air pollution concentration values ​​of and CO are taken as the first target label; that is, the continuous images from the 2nd to the 13th hour, the six air pollutant concentrations, the seven meteorological elements, and the time information are taken as the second historical sequence, and the PM from the 14th to the 25th hour are taken as the second historical sequence. 2.5 、PM 10 、SO 2 、NO 2 , O 3The six air pollution concentration values ​​of CO and CO are used as the second target label; and so on, new samples are generated until the pollutant data at the last moment of the entire time series data is included in the last target label. According to the sequence generation method, a total of 8365 sequence samples are generated for the self-collected air quality image dataset, and then all sequence samples are divided into training set and test set in a 1:1 ratio for training and testing respectively.

[0041] Step 6: Set the loss function of the model to mean square error (MSE); set the gradient descent optimization algorithm to Adam optimizer, and the learning rate to 0.00001; set the number of training rounds to 100, and the batch size to 8. Then, the dual-channel integrated deep learning model was trained using the training sample set, and finally an air quality regression prediction model with prediction capability was obtained;

[0042] Step 7: Input each historical sequence in the test set into the regression model obtained in step 6 and output its corresponding PM in the next 12 hours. 2.5 、PM 10 、SO 2 、NO 2 , O 3 and CO concentration.

[0043] In this example, the coefficient of determination (R 2 ) and root mean square error (RMSE) were used to evaluate the performance of the method on the test set. A good model performance should have a high coefficient of determination R 2 and low RMSE. 2 And the RMSE calculation formula is as follows:

[0044]

[0045] Finally, it can be found that the air quality prediction method of integrating monitoring image data proposed in the present invention achieves good performance on the test set, such as Figure 3 As shown, each air pollutant has a higher R 2 and lower RMSE: PM 2.5 R 2 and RMSE were 0.95 and 4.79 μg / m 3 , PM 10 R 2 and RMSE were 0.90 and 11.51 μg / m 3 , SO 2 R 2 and RMSE were 0.78 and 1.17 μg / m 3 , NO 2 R 2and RMSE were 0.95 and 5.17 μg / m 3 , O 3 R 2 and RMSE were 0.96 and 7.57 μg / m 3 , R of CO 2 and RMSE were 0.92 and 0.07 μg / m 3 .

[0046] At the same time, an example of the prediction result of the method of the present invention for the test image is provided ( Figure 4 ) and the comparison results between the proposed method and the traditional method on the test set ( Figure 5 ).

[0047] It can be seen that the evaluation result of the method of the present invention is better than that of the traditional machine learning forecasting method, and the method of the present invention effectively improves the forecast accuracy of air quality.

Claims

1. An air quality forecasting method integrating monitoring image data, characterized in that: The following steps are involved: S1: Build a long short-term memory (LSTM) single-channel deep learning network to learn the deep features of air pollutant concentrations, meteorological elements, and time information data at multiple historical moments; S2: Build a CNN-LSTM single-channel deep learning network for monitoring image feature extraction. Connect the convolutional neural network CNN with LSTM. CNN is used as an image spatial feature extractor, and LSTM is used as an image temporal feature extractor to jointly extract the spatiotemporal information contained in the monitoring image data at multiple historical moments. S3: The constructed two single-channel network models, LSTM and CNN-LSTM, are subjected to feature fusion, and a mapping relationship between the fused features and multiple air quality indicators at multiple moments in the future is established, forming a dual-channel integrated deep learning model with multi-step and multi-variable outputs; S4: Collect long-term continuous surveillance camera video images, continuously collected air pollutant concentrations and meteorological data, and combine the data records at the same time one by one based on the timestamp of the data to build a multi-source air quality data set; S5: Using the sliding window method and the "T1:T2" mode, we generate sequence samples for the constructed air quality multi-source dataset and divide it into training sets; S6: Use the sequence sample training set to train the dual-channel integrated deep learning model to obtain an air quality regression model with forecasting capabilities; S7: Input the image sequence to be detected into the regression model obtained in step S6, and output the predicted values ​​of multiple air pollutant concentrations at multiple moments in the future.

2. The air quality forecasting method of integrating monitoring image data according to claim 1 is characterized in that: The specific process of constructing the LSTM single-channel deep learning network in step S1 is: First, the homogeneous data set consisting of historical air pollutant concentration data, meteorological element data, and time information of T1 time steps is taken as input; then, it is transmitted to the LSTM layer to learn and extract the temporal features of multi-source homogeneous data, and finally the feature vector F is output. lstm .

3. The air quality forecasting method of integrating monitoring image data according to claim 2 is characterized in that: The specific process of constructing the CNN-LSTM single-channel deep learning network in step S2 is: First, a sequence of multiple surveillance images with a dimension of (T1, H, W, C) is used as input, where T1 is the length of the image sequence, which is also the time dimension of the historical data to be learned, (H, W, C) is the dimension of a single surveillance image, and H, W, C are the height, width, and number of color channels of the input image; then, the fully connected layer and output layer in the complete CNN framework are removed, and only the previous feature extractor part is retained for spatial feature learning of the image; Then, the LSTM network layer is seamlessly connected, and the spatial features of the image sequence output by the CNN feature extractor are used as the input of the LSTM layer, stored and written into the memory unit of the LSTM, and the temporal characteristics between them are learned, thereby obtaining the spatiotemporal feature vector F of the monitoring image. cnn-lstm .

4. The air quality forecasting method of integrating monitoring image data according to claim 3 is characterized in that: The network structure of the dual-channel integrated deep learning model in step S3 is: The F output of the two single-channel networks is lstm and F cnn-lstm The two feature vectors are fused, and a fully connected layer and an output layer are added after the fusion layer. The fused features are input into the fully connected layer to achieve mapping with the output target. The number of neurons in the output layer is consistent with the length of time to be predicted. Among them, the output layer of the model sets a sigmoid activation function to control the output value within the range of [0,1].

5. The air quality forecasting method of integrating monitoring image data according to claim 1 is characterized in that: In the process of constructing the data set in step S4, the data needs to be preprocessed, and the specific process is as follows: First, all monitoring images in the dataset are compressed to the input size required by the deep learning network. Then, the image data and air quality data are normalized to speed up the convergence of model training. For image data, the pixel value of the input image is divided by the pixel upper limit of 255 to scale it from 0-255 to 0-1. For air pollutant concentration data and meteorological data, the minimum-maximum normalization method is used to map air pollutant concentration and meteorological element indicators to between 0 and 1.

6. The air quality forecasting method of fusing monitoring image data according to claim 1 is characterized in that: The method for generating the sequence samples in step S5 is: The sliding window method is used to generate sequence samples from the entire data, and the "T1:T2" mode is used to set the time length of historical data and the time length of the target to be predicted. That is, the data collected for consecutive T1+T2 hours is used as a time series sample. In this time series, the monitoring image data, air quality data, meteorological data and time information of the first T1 hours are used as historical data for prediction, and the air pollutant concentration values ​​of the last T2 hours are used as the target to be estimated.

7. The air quality forecasting method of integrating monitoring image data according to claim 1 is characterized in that: The model training process in step S6 requires initializing the model hyperparameters in advance, specifically including: Loss function, gradient descent optimizer during backpropagation, learning rate for parameter update, number of training epochs, batch size.

Images