Air quality prediction method and system based on incomplete data

Abstract

The application discloses an air quality prediction method based on incomplete data, and the method comprises the following steps: 1) given P monitoring points, M pollutants (including PM2.5, PM10 and the like) and T days, historical air quality monitoring data is organized into a three-dimensional tensor, the value of the mth pollutant monitored by the pth monitoring point; 2) given a set Omega, the set Omega comprises indexes of known elements (i.e. observable elements) in the three-dimensional tensor; 3) given a prediction day number K and a tensor dimension I2, an air quality tensor is constructed; 4) based on the set Omega, a weight tensor is constructed; 5) based on the air quality tensor and the weight tensor, a target function f (A, B, C, D) is constructed, and then an optimization problem is obtained, wherein A, B, C and D are factor matrices; 6) the estimation value of the factor matrix is obtained by using an alternating minimization algorithm, and the value of each pollutant at each monitoring point from T+1 day to T+K day is calculated; the application solves the problem of air quality prediction on incomplete data.

Description

Technical Field

[0001] This invention relates to the field of urban computing, and in particular to an air quality prediction method and system based on incomplete data. Background Technology

[0002] Air quality forecasting aims to predict the levels of various pollutants at one or more monitoring stations over a future period. On the one hand, most existing methods assume that air quality monitoring data is complete. In reality, due to sensor malfunctions, network outages, and other reasons, air quality monitoring data often contains missing values. Therefore, designing a method capable of air quality forecasting based on incomplete data remains an unsolved problem. On the other hand, pollutants exhibit complex spatial, temporal, and categorical correlations. For example, pollutants at one monitoring station can be influenced by pollutants at neighboring monitoring stations, leading to similar pollutant values ​​at two adjacent stations. Most existing methods do not fully utilize the temporal, spatial, and categorical correlations inherent in air quality monitoring data to improve forecasting performance. Summary of the Invention

[0003] The purpose of this invention is to provide an air quality prediction method and system based on incomplete data. This method not only allows for direct prediction on incomplete data (air quality monitoring data containing missing values), but also fully utilizes the spatial, temporal, and categorical correlations inherent in the air quality monitoring data to improve prediction performance. (Note: Incomplete data refers to air quality monitoring data containing missing values.)

[0004] The technical solution of this invention: an air quality prediction method based on incomplete data, comprising the following steps:

[0005] Step 1: Given P monitoring points, M types of pollutants (including PM2.5, PM10, etc.) and T days, organize the historical air quality monitoring data into a three-dimensional tensor. in This represents the value of the m-th pollutant detected at the p-th monitoring point on day t;

[0006] Step 2: Given a set Ω, which contains three-dimensional tensors The index of a known element (i.e., an observable element) in the set, i.e., if x... ptm If these are known, then (p, t, m) ∈ Ω;

[0007] Step 3: Given the number of prediction days K and the tensor dimension I2, construct the air quality tensor.

[0008] Step 4: Construct the weight tensor based on the set Ω.

[0009] Step 5: Based on the air mass tensor and weight tensor Construct the objective function f(A, B, C, D), and then obtain the optimization problem. Where A, B, C, and D are factor matrices;

[0010] Step Six: Use the alternating minimization algorithm to obtain the estimated value of the factor matrix.

[0011] Step 7: Based on the estimated values ​​of the factor matrix This invention calculates the values ​​of various pollutants at each monitoring point from day T+1 to day T+K. It effectively solves the problem of air quality prediction based on incomplete data.

[0012] In step three, given the number of prediction days K and the tensor dimension I2, an air quality tensor is constructed. Specifically, it includes:

[0013] Will Expand to in 1≤p≤P, 1≤t≤T+K, 1≤m≤M;

[0014] Construct a four-dimensional tensor This is used to represent air quality monitoring data. Here, I1 = P, I4 = M, which represent the number of monitoring points and the number of pollutant types to be predicted. I2 is set to a multiple of 7, such as 7, 14, 21, etc., and I2 > K; in This represents the floor function, which returns the largest integer less than or equal to a given number. Let T′ = T + K - I² × I³, Mapped to according to the following rules

[0015]

[0016] 1≤p≤P, T′+1≤t≤T+K, 1≤m≤M. “mod” represents the modulo operation, returning the remainder when one number is divided by another. This represents the floor function, which returns the smallest integer greater than or equal to a given number.

[0017] In step four, the weight tensor is constructed based on the set Ω. Specifically, it includes:

[0018] Construct weight tensor in The relationship between (p, t, m) and (p, t, m) is determined by the following mapping rule:

[0019]

[0020] 1≤p≤P, T′+1≤t≤T+K, 1≤m≤M.

[0021] In step five, the air mass tensor-based... and weight tensor Construct the objective function f(A, B, C, D), and then obtain the optimization problem. Where A, B, C, and D are factor matrices, specifically including:

[0022] Construct the objective function Where * indicates bitwise multiplication, ||·|| F This represents the Frobenius norm. It is a factor matrix. R is a positive integer called a tensor. CP rank. That is to put It can be decomposed into a sum of tensors of rank 1. It is a regularization term, here we let It is a parameter used to control the effect of the regularization term on the objective function.

[0023] Based on this, the optimization problem can be obtained.

[0024] In step seven, the statement based on Calculate the values ​​of various pollutants at each monitoring point from day T+1 to day T+K, specifically including:

[0025] On day t, the predicted value of the m-th pollutant at the p-th monitoring point is obtained by the following formula:

[0026]

[0027] 1≤p≤P, T+1≤t≤T+K, 1≤m≤M. Wherein,

[0028] An air quality prediction system based on incomplete data, wherein incomplete data refers to air quality monitoring data with missing values. The system is characterized by comprising: a data processing unit, a tensor construction unit, an optimization objective modeling unit, a model training unit, and an air quality prediction unit.

[0029] The data processing unit is used to organize historical air quality monitoring data into a three-dimensional tensor. Where P is the number of monitoring points, M is the number of pollutant types, and T is the number of days. This represents the value of the m-th pollutant detected at the p-th monitoring point on day t;

[0030] Tensor construction units are used to construct tensors based on a given number of prediction days K, tensor dimension I², and three-dimensional tensor. Constructing the air mass tensor

[0031] The tensor construction unit is also used to construct a weight tensor based on a given set Ω.

[0032] Optimize the target modeling unit based on the air mass tensor and weight tensor Construct the objective function f(A, B, C, D), and then obtain the optimization problem. Where A, B, C, and D are factor matrices;

[0033] The model training unit is used to obtain estimates of the factor matrix using an alternating minimization algorithm.

[0034] Air quality prediction unit, used to predict air quality based on Calculate the values ​​of various pollutants at each monitoring point from day T+1 to day T+K.

[0035] The data processing unit is used to organize historical air quality monitoring data into a three-dimensional tensor. Where P is the number of monitoring points, M is the number of pollutant types, and T is the number of days. This represents the value of the m-th pollutant detected at the p-th monitoring point on day t;

[0036] Tensor construction units are used to construct tensors based on a given number of prediction days K, tensor dimension I², and three-dimensional tensor. Constructing the air mass tensor

[0037] The tensor construction unit is also used to construct a weight tensor based on a given set Ω.

[0038] Optimize the target modeling unit based on the air mass tensor and weight tensor Construct the objective function f(A, B, C, D), and then obtain the optimization problem. Where A, B, C, and D are factor matrices;

[0039] The model training unit is used to obtain estimates of the factor matrix using an alternating minimization algorithm.

[0040] Air quality prediction unit, used to predict air quality based on Calculate the values ​​of various pollutants at each monitoring point from day T+1 to day T+K.

[0041] Furthermore, the tensor construction unit is specifically used to... Expand to in 1≤p≤P, 1≤t≤T+K, 1≤m≤M;

[0042] Construct a four-dimensional tensor Let I1 = P and I4 = M, representing the number of monitoring points and pollutant types to be predicted. I2 is set to a multiple of 7, such as 7, 14, 21, etc., and I2 > K. in This represents the floor function, which returns the largest integer less than or equal to a given number. Let T′ = T + K - I² × I³, Mapped to according to the following rules

[0043]

[0044] 1≤p≤P, T′+1≤t≤T+K, 1≤m≤M. “mod” represents the modulo operation, returning the remainder when one number is divided by another. This represents the floor function, which returns the smallest integer greater than or equal to the given number;

[0045] Construct weight tensor in The relationship between (i1, i2, i3, i4) and (p, t, m) is determined by the following mapping rule:

[0046]

[0047] 1≤p≤P, T′+1≤t≤T+K, 1≤m≤M.

[0048] Furthermore, the optimization target modeling unit is specifically used to construct the objective function. Where * indicates bitwise multiplication, ||·|| F This represents the Frobenius norm. It is a factor matrix. R is a positive integer called a tensor. CP rank. That is to put It can be decomposed into a sum of tensors of rank 1. It is a regularization term, here we let λ is a parameter used to control the effect of the regularization term on the objective function.

[0049] Based on this, the optimization problem can be obtained.

[0050] Furthermore, the air quality prediction unit is specifically used to obtain the predicted value of the m-th pollutant at the p-th monitoring point on day t, according to the following formula:

[0051]

[0052] 1≤p≤P, T+1≤t≤T+K, 1≤m≤M. Wherein,

[0053] Beneficial effects: This invention effectively solves the problem of air quality prediction based on incomplete data, and improves the predictive performance of the model by comprehensively considering the temporal, spatial and category correlations contained in the air quality monitoring data. Attached Figure Description

[0054] Figure 1 This is a flowchart of an air quality prediction method based on incomplete data provided in Embodiment 1 of the present invention.

[0055] Figure 2 This is a schematic diagram of an air quality prediction system based on incomplete data provided in Embodiment 2 of the present invention. Detailed Implementation

[0056] The invention will now be further explained with reference to the accompanying drawings. Figure 1 This is a flowchart of an air quality prediction method based on incomplete data provided in Embodiment 1 of the present invention. Figure 1 As shown, this embodiment includes the following steps:

[0057] Step 1: Given P monitoring points, M types of pollutants (such as PM2.5, PM10, etc.) and T days, organize the historical air quality monitoring data into a three-dimensional tensor. in This represents the value of the m-th pollutant detected at the p-th monitoring point on day t.

[0058] Assuming there are 2 monitoring stations, 2 pollutants (PM2.5, PM10) (other types of pollutants, such as VOCs and sulfides, can be included), and 8 days (November 1, 2022 – November 8, 2022), the historical air quality monitoring data is as follows:

[0059]

[0060] Blank cells indicate that the air quality monitoring data corresponding to that cell is missing.

[0061] Based on the above historical air quality monitoring data, use three-dimensional tensors in This represents the value of the m-th pollutant detected at the p-th monitoring point on day t; organized into a three-dimensional tensor. but The matrix obtained by expanding along the first dimension is

[0062]

[0063] Step 2: Given a set Ω, which contains The index of a known element (i.e., an observable element) in the set, i.e., if x... ptm If it is known, then (p, t, m) ∈ Ω. t can also be a certain time on a certain day.

[0064] The historical air quality monitoring data given in Step 1 corresponds to the following set Ω:

[0065]

[0066] Step 3: Given the number of prediction days K and the tensor dimension I2, construct the air quality tensor. Specifically as follows:

[0067] Will Expand to in 1≤p≤P, 1≤t≤T+K, 1≤m≤M;

[0068] Given a set Ω containing three-dimensional tensors The index of a known element (i.e., an observable element) in the set, i.e., if x... ptm If these are known, then (p, t, m) ∈ Ω;

[0069] Step 3: Given the number of prediction days K and the tensor dimension I2, construct the air quality tensor.

[0070] Step 4: Construct the weight tensor based on the set Ω.

[0071] Given the number of prediction days K and the tensor dimension I2, construct the air quality tensor. Specifically, it includes:

[0072] Will Expand to in 1≤p≤P, 1≤t≤T+K, 1≤m≤M;

[0073] Construct a four-dimensional tensor Let I1 = P and I4 = M, representing the number of monitoring points and pollutant categories to be predicted. I2 is set to a multiple of 7, such as 7, 14, 21, etc., and I2 > K. in This represents the floor function, which returns the largest integer less than or equal to a given number. Let T′ = T + K - I² × I³, Mapped to according to the following rules Here, I1 = P, I4 = M.

[0074] Assuming K = 6 and I² = 7, the air quality tensor corresponding to the historical air quality monitoring data given in step one. The matrix obtained by expanding along the first dimension is

[0075]

[0076] Step 4: Construct the weight tensor based on the set Ω. Specifically as follows:

[0077] Construct weight tensor in The relationship between (p, t, m) and (p, t, m) is determined by the following mapping rule:

[0078]

[0079] 1≤p≤P, T′+1≤t≤T+K, 1≤m≤M.

[0080] "mod" represents the modulo operation, which returns the remainder when one number is divided by another.

[0081] This represents the floor function, which returns the smallest integer greater than or equal to a given number.

[0082] In step four, the weight tensor is constructed based on the set Ω. Specifically, it includes:

[0083] Construct weight tensor in The relationship between (i1, i2, i3, i4) and (p, t, m) is determined by the following mapping rule:

[0084] The historical air quality monitoring data given in Step 1, and the corresponding weight tensor The matrix obtained by expanding along the first dimension is

[0085]

[0086] Step 5: Based on the air mass tensor and weight tensor Construct the objective function f(A, B, C, D), and then obtain the optimization problem. Where A, B, C, and D are factor matrices, as follows:

[0087] Construct the objective function Where * indicates bitwise multiplication, ||·|| F This represents the Frobenius norm. It is a factor matrix. R is a positive integer called a tensor. CP rank. That is to put It can be decomposed into a sum of tensors of rank 1. It is a regularization term, here we let λ is a parameter used to control the effect of the regularization term on the objective function.

[0088] Based on this, the optimization problem can be obtained.

[0089] Step Six: Use the alternating minimization algorithm to obtain the estimated value of the factor matrix.

[0090] Step Seven: According to The values ​​of various pollutants at each monitoring point were calculated from day T+1 to day T+K, as follows:

[0091] On day t, the predicted value of the m-th pollutant at the p-th monitoring point is obtained by the following formula:

[0092]

[0093] 1≤p≤P, T+1≤t≤T+K, 1≤m≤M. Wherein,

[0094] Accordingly, the present invention provides an air quality prediction system based on incomplete data. Figure 2 This is a schematic diagram of an air quality prediction system based on incomplete data provided in Embodiment 2 of the present invention. The system includes: a data processing unit 110, a tensor construction unit 120, an optimization target modeling unit 130, a model training unit 140, and an air quality prediction unit 150.

[0095] Data processing unit 110 is used to organize historical air quality monitoring data into a three-dimensional tensor. Where P is the number of monitoring points, M is the number of pollutant categories, and T is the number of days. This represents the value of the m-th pollutant detected at the p-th monitoring point on day t.

[0096] Tensor construction unit 120 is used to construct data based on a given number of prediction days K, tensor dimension I2, and three-dimensional tensor. Constructing the air mass tensor

[0097] The tensor construction unit 120 is further configured to construct a weight tensor based on a given set Ω.

[0098] Optimize target modeling unit 130, used to model based on air mass tensor and weight tensor Construct the objective function f(A, B, C, D), and then obtain the optimization problem. Where A, B, C, and D are factor matrices;

[0099] Model training unit 140 is used to obtain estimates of the factor matrix using an alternating minimization algorithm.

[0100] Air quality prediction unit 150, used to predict air quality based on Calculate the values ​​of various pollutants at each monitoring point from day T+1 to day T+K.

[0101] Preferably, the tensor construction unit 120 is specifically used to... Expand to in 1≤p≤P, 1≤t≤T+K, 1≤m≤M;

[0102] Construct a four-dimensional tensor Let I1 = P and I4 = M, representing the number of monitoring points and pollutant categories to be predicted. I2 is set to a multiple of 7, such as 7, 14, 21, etc., and I2 > K. in This represents the floor function, which returns the largest integer less than or equal to a given number. Let T′ = T + K - I² × I³, Mapped to according to the following rules

[0103]

[0104] 1≤p≤P, T′+1≤t≤T+K, 1≤m≤M. “mod” represents the modulo operation, returning the remainder when one number is divided by another. This represents the floor function, which returns the smallest integer greater than or equal to the given number;

[0105] Construct weight tensor in The relationship between (i1, i2, i3, i4) and (p, t, m) is determined by the following mapping rule:

[0106]

[0107] 1≤p≤P, T′+1≤t≤T+K, 1≤m≤M.

[0108] Preferably, the optimization target modeling unit 130 is specifically used to construct the objective function. Where * denotes bitwise multiplication, and ||·||F denotes the Frobenius norm. It is a factor matrix. R is a positive integer called a tensor. CP rank. That is to put It can be decomposed into a sum of tensors of rank 1. It is a regularization term, here we let λ is a parameter used to control the effect of the regularization term on the objective function.

[0109] Based on this, the optimization problem can be obtained.

[0110] Preferably, the air quality prediction unit 150 is specifically used to obtain the predicted value of the m-th pollutant at the p-th monitoring point on day t according to the following formula:

[0111]

[0112] 1≤p≤P, T+1≤t≤T+K, 1≤m≤M. Wherein,

[0113] Those skilled in the art will further recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of both. To clearly illustrate the interchangeability of hardware and software, the components and steps of the various examples have been generally described in terms of functionality in the foregoing description. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementations should not be considered beyond the scope of this invention.

[0114] The steps of the methods or algorithms described in conjunction with the embodiments disclosed in this invention can be implemented in hardware, software modules executed by a processor, or a combination of both. The software modules can be located in random access memory (RAM), main memory, read-only memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, registers, hard disks, removable disks, CD-ROMs, or any other form of storage medium known in the art.

[0115] The above description is only a preferred embodiment of the present invention. It should be noted that for those skilled in the art, several improvements and modifications can be made without departing from the principle of the present invention, and these improvements and modifications should also be considered within the scope of protection of the present invention.

Claims

1. An air quality prediction method based on incomplete data, characterized in that, Includes the following steps: Step 1: Given 10 monitoring stations, including PM2.5 and PM10. Pollutants and Today, historical air quality monitoring data will be organized into a three-dimensional tensor. ,in Indicates the first Heaven, the first The first monitoring point detected the The values ​​of various pollutants; Step 2: Given a set It contains three-dimensional tensors The index of a known element, that is, an observable element, i.e., if If it is known, then ; Step 3: Given the number of days to predict and tensor dimension Construct the air mass tensor ; Step 4: Based on sets Construct weight tensors ; Step 5: Based on the air mass tensor and weight tensor Construct the objective function This leads to the optimization problem. ,in , , , It is a factor matrix; Step Six: Use the alternating minimization algorithm to obtain the estimated value of the factor matrix. , , , ; Step 7: Based on the estimated values ​​of the factor matrix , , , ,calculate Heaven The values ​​of various pollutants at each monitoring point in the sky; Solve the problem of air quality forecasting based on incomplete data; In step three, the given number of predicted days and tensor dimension Construct the air mass tensor Specifically, it includes: Will Expand to ,in , , , ; Construct a four-dimensional tensor To represent air quality monitoring data; here, , That is, the number of monitoring points and the number of pollutant types to be predicted; Set to a multiple of 7, and ; ,in This represents the floor function, which returns the largest integer less than or equal to a given number; let... , Mapped to according to the following rules : , ; In step four, the set-based approach... Construct weight tensors Specifically, it includes: Construct weight tensor ,in ; and The relationship between them is determined by the following mapping rules: , , , ; In step five, the air mass tensor-based... and weight tensor Construct the objective function This leads to the optimization problem. ,in , , , It is a factor matrix, specifically including: Construct the objective function ;in, This indicates digit-wise multiplication. Denotes the Frobenius norm; , , , It is a factor matrix; It is a positive integer, called a tensor. CP rank; That is to put It can be decomposed into a sum of tensors of rank 1; It is a regularization term, here we let ; It is a parameter used to control the effect of the regularization term on the objective function; Based on this, we obtain the optimization problem. .

2. The air quality prediction method based on incomplete data according to claim 1, characterized in that, In step seven, the statement based on , , , ,calculate Heaven In the sky, the values ​​of various pollutants at each monitoring point include: No. Heaven, the first The first monitoring point The predicted values ​​for the pollutants are obtained by the following formula: ， , , ;in, .

3. An air quality prediction system based on incomplete data, characterized in that, It includes a data processing unit, a tensor construction unit, an optimization objective modeling unit, a model training unit, and an air quality prediction unit; The data processing unit is used to organize historical air quality monitoring data into a three-dimensional tensor. ,in It is the number of monitoring points, It is the number of pollutant types. It is the number of days. Indicates the first Heaven, the first The first monitoring point detected the The values ​​of various pollutants; Tensor construction units are used to construct prediction days based on the number of prediction days given by the data processing unit. Tensor Dimension and three-dimensional tensor Construct the air mass tensor The tensor construction unit is also used to construct from a given set. Construct weight tensors ; Optimize the target modeling unit to provide the sum and air mass tensor based on the tensor construction unit. and weight tensor Construct the objective function This leads to the optimization problem. ,in , , , It is a factor matrix; The model training unit is used to obtain estimates of the factor matrix given by the optimization objective modeling unit using an alternating minimization algorithm. , , , ; Air quality prediction unit, used to predict air quality based on , , , ,calculate Heaven The values ​​of various pollutants at each monitoring point in the sky; The tensor construction unit, specifically, involves: converting a three-dimensional tensor... Expand to ,in , , , ; Construct a four-dimensional tensor To represent air quality monitoring data; here, , That is, the number of monitoring points and the number of pollutant types to be predicted; Set to a multiple of 7, and ; ,in This represents the floor function, which returns the largest integer less than or equal to a given number; let... , Mapped to according to the following rules : , , , "mod" represents the modulo operation, which returns the remainder when one number is divided by another. This represents the floor function, which returns the smallest integer greater than or equal to the given number; Construct weight tensor ,in ; and The relationship between them is determined by the following mapping rules: , , , ; The optimization objective modeling unit is specifically used to construct the objective function. ;in, This indicates digit-wise multiplication. Denotes the Frobenius norm; , , , It is a factor matrix; It is a positive integer, called a tensor. CP rank; That is to put It can be decomposed into a sum of tensors of rank 1; It is a regularization term, here we let ; It is a parameter used to control the impact of the regularization term on the objective function; based on this, the optimization problem is obtained. .

4. The air quality prediction system based on incomplete data according to claim 3, characterized in that, The air quality prediction unit is specifically used to obtain the following formula: Heaven, the first The first monitoring point Predicted values ​​for the pollutants: ， , , ;in, .

Images