Air quality correlation estimation method and system based on moment matching prior network
By using the method of using the moment-matched prior network in air quality prediction, independently decoupling the model and data uncertainty, the problems of high training costs and inaccurate uncertainty in the prior art are solved, and more efficient air Quality prediction and change understanding.
Patent Information
- Application Number
- CN202410784844.3
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-06-18
- Publication Date
- 2024-11-08
- Estimated Expiration
- 2044-06-18
AI Technical Summary
The existing air quality prediction methods require a large amount of out-of-distribution data to assist in training during training, resulting in high computational costs, inaccurate uncertainty estimates, and difficult to explain the correlations of different variables.
Using a method based on the moment matching prior network, the conjugated prior distribution form of the Gaussian likelihood function is independently decoupled model uncertainty and data uncertainty, and the covariance matrix in the predicted distribution is estimated by using moment matching to reduce training Cost and provide more accurate uncertainty estimates.
It realizes providing more accurate uncertainty estimates at low computational costs, helps understand and predict air quality changes, reduces dependence on external data, and improves model training speed and prediction accuracy.
Smart Images

Figure CN118917404A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field of data mining, and in particular to an air quality correlation estimation method and system based on a moment matching priori network. Background Art
[0002] Air quality prediction is an important modeling task and an important issue concerning public health and safety. It involves many livelihood areas such as agricultural production, water resource management, urban transportation planning, etc., and directly affects the quality of human life. However, there are many types of pollutants in the air (such as particulate matter, sulfide, etc.), and these gases are affected by spatial and temporal correlations. In addition, air quality is also affected by meteorological conditions, such as whether it is windy, whether it is raining, local temperature and other conditions. Therefore, combining multiple factors to predict air quality is a very worthy issue to study.
[0003] With the development of machine learning and deep learning, neural networks have shown obvious advantages in processing complex nonlinear relationships and have high prediction accuracy. At present, air quality prediction using neural networks has become one of the hot topics in the research field. However, the air quality prediction task belongs to the multivariate regression task. When the neural network is deployed in the multivariate regression task, uncertainty estimation is crucial for reliable prediction and decision-making.
[0004] At present, in order to make accurate uncertainty estimation in multivariate deep regression, Bayesian Neural Network (BNN) usually uses weighted posteriors to capture uncertainty. However, estimating the weighted posteriors of neural networks is usually intractable, resulting in a significant increase in computational and memory costs during training and inference. In order to reduce the cost of BNN, researchers in related fields have proposed evidence methods that use deep neural networks to directly predict the parameters of evidence priors, efficiently capturing uncertainty with a single network and a single forward propagation. However, for cases where the output dimension of the neural network increases significantly, these methods are limited to (when the dimension is less than 2) the estimation of the diagonal oblique variance matrix. These methods require a large amount of out-of-distribution data to assist in training to facilitate the quantification of epistemic uncertainty, which usually results in the training process of existing evidence methods being generally numerically unstable and heavily relying on complex regularization techniques, resulting in inaccurate air quality correlation estimation results, affecting the accuracy of subsequent air quality prediction results. Summary of the invention
[0005] 1. Technical issues to be solved
[0006] In view of the shortcomings of the prior art, the present invention provides an air quality correlation estimation method and system based on a moment matching prior network, which solves the technical problem that the prior method requires a large amount of out-of-distribution data to assist in training to promote the quantification of cognitive uncertainty. It can also explain the correlation between different variables.
[0007] (II) Technical solution
[0008] To achieve the above objectives, the present invention is implemented through the following technical solutions:
[0009] In a first aspect, the present invention provides an air quality correlation estimation method based on a moment matching prior network, by pre-constructing a correlation estimation model based on a moment matching prior network to obtain the correlation between different variables in an air quality data set, and predicting changes in air quality, wherein the process of constructing a correlation estimation model based on a moment matching prior network includes:
[0010] Acquire and preprocess multivariate air quality datasets from monitoring stations;
[0011] The correlation estimation model based on the moment matching prior network is obtained by training the moment matching prior network with the preprocessed multivariate air quality data set.
[0012] Among them, the conjugate prior distribution form of the Gaussian likelihood function in the correlation estimation model based on the moment matching prior network is Among them, Λ0 represents arbitrary uncertainty, Σ represents the model uncertainty of the correlation estimation model, the model uncertainty is determined by the covariance of the correlation estimation model, Λ represents the point estimate of data uncertainty, represents the predicted value of the deep model considering the unmodeled data noise, μ represents the expected value of the correlation estimation model prediction, and the expectation is determined by the average value of the predicted value of the correlation estimation model.
[0013] Preferably, the multivariate air quality data set of the pre-processed monitoring station includes:
[0014] Perform data cleaning, missing value filling and outlier detection on the data in the multivariate air quality dataset;
[0015] According to the characteristics of air quality data, the window size of the time series data prediction task is determined, and the multivariate air quality data set is divided according to the window size to obtain several window data;
[0016] Normalize the data in each window.
[0017] Preferably, the calculation method of the model uncertainty of the correlation estimation model includes:
[0018]
[0019] Σ sample =aΣ low +bΣ diag
[0020]
[0021] Among them, S represents the number of samples generated by the deep neural network in moment matching, represents the predicted value of the model corresponding to the sample, represents the sample set, Σ sample represents the covariance estimate based on these deep samples; Σ(;θ) represents the estimate of the model uncertainty Σ in the prediction distribution of the parameterized θ deep neural network, a and b represent weight parameters, Σ low (x; θ) represents the sample covariance matrix of rank S, Σ diag A diagonal matrix representing the variance calculation for each dimension of air quality.
[0022] Preferably, the expected calculation method of the correlation estimation model prediction value includes:
[0023]
[0024] Among them, S represents the number of samples generated by the deep neural network in moment matching, represents the predicted value of the model corresponding to the sample, and u(x;θ) represents the estimate of the expected μ in the predictive distribution of the parameterized θ deep neural network.
[0025] Preferably, the expression of the point estimate Λ of data uncertainty includes:
[0026]
[0027] When Λ is homoscedastic, the parameter θ of the correlation estimation model is related to the variable σ 2 are all considered as trainable parameters. When Λ is heteroscedastic, the neural network is required to In addition, data uncertainty σ(x;θ) is also output, σ 2 Predicted by the deep model, I represents the unit diagonal matrix, σ 2 (x)I, indicates the correlation with the input data x, σ 2 I represents a scalar that is independent of the input data.
[0028] Preferably, during the training process of the correlation estimation model based on the moment matching prior network, the parameters of the model are optimized by maximizing the likelihood function, and the model is trained using a stochastic gradient descent method until convergence.
[0029] Preferably, the expression of the likelihood function includes:
[0030]
[0031]
[0032] Among them, θ represents the parameters of the correlation estimation model, and y represents the true value of air quality.
[0033] In a second aspect, the present invention provides an air quality correlation estimation system based on a moment matching prior network, wherein the air quality correlation estimation system obtains the correlation between different variables of an air quality data set by pre-building a correlation estimation model module based on a moment matching prior network, and predicts changes in air quality, wherein the pre-building a correlation estimation model module based on a moment matching prior network includes:
[0034] A data processing unit for acquiring and preprocessing multivariate air quality data sets from monitoring stations;
[0035] A model training unit, used for training a moment matching prior network by using a preprocessed multivariate air quality data set to obtain a correlation estimation model based on the moment matching prior network;
[0036] Among them, the conjugate prior distribution form of the Gaussian likelihood function in the correlation estimation model based on the moment matching prior network is Among them, Λ0 represents arbitrary uncertainty, Σ represents the model uncertainty of the correlation estimation model, the model uncertainty is determined by the covariance of the correlation estimation model, Λ represents the point estimate of data uncertainty, represents the predicted value of the deep model considering the unmodeled data noise, μ represents the expected value of the correlation estimation model prediction, and the expectation is determined by the average value of the predicted value of the correlation estimation model.
[0037] In a third aspect, the present invention provides a computer-readable storage medium storing a computer program for air quality correlation estimation based on a moment matching prior network, wherein the computer program enables a computer to execute the air quality correlation estimation method based on a moment matching prior network as described above.
[0038] In a fourth aspect, the present invention provides an electronic device, comprising:
[0039] One or more processors, a memory, and one or more programs, wherein the one or more programs are stored in the memory and are configured to be executed by the one or more processors, and the programs include methods for executing the air quality correlation estimation method based on the moment matching prior network as described above.
[0040] (III) Beneficial effects
[0041] The present invention provides an air quality correlation estimation method and system based on a moment matching prior network. Compared with the prior art, it has the following beneficial effects:
[0042] The present invention selects the conjugate prior distribution form of the Gaussian likelihood function as In this conjugate prior distribution form, model uncertainty Σ and data uncertainty Λ are independent of each other, decoupling the connection between epistemic uncertainty and arbitrary uncertainty. At the same time, the embodiment of the present invention introduces a new modeling method to implicitly learn the evidence prior form {μ, Σ, Λ0}, which is different from the traditional method of learning model parameters. The deep neural network does not directly model the parameters in the prediction distribution, but uses moment matching to estimate the parameters in the prediction distribution. There is no need to introduce a large amount of out-of-distribution data, which can reduce training costs, speed up training, provide more accurate uncertainty estimates, and help researchers better understand and predict air quality changes. BRIEF DESCRIPTION OF THE DRAWINGS
[0043] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.
[0044] Figure 1 A schematic diagram of a process for constructing a correlation estimation model based on a moment matching prior network in an embodiment of the present invention;
[0045] Figure 2 A visualization diagram of the moment matching prior network;
[0046] Figure 3 A schematic diagram of the results of correlation estimation of air quality data sets using a correlation estimation model based on a moment matching prior network;
[0047] Figure 4 Uncertainty calibration curves for different methods in air quality data. DETAILED DESCRIPTION
[0048] In order to make the purpose, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention are clearly and completely described. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.
[0049] The embodiments of the present application provide an air quality correlation estimation method and system based on a moment matching prior network, which solves the technical problem that the existing methods require a large amount of out-of-distribution data to assist in training to promote the quantification of cognitive uncertainty. It provides more accurate uncertainty estimation at a relatively low computational cost, helping researchers to better understand and predict air quality changes.
[0050] The technical solution in the embodiment of the present application is to solve the above technical problems, and the overall idea is as follows:
[0051] Deep neural networks have demonstrated impressive performance on a variety of regression problems, where accurate uncertainty estimation plays a key role in model prediction, calibrating model confidence intervals, and expert optimal decision-making. Especially in high-risk scenarios, uncertainty estimation is often more important than accuracy, thereby preventing rare but costly accidents. Although significant progress has been made in the field of univariate deep regression, uncertainty estimation in multivariate deep regression tasks is still relatively rare, mainly due to the high-dimensional covariance matrix in multivariate regression tasks. Take the air quality prediction task as an example. This multivariate regression task is particularly important for government policymakers and decision makers, who rely on air quality predictions to take preventive measures. Accurate uncertainty estimation can provide confidence intervals for model predictions, help decision makers assess the reliability of current models, and take appropriate measures at the community level. On the other hand, air quality is affected by many factors, including meteorological conditions, human activities, and seasonal changes. Air quality prediction under the influence of multiple factors is inherently challenging. Reliable deep regression algorithms can identify the uncertainties introduced by these factors, which will help risk control and management, thereby promoting the development of science and technology.
[0052] The present embodiment introduces a Moment Matching Prior Network (MMPN), which uses new evidential priors when capturing arbitrary and epistemic uncertainties and can characterize the correlation of multivariate response variables.
[0053] In order to better understand the above technical solution, the above technical solution will be described in detail below in conjunction with the accompanying drawings and specific implementation methods.
[0054] The embodiment of the present invention provides an air quality correlation estimation method based on a moment matching prior network. The method obtains the correlation between different variables in an air quality data set by pre-constructing a correlation estimation model based on a moment matching prior network, and predicts the change of air quality. The process of constructing the correlation estimation model based on a moment matching prior network is as follows: Figure 1 As shown, including:
[0055] S1. Obtain and preprocess the multivariate air quality dataset of monitoring stations;
[0056] S2. The moment matching prior network is trained by preprocessing the multivariate air quality data set to obtain a correlation estimation model based on the moment matching prior network. The conjugate prior distribution form of the Gaussian likelihood function in the correlation estimation model based on the moment matching prior network is: Among them, Λ0 represents arbitrary uncertainty, Σ represents the model uncertainty of the correlation estimation model, the model uncertainty is determined by the covariance of the correlation estimation model, Λ represents the point estimate of data uncertainty, represents the predicted value of the deep model considering the unmodeled data noise, μ represents the expected value of the correlation estimation model prediction, and the expectation is determined by the average value of the predicted value of the correlation estimation model.
[0057] The embodiment of the present invention selects the conjugate prior distribution form of the Gaussian likelihood function as follows: The problem of uncertainty estimation of deep learning models in the face of complex data is innovatively solved. In the conjugate prior distribution form, model uncertainty Σ and data uncertainty Λ are independent of each other, which effectively decouples the relationship between epistemic uncertainty (uncertainty of the model's understanding of the data) and arbitrary uncertainty (uncertainty of data noise), which helps experts in related fields analyze and understand the predictive behavior of deep models. At the same time, the embodiment of the present invention introduces a new modeling method to implicitly learn the evidence prior form {μ, Σ, Λ0}. Compared with the traditional method of learning model parameters before, this method has significant advantages. The deep neural network does not directly model the parameters in the predictive distribution, but uses moment matching to estimate the covariance matrix (including the potential connection between different variables) in the predictive distribution without introducing a large amount of out-of-distribution data. This method not only reduces the training cost and speeds up the training speed of the deep model, but also provides more accurate uncertainty estimation. By reducing the dependence on external data and optimizing the training process of the model, it provides a more efficient and accurate uncertainty processing mechanism for deep learning models. This not only enables the model to quickly learn complex data distributions, but also improves the accuracy and reliability of the model's prediction of unknown data. In summary, the examples of the present invention provide a powerful tool for researchers in applied fields such as environmental science to help them more accurately understand and predict the behavior of complex systems, especially in the key field of air quality prediction, and also provide substantial support for public health and environmental protection.
[0058] The following is a detailed description of each step:
[0059] In one embodiment, S1, obtaining and preprocessing a multivariate air quality data set of a monitoring station. The specific implementation process is as follows:
[0060] S101. Obtain a multivariate air quality data set from the monitoring station, including:
[0061] The data dimension in the data set is 12 dimensions, including weather parameters such as time, wind speed, temperature, and concentration indicators related to various air pollutants such as PM2.5 and NO2. In the embodiment of the present invention, there are 43,824 data in total, and the time dimension includes data from March 2004 to February 2005 for nearly a year. The monitoring stations of these data sets are placed in some seriously polluted areas. The scale of the data set is large enough to effectively and comprehensively test the performance of the correlation estimation model based on the moment matching prior network.
[0062] S102, preprocessing the multivariate air quality data set, specifically including:
[0063] After the data in the multivariate air quality dataset is cleaned, missing values are filled, and outliers are detected, the window size of the time series data prediction task is determined based on the characteristics of the air quality data. For example, if 7 days of time series data are used to predict tomorrow's value, the window size is selected as 7. The step size determines the interval at which the window moves and is generally set to 1. This processing method means that the sliding window moves forward one time step each time. After integrating the window data in the time series prediction data, the time series segments in each window are normalized or standardized to obtain the training set. This helps reduce the model’s reliance on specific time features and further improves performance. i represents the i-th window data after segmentation, x i Contains all historical data in the i-th window data. Through this sliding window, MMPN ensures that the sample x i The assumption that they are independent and identically distributed is satisfied, y i represents the true value of the air quality in the i-th window, and N represents the total number of windows.
[0064] Finally, the data is randomly divided into training, validation, and test sets (60%, 20%, 20%). The validation set is only used to tune hyperparameters, and the dataset is scaled to [0, 1] by subtracting the minimum value of each feature and dividing by the range of values. This avoids outliers and improves the numerical stability of the training process.
[0065] In one embodiment, S2, a moment matching prior network is trained by using the preprocessed multivariate air quality data set to obtain a correlation estimation model based on the moment matching prior network. The specific implementation process is as follows:
[0066] In an embodiment of the present invention, in order to more effectively process air quality data sets, a Moment-Matching Prior Network (MMPN) is proposed to model uncertainty prediction tasks in air quality data sets. MMPN is a scalable evidence probability framework in which a deep model learns evidence prior distributions by means of moment matching, effectively combining accurate predictions with uncertainty estimates. Specifically, MMPN can provide accurate prediction results and efficient uncertainty estimates (i.e., the interaction of different dimensions in multivariate prediction regression tasks) using a single forward propagation on a single neural network. Moment-Matching Prior Network is a deep neural network.
[0067] In the correlation estimation model based on the moment matching prior network, the evidence prior forms include:
[0068] The conjugate prior distribution of the Gaussian likelihood function is selected as This modeling approach has two advantages:
[0069] (1) It decouples the relationship between epistemic uncertainty and arbitrary uncertainty (i.e., model uncertainty Σ (uncertainty in the model’s understanding of the data) and data uncertainty Λ (randomness or noise in the data itself) are independent of each other), thus enhancing the numerical stability of the model optimization process. The embodiments of the present invention provide a more accurate and reliable framework for deep learning models, which facilitates the application and generalization of models in complex environments (e.g., air quality prediction).
[0070] (2) Since the correlation of the predicted variable y is characterized by epistemic uncertainty Σ. In the specific implementation process, point estimation is used to model data uncertainty Λ to reduce additional computational overhead and storage space. The point estimation modeling method not only simplifies the model structure, but also effectively captures data uncertainty, thereby improving the accuracy and credibility of model predictions. For scenarios that require processing large-scale data sets, this discovery is of great significance for designing and implementing efficient and accurate deep learning models.
[0071] In the specific implementation process, in the correlation estimation model based on the moment matching prior network, μ and Σ are estimated in the following way:
[0072] The deep neural network takes the feature vector x corresponding to the time series data as input, and the model generates a series of deep samples (That is, the output dimension of the network is D×S), where D represents the dimension predicted by the model and S represents the number of samples required in the moment matching estimation. The model then obtains estimates of the parameters μ and Σ in the predicted distribution of the air quality dataset by matching the first-order and second-order moments of these depth samples. The specific operation formula is as follows:
[0073]
[0074]
[0075] Where S represents the number of samples generated by the deep neural network in moment-matching. Represents the predicted value of the model corresponding to these samples. Σ sample Represents the covariance estimate generated based on these depth samples, which can be used to analyze the correlation between predicted values. u(x; θ) and Σ(; θ) respectively represent the estimates of the mean μ (the average value of the model prediction value approximates the expectation of the predicted distribution) and covariance Σ (that is, model uncertainty) in the predicted distribution of the parameterized θ neural network. These estimates are based on the sample set generated from the model. To obtain, specifically, Σ sample The calculation of includes two key parts Σ low and Σ diag , these two components are further elaborated and explained in the following paragraphs.
[0076] In order to further simplify the computational overhead, MMPN uses a low-rank diagonal approximation method to estimate the uncertainty in the air quality prediction task. The sample covariance matrix of rank S is specifically defined as follows:
[0077]
[0078] In order to avoid the problem of numerical instability during training (to ensure the positive definite property of the uncertainty matrix Σ), the embodiment of the present invention additionally uses the variance of each dimension in the air quality to calculate the corresponding diagonal matrix as a preliminary estimate of the data uncertainty:
[0079]
[0080] First, the air quality task is estimated to be Σ diag With Σ low Combined, the specific analytical form of the covariance matrix in MMPN is as follows:
[0081]
[0082] During the training process, the two matrices Σ low With Σ diag Being set to the same weights, MMPN can also adjust these weights to obtain better performance in deep models.
[0083] In the air quality task, arbitrary uncertainty Λ0 is set as a scalar, that is, the data noise between different dimensions of the predicted variable y is characterized by arbitrary uncertainty Σ. The MMPN method divides arbitrary uncertainty into homoscedastic and heteroscedastic uncertainty. Specifically, data uncertainty Λ can be formally defined as:
[0084]
[0085] When Λ is homoscedastic, we associate the model parameters θ with the variables σ 2 are all considered as trainable parameters. When Λ is heteroskedastic, we require the neural network to In addition, the model uncertainty σ(x;θ) is also output.
[0086] In order to more efficiently and comprehensively model the data uncertainty Λ, the embodiment of the present invention considers the homoscedasticity and heteroscedasticity cases respectively (as shown in the above formula), where σ 2 It is predicted by the deep model, and I represents the unit diagonal matrix. Specifically, σ 2 (x)I, indicates the correlation with the input data x, σ 2 I represents a scalar that is independent of the input data, and σ 2 Combined with I, it ensures the positive definiteness of the data uncertainty, which is crucial for maintaining numerical stability during training.
[0087] Uncertainty can be generally divided into two parts: data uncertainty Λ and model uncertainty Σ, whose sum represents the final uncertainty estimate (i.e. Λ+Σ). The data uncertainty Λ is determined by the parameter σ 2 It is determined together with the unit matrix I (as shown above), which represents any kind of measurement error (such as missing deletion, etc.) in the process of data collection and processing, while the other part of the model uncertainty Σ is obtained through moment matching, which is composed of Σ low With Σ diag Two parts (i.e. Σ=Σ low +Σ diagg ). Therefore, the overall uncertainty calculation involves these three key parameters: 2 ,Σ low ,Σ diag}.
[0088] Based on the correlation estimation model training process of the moment matching prior network, the parameters of the deep model are optimized by maximizing the likelihood function, and the model is trained by stochastic gradient descent until convergence. For example, a total of 50 rounds of training are performed on the entire data set, and the stochastic gradient descent method is used to maximize the likelihood function as the objective function to optimize the parameters of the deep model.
[0089]
[0090] However, since the neural network needs to minimize the objective function when optimizing using the gradient descent method in the optimization process, the optimization objective is changed to minimizing the negative log-likelihood.
[0091] Air quality dataset The negative log-likelihood function L(θ) on can be expressed as:
[0092]
[0093] Where c is a constant, and {μ,Σ,Λ0} can be obtained in turn through the above equations. According to the formula, different types of uncertainty can be modeled as the uncertainty of the overall distribution (uncertainty of the predicted distribution) is Σ+Λ0, which consists of epistemic uncertainty Σ and arbitrary uncertainty Λ0.
[0094] Using negative log-likelihood (NLL) as the objective function for model training includes some key benefits. Negative log-likelihood measures the difference between the probability distribution predicted by the model and the probability distribution of the actual time series data generation process. Minimizing this difference makes the model's predicted distribution as close to the true data distribution as possible, thereby helping the deep model to model the predicted distribution of the data generation process, providing richer information for the interpretation and evaluation of the model, and improving the generalization ability of over-parameterized deep models.
[0095] The visualization of the moment matching prior network is as follows Figure 2 As shown in the figure. The evidence prior indicates that the neural network generates multiple samples and estimates the parameters of the evidence prior {,Σ} by matching the first-order and second-order moments; the likelihood function indicates the prediction result of the model after introducing data uncertainty Λ0. It can be seen from the overall likelihood function that the final prediction distribution of the moment matching network obeys the Gaussian distribution, and can obtain a closed-form solution through the conjugate property and produce flexible prediction results.
[0096] At the same time, it should be noted that the moment matching prior network of the embodiment of the present invention can effectively reduce the output dimension. From the above, it can be seen that the estimation of model uncertainty Σ sample By Σ diag With Σ low The diagonal matrix includes only the variance values of each dimension on the diagonal, so the dimension is D, where D represents the dimension of the multivariate target variable y. Since the covariance matrix is symmetric, the other part of the covariance estimate Σ low We only need to predict the value of the upper triangular part, so the output dimension is The remaining dimension 2 is used to represent the point estimate of data uncertainty. That is, the output dimension of the existing method is In contrast, the output dimension of MMPN only needs D(S+1)+1, which greatly reduces the output dimension of the model. Because MMPN uses moment estimation, it only needs to output S D-dimensional prediction vectors to estimate the covariance matrix Σ sample , using a single D-dimensional vector to predict the value μ sample , and the last dimension represents a point estimate of the uncertainty in the data. Compared with D(S+1)+1, the advantages of the MMPN method can be clearly seen.
[0097] Therefore, as the dimension increases, MMPN significantly reduces the output dimension compared to other baseline methods. This modeling approach can help deep models effectively characterize the relationship between multivariate variables and further solve the optimization challenges of over-parameterized deep models in high-dimensional space.
[0098] The effectiveness of the embodiments of the present invention is verified by verification experiments below:
[0099] The present invention verifies the effectiveness of the present invention through negative log likelihood (NLL), root mean square error (RMSE) and empirical correlation coefficient (CORR), where the higher the CORR index, the better, and the lower the other indexes, the better. The calculation of RMSE is as follows:
[0100]
[0101] Among them, y and μ(x i ,θ) represents the true value and depth sample of air quality prediction Another widely used indicator, CORR, can be used to measure the relationship between the true distribution and the predicted variable. Specifically, it can be calculated as follows:
[0102]
[0103] in, and are the true value and predicted value on the test set. The negative log likelihood is calculated by predicting the distribution. As shown in Table 1, in the air quality prediction task, MMPN (the method of the embodiment of the present invention) always outperforms other methods in the above indicators. Therefore, MMPN can accurately estimate the prediction uncertainty of the air quality dataset using a single forward pass on a single network, highlighting the effectiveness and flexibility of using moment matching to learn deep samples.
[0104] Table 1 Comparison of performance of different methods for predicting air quality
[0105]
[0106] Air Quality Task Analysis
[0107] like Figure 2 As shown in Figure 2, in order to analyze the covariance estimation of MMPN more intuitively, we visualize the connection between different variables in the multivariate regression (uncertainty estimation results) on the air quality dataset to demonstrate the effect of the MMPN algorithm. MMPN can accurately capture the strong correlation between atmospheric chemicals, toxic gas concentrations, and temperature.
[0108] exist Figure 3 In the results, it can be observed that MMPN can accurately capture the correlations between different variables in the air quality dataset: (1) The first nine air quality detection indicators show significant positive correlations, reflecting the high correlation between the concentrations of chemicals and toxic gases in the air. (2) In particular, PTO8.S1 and PTO8.S4 (hourly average oxide response) are consistent with the estimated values of CO and NO2 concentrations, respectively. (3) Temperature is negatively correlated with chemicals and toxic gases in the air, that is, lower temperatures may lead to higher concentrations of toxic gases. This qualitative analysis verifies the importance of capturing correlations between multivariate prediction vectors and demonstrates the effectiveness of MMPN in achieving this goal.
[0109] like Figure 4 To further compare the effectiveness of the MMPN method for uncertainty estimation in air quality tasks, Figure 4 In the figure, the uncertainty calibration curves of different methods in air quality data are further compared. It can be observed that the work of other uncertainty estimation methods often shows an overconfidence effect, while the method of the embodiment of the present invention is significantly better than other methods (closest to the diagonal line) and shows the best calibration performance.
[0110] In order to better test the effectiveness of the MMPN method at different scales on the air quality dataset, this verification process explores the impact of the number of samples S on the performance of the MMPN. Table 2 shows that as the number of samples increases, the performance of the larger-scale MMPN algorithm does not increase monotonically. This may be because increasing the number of samples will bring more accurate sample moments, but it also increases the challenge and difficulty of the optimization problem. In general, it can be observed that placing samples in the range of 3-6 will produce good optimization effects and prediction performance.
[0111] Table 2 Performance of MMPN at different scales on air quality datasets
[0112]
[0113] It should be noted that, in the specific implementation process, the correlation estimation model based on the moment matching prior network proposed in the embodiment of the present invention can also be applied to other data, such as analyzing the correlation between shared bicycle usage data and urban weather and seasons, so as to facilitate the subsequent prediction of shared bicycle usage data according to seasons and weather, and facilitate merchants to make decisions on the data of deploying bicycles.
[0114] The embodiment of the present invention further provides an air quality correlation estimation system based on a moment matching prior network, which obtains the correlation between different variables in an air quality data set by pre-building a correlation estimation model module based on a moment matching prior network, and predicts changes in air quality, wherein the pre-building a correlation estimation model module based on a moment matching prior network includes:
[0115] A data processing unit for acquiring and preprocessing multivariate air quality data sets from monitoring stations;
[0116] A model training unit, used for training a moment matching prior network by using a preprocessed multivariate air quality data set to obtain a correlation estimation model based on the moment matching prior network;
[0117] Among them, the conjugate prior distribution form of the Gaussian likelihood function in the correlation estimation model based on the moment matching prior network is Among them, Λ0 represents arbitrary uncertainty, Σ represents the model uncertainty of the correlation estimation model, the model uncertainty is determined by the covariance of the correlation estimation model, Λ represents the point estimate of data uncertainty, represents the predicted value of the deep model considering the unmodeled data noise, μ represents the expected value of the correlation estimation model prediction, and the expectation is determined by the average value of the predicted value of the correlation estimation model.
[0118] It can be understood that the air quality correlation estimation system based on the moment matching prior network provided in the embodiment of the present invention corresponds to the above-mentioned air quality correlation estimation method based on the moment matching prior network. The explanations, examples, beneficial effects and other parts of the relevant contents can refer to the corresponding contents in the air quality correlation estimation method based on the moment matching prior network, and will not be repeated here.
[0119] An embodiment of the present invention also provides a computer-readable storage medium, which stores a computer program for air quality correlation estimation based on a moment matching prior network, wherein the computer program enables a computer to execute the air quality correlation estimation method based on a moment matching prior network as described above.
[0120] An embodiment of the present invention further provides an electronic device, including:
[0121] one or more processors;
[0122] Memory; and
[0123] One or more programs, wherein the one or more programs are stored in the memory and are configured to be executed by the one or more processors, the programs including instructions for executing the air quality correlation estimation method based on the moment matching prior network as described above.
[0124] In summary, compared with the prior art, the present invention has the following beneficial effects:
[0125] The embodiment of the present invention selects the conjugate prior distribution form of the Gaussian likelihood function as follows: The problem of uncertainty estimation of deep learning models in the face of complex data is innovatively solved. In the conjugate prior distribution form, model uncertainty Σ and data uncertainty Λ are independent of each other, which effectively decouples the relationship between epistemic uncertainty (uncertainty of the model's understanding of the data) and arbitrary uncertainty (uncertainty of data noise), which helps experts in related fields analyze and understand the predictive behavior of deep models. At the same time, the embodiment of the present invention introduces a new modeling method to implicitly learn the evidence prior form {μ, Σ, Λ0}. Compared with the traditional method of learning model parameters before, this method has significant advantages. The deep neural network does not directly model the parameters in the predictive distribution, but uses moment matching to estimate the covariance matrix (including the potential connection between different variables) in the predictive distribution without introducing a large amount of out-of-distribution data. This method not only reduces the training cost and speeds up the training speed of the deep model, but also provides more accurate uncertainty estimation. By reducing the dependence on external data and optimizing the training process of the model, it provides a more efficient and accurate uncertainty processing mechanism for deep learning models. This not only enables the model to quickly learn complex data distributions, but also improves the accuracy and reliability of the model's prediction of unknown data.
[0126] It should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the existence of other identical elements in the process, method, article or device including the elements.
[0127] The above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit the same. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that the technical solutions described in the aforementioned embodiments may still be modified, or some of the technical features may be replaced by equivalents. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.
Claims
1. A method for estimating air quality correlation based on a moment matching prior network, characterized in that: By pre-building a correlation estimation model based on a moment matching prior network, the correlation between different variables in the air quality data set is obtained to predict changes in air quality. The process of building a correlation estimation model based on a moment matching prior network includes: Acquire and preprocess multivariate air quality datasets from monitoring stations; The correlation estimation model based on the moment matching prior network is obtained by training the moment matching prior network with the preprocessed multivariate air quality data set. Among them, the conjugate prior distribution form of the Gaussian likelihood function in the correlation estimation model based on the moment matching prior network is Among them, Λ0 represents arbitrary uncertainty, Σ represents the model uncertainty of the correlation estimation model, the model uncertainty is determined by the covariance of the correlation estimation model, Λ represents the point estimate of data uncertainty, represents the predicted value of the deep model considering the unmodeled data noise, μ represents the expected value of the correlation estimation model prediction, and the expectation is determined by the average value of the predicted value of the correlation estimation model.
2. The air quality correlation estimation method based on moment matching prior network according to claim 1, characterized in that: The multivariate air quality dataset of the pre-processed monitoring stations includes: Perform data cleaning, missing value filling and outlier detection on the data in the multivariate air quality dataset; According to the characteristics of air quality data, the window size of the time series data prediction task is determined, and the multivariate air quality data set is divided according to the window size to obtain several window data; Normalize the data in each window.
3. The air quality correlation estimation method based on moment matching prior network according to claim 1, characterized in that: The calculation method of the model uncertainty of the correlation estimation model includes: ∑ sample =a∑ low +b∑ diag Among them, S represents the number of samples generated by the deep neural network in moment matching, represents the predicted value of the model corresponding to the sample, represents the sample set, ∑ sample represents the covariance estimate based on these deep samples; ∑(x; θ) represents the estimate of the model uncertainty ∑ in the prediction distribution of the parameterized θ deep neural network, a and b represent weight parameters, ∑ low (x; θ) represents the sample covariance matrix of rank S, ∑ diag A diagonal matrix representing the variance calculation for each dimension of air quality.
4. The air quality correlation estimation method based on moment matching prior network according to claim 1, characterized in that: The expected calculation method of the correlation estimation model prediction value includes: Among them, S represents the number of samples generated by the deep neural network in moment matching, represents the predicted value of the model corresponding to the sample, and u(x;θ) represents the estimate of the expected μ in the predictive distribution of the parameterized θ deep neural network.
5. The air quality correlation estimation method based on moment matching prior network according to claim 1, characterized in that: The point estimate of data uncertainty Λ is expressed as: When Λ is homoscedastic, the parameter θ of the correlation estimation model is related to the variable σ 2 are all considered as trainable parameters. When Λ is heteroscedastic, the neural network is required to In addition, data uncertainty σ(x;θ) is also output, σ 2 Predicted by the deep model, I represents the unit diagonal matrix, σ 2 (x)I, which is related to the input data x, σ 2 I represents a scalar that is independent of the input data.
6. The air quality correlation estimation method based on moment matching prior network according to any one of claims 1 to 5, characterized in that: During the training process, the correlation estimation model based on the moment matching prior network optimizes the parameters of the model by maximizing the likelihood function, and trains the model using a stochastic gradient descent method until convergence.
7. The air quality correlation estimation method based on moment matching prior network according to claim 6, characterized in that: The expression of the likelihood function includes: Among them, θ represents the parameters of the correlation estimation model, and y represents the true value of air quality.
8. An air quality correlation estimation system based on moment matching prior network, characterized in that: The air quality correlation estimation system obtains the correlation between different variables in the air quality data set by pre-building a correlation estimation model module based on a moment matching prior network, and predicts the change of air quality, wherein the pre-building correlation estimation model module based on a moment matching prior network includes: A data processing unit for acquiring and preprocessing multivariate air quality data sets from monitoring stations; A model training unit, used for training a moment matching prior network by using a preprocessed multivariate air quality data set to obtain a correlation estimation model based on the moment matching prior network; Among them, the conjugate prior distribution form of the Gaussian likelihood function in the correlation estimation model based on the moment matching prior network is Among them, Λ0 represents arbitrary uncertainty, Σ represents the model uncertainty of the correlation estimation model, the model uncertainty is determined by the covariance of the correlation estimation model, Λ represents the point estimate of data uncertainty, represents the predicted value of the deep model considering the unmodeled data noise, μ represents the expected value of the correlation estimation model prediction, and the expectation is determined by the average value of the predicted value of the correlation estimation model.
9. A computer-readable storage medium, characterized in that: It stores a computer program for estimating air quality correlation based on a moment matching a priori network, wherein the computer program enables a computer to execute the air quality correlation estimation method based on a moment matching a priori network as claimed in any one of claims 1 to 7.
10. An electronic device, characterized in that: include: One or more processors, a memory, and one or more programs, wherein the one or more programs are stored in the memory and are configured to be executed by the one or more processors, and the programs include a method for executing the air quality correlation estimation method based on the moment matching prior network as described in any one of claims 1 to 7.
Citation Information
Patent Citations
Rockburst grade predicting method based on information vector machine
CN104732070A
Cancer gene prognosis screening method and system based on improved Cox model
CN115620808A
Air quality data anomaly detection model based on DBN-OCSVM
CN116861232A
Deep neural network hyperparameter optimization method, electronic device and storage medium
WO2021007812A1
Cited By
Cerebral arterial thrombosis prediction system based on big data
CN120260924A
