Space-time interpolation neural network model and spatio-temporal data modeling method based on same

Abstract

The invention discloses a spatio-temporal interpolation neural network model and a spatio-temporal data modeling method based on the spatio-temporal interpolation neural network model, relates to the technical field of spatial interpolation, and aims to accurately model spatial dependence by introducing an embedded layer of spatial coordinates and a spatio-temporal basis function so as to establish a direct relation between DNN and Kriging spatial prediction. By means of a deep neural network structure, the model can automatically extract features in large-scale and dynamically-changing spatial data, and effectively identify a nonlinear relationship between spatial points, thereby realizing more accurate spatial interpolation. The method is not only suitable for non-linear prediction with covariables, but also can process non-Gaussian or non-stationary data, and greatly improves the applicability and robustness of the model. Besides, the deep learning model is excellent in performance on solving the local grid edge matching problem, the edge matching area can be smoothed, and the accuracy and reliability of gridding processing are improved.

Description

Technical Field

[0001] This invention belongs to the field of spatiotemporal interpolation technology, specifically relating to a spatiotemporal interpolation neural network model and a spatiotemporal data modeling method based thereon. Background Technology

[0002] The equidistant grid subdividing coding model is a model developed based on the GeoSOT (Geographic Coordinate Subdividing Grid with One Dimension Integral Coding on a 2n Tree) Earth subdivision framework. Due to its seamless, non-overlapping global latitude and longitude coverage, GeoSOT grids can achieve multi-scale representation and efficient coding computation. Currently, it has been widely used in disaster reduction and emergency response, aerospace remote sensing, land surveying, smart cities, and smart agriculture. It has been incorporated into national and military standards and has played a role in the development of international standards by organizations such as the Open Geospatial Consortium (OGC), the International Organization for Standardization (ISO), and the Institute of Electrical and Electronics Engineers (IEEE).

[0003] In the equidistant grid partitioning and coding model, equidistant grid data from different zones, regions, or scales exhibit discontinuities and significant local differences at their boundaries (border areas) due to differences in coordinate system uniformity, data alignment, and local consistency. This phenomenon is termed the "grid border problem" in this invention. Figure 1 As shown, the green grid matrix represents the equidistant grid of Beijing, the orange grid matrix represents the equidistant grid of Tianjin, and the red part represents the overlapping area of ​​the two equidistant grids. The core of solving the grid edge problem lies in achieving a smooth transition and seamless integration of data in the edge area.

[0004] The core challenge of cross-regional grid data fusion lies in eliminating data discontinuities caused by different spatial scales, grid partitioning rules, and coordinate system selection. These discontinuities not only affect data integrity and consistency but also hinder subsequent analysis and applications. Spatial interpolation techniques support cross-regional data fusion by mapping discrete observation data to a continuous spatial distribution. Traditional geostatistical interpolation methods place high demands on the normality and spatial stationarity of the data, making them computationally complex and requiring tedious parameter tuning when processing large-scale or cross-regional data, resulting in low efficiency in large-scale applications. With the continuous development of artificial intelligence technology, deep learning-based spatial interpolation methods have shown great promise, performing excellently in handling complex nonlinear features, but their ability to model spatial dependencies is limited. Summary of the Invention

[0005] To address the problem that existing interpolation methods cannot fully capture the nonlinearity and spatiotemporal dependence of spatial data, this invention proposes a spatiotemporal interpolation neural network model (STKriging) for spatial interpolation and prediction. It can automatically extract features from large-scale and dynamically changing spatial data, effectively identify nonlinear relationships between spatial points, and thus achieve more accurate spatial interpolation.

[0006] This invention protects a spatiotemporal interpolation neural network model that uses spatiotemporal basis functions of a spatiotemporal process to generate an embedding layer, where a spatiotemporal process γ(s,t) is approximately a finite sum Where ω p It is an independent random variable, Φ p (s,t) are pairwise orthogonal spacetime basis functions, derived from the space basis function φ. i (s), i = 1, 2, ..., G, and the time basis function χ j The stacking of (t), j=1,2,…,H yields Φ(s,t)={φ1(s),…,φ G (s),χ1(t),…,χ H (t)} T ;

[0007] X Φ (s,t) are used as the input to the neural network, X Φ (s,t)=(Φ(s,t) T ,X vec (s,t) T ) T X vec (s,t) covariates, and the neural network layer is as follows:

[0008] First layer, h1(s,t)=W1X Φ (s,t)+b1,a1(s,t)=ψ1(h1(X Φ(s,t))),

[0009] In the second layer, h2(s,t)=W2a1(s,t)+b2, a2(s,t)=ψ2(h2(a1(s,t))),

[0010] …

[0011] Layer L, h L (s,t)=W L a L-1 (s,t)+b L ,

[0012] For the output of the model, for models containing M l The l-th layer of neurons, where l = 1, 2, ..., L, W l For M l ×M l-1 Weight matrix, b l For a length of M l The bias vector, a l For a length of M l The neuron vector, ψ l (·) is the activation function, and ψ(τ,·) is the activation function of the last layer.

[0013] Preferably, the spatial basis functions employ multi-resolution compactly supported Wendland radial basis functions, defined as follows: in u i Let G be a point in a rectangular grid, i = 1, 2, ..., G. This is the scale parameter.

[0014] Preferably, the time basis function is a Gaussian radial basis function, defined as χ. j (t)=exp(-0.5(tv j ) 2 / κ 2 ), where the time anchor point v∈{v1,v2,...,v... j ,···,v H The ratio κ = |v1 - v2|.

[0015] Preferably, a nonlinear function is used. To perform quantile prediction, an empirical version of the risk function R1(·,·) is minimized using stochastic gradient descent. Let the parameter set of the spatiotemporal interpolation neural network model be θ={W l ,b l}, then we have

[0016]

[0017] Where, ρ τ (v) = v(τ-I(v<0)), where v<0 is the check loss function.

[0018] Preferably, an appropriate activation function ψ is selected to control the activation for different τ values. Output:

[0019]

[0020] in This is the model's estimate at the quantile level of 0.5. The hyperparameter λ controls the deviation of the upper or lower quantile from the median, and is taken as λ∝σ. range / 2,σ range =maxZ N,K -minZ N,K .

[0021] This invention also protects a spatiotemporal data modeling method, which performs spatiotemporal interpolation on spatial data based on the above-mentioned spatiotemporal interpolation neural network model.

[0022] This invention, by introducing an embedding layer of spatial coordinates and spatiotemporal basis functions, can accurately model spatial dependencies, thereby establishing a direct link between DNN and Kriging spatial prediction. Utilizing a deep neural network structure, the model can automatically extract features from large-scale and dynamically changing spatial data, effectively identifying nonlinear relationships between spatial points, thus achieving more accurate spatial interpolation.

[0023] Compared to the parameterized covariance function constraints faced by traditional spatial statistical methods, this invention enables the adaptive construction of spatially dependent structures, rather than simply using geographic coordinates as scalar inputs; at the same time, the calculation process does not require complex matrix operations and can be efficiently extended to large-scale datasets.

[0024] This invention is not only applicable to nonlinear predictions with covariates, but also capable of handling non-Gaussian or non-stationary data, greatly improving the model's applicability and robustness. Furthermore, the deep learning model excels in solving local grid edge problems, smoothing edge regions and improving the accuracy and reliability of gridding processing. Attached Figure Description

[0025] Figure 1 Example of grid coverage for adjacent administrative divisions;

[0026] Figure 2 A diagram of the neural network model structure for spatiotemporal interpolation;

[0027] Figure 3 PM2.5 dataset for Xi'an area

[0028] Figure 4 This is a spatial interpolation result image of PM2.5, where Figure 4 (a) Corresponding to the IDW algorithm, Figure 4 (b) Corresponding to the Kriging algorithm, Figure 4 (c) Corresponds to the classic DNN algorithm, Figure 4 (d) corresponds to the STKriging algorithm;

[0029] Figure 5 Comparison chart of grid interpolation reconstruction efficiency. Detailed Implementation

[0030] The present invention will now be described in further detail with reference to the accompanying drawings and specific embodiments. The embodiments of the present invention are given for illustrative and descriptive purposes only, and are not intended to be exhaustive or to limit the invention to the forms disclosed. Many modifications and variations will be apparent to those skilled in the art. The embodiments were chosen and described to better illustrate the principles and practical application of the invention, and to enable those skilled in the art to understand the invention and design various embodiments with various modifications suitable for a particular purpose.

[0031] Example 1

[0032] To address the modeling requirements of complex nonlinear features and multi-scale spatiotemporal correlations in spatiotemporal data, this embodiment proposes a neural network framework based on multi-level spatiotemporal kriging, abbreviated as STKriging.

[0033] The spatiotemporal interpolation neural network model disclosed in this embodiment uses the spatiotemporal basis functions of the spatiotemporal process to generate the embedding layer, instead of directly inputting the observed coordinates into the DNN. A spatiotemporal process γ(s,t) is approximated as a finite sum Where ω p It is an independent random variable, Φ p (s,t) are pairwise orthogonal spacetime basis functions, derived from the space basis function φ. i (s), i = 1, 2, ..., G, and the time basis function χ j The stacking of (t), j=1,2,…,H yields Φ(s,t)={φ1(s),…,φ G (s),χ1(t),…,χ H (t)} T Therefore, the interpolation problem can be viewed as having these basis functions Φ p Linear regression of (s,t).

[0034] First, it is necessary to select a K value and basis functions to approximate the spatial process v(s,t). In this embodiment, a multi-resolution compactly supported Wendland radial basis function was selected from a large number of existing basis functions for spatial embedding. The radial basis function for each resolution is constructed using the correlation function of Wendland compact support, and the nodes are arranged on a rectangular grid.

[0035] Specifically, at a certain resolution level, u i Let G be a point in a rectangular grid (or a node in the radial basis function terminology), i = 1, 2, ..., G. Let be the scale parameter. The spatial basis functions are: in

[0036]

[0037] Therefore, the embedding layer uses local mutual distances for each node, meaning that spatial patterns are locally position-invariant. Thus, the STKriging proposed in this embodiment can model spatial nonstationarity. Scale parameters The mesh size is set to 2.5 times the relevant node spacing. The mesh size is increased by a factor of 2 for each finer level, and the basis functions are scaled to constant overlap. Specifically, at level h, the number of nodes is chosen to be K. h = (9×2) h-1 +1) d , where d is the spatial dimension.

[0038] For a large-scale dataset, to ensure K ≥ N, the following conditions must be met: Therefore, taking a four-level model as an example, in one-dimensional space, K = 10 + 19 + 37 + 73 = 139 basis functions are needed, and in two-dimensional space, K = 10 2 +19 2 +37 2 +73 2 =7159 basis functions. This method can approximate the standard covariance function well and has the flexibility to adapt to more complex shapes.

[0039] To extract time dependencies, this embodiment uses a Gaussian radial basis function in the time domain, defined as χ². j (t)=exp(-0.5(tv j ) 2 / κ 2 ), where the time anchor point v∈{v1,v2,...,v... j ,···,v H The ratio κ = |v1 - v2|.

[0040] Geographic coordinates are embedded into the input layer of a deep neural network (DNN) using latitude and longitude encoding, effectively capturing the spatial correlation of the data. To reduce computation, STKriging avoids calculating tensor products; instead, it stacks spatial and temporal basis functions together to obtain Φ(s,t)={φ1(s),…,φ G (s),χ1(t),…,χ H (t)} T The cardinality of Φ(s,t) is Q = G + H. The choice of basis functions does not imply spatial and temporal separability. In deep neural networks, weights are shared by each node, meaning deep neural networks can learn the interaction between spatial and temporal basis functions. A large set of basis functions can be created by choosing different values ​​of G and H. Therefore, the model can explain both long-term and short-term spatial and temporal dependencies.

[0041] This embodiment uses a single-output deep neural network structure to construct a spatiotemporal deep kriging framework, the structure of which is as follows: Figure 2 As shown. Define X. Φ (s,t)=(Φ(s,t) T ,X vec (s,t) T ) T , and as input to the neural network, X vec (s,t) covariates, and the neural network layer is as follows:

[0042] First layer, h1(s,t)=W1X Φ (s,t)+b1,a1(s,t)=ψ1(h1(X Φ (s,t))),

[0043] In the second layer, h2(s,t)=W2a1(s,t)+b2, a2(s,t)=ψ2(h2(a1(s,t))),

[0044] …

[0045] Layer L, h L (s,t)=W L a L-1 (s,t)+b L ,

[0046] For the output of the model, for models containing M l The l-th layer of neurons, where l = 1, 2, ..., L, W l For M l ×M l-1 Weight matrix, b l For a length of M l The bias vector, al For a length of M l The neuron vector, ψ l (·) is the activation function, and ψ(τ,·) is the activation function of the last layer. The parameter set of this network is θ={W l ,b l} is a vector of unknown weights and biases, estimated based on the training samples.

[0047] This embodiment uses a nonlinear function. To perform quantile prediction, an empirical version of the risk function R1(·,·) is minimized using stochastic gradient descent. Let the parameter set of the spatiotemporal interpolation neural network model be θ={W l ,b l}, then we have Where, ρ τ (v) = v(τ-I(v<0)), where v<0 is the check function for the quantile loss function.

[0048] According to the definition of quantiles, conditional quantile functions do not intersect. That is, when τ1≤τ2, This condition is satisfied for all s0 and t. To ensure this, this embodiment selects an appropriate activation function ψ to control the activation for different τ values. Output:

[0049]

[0050] in This is the model's estimate at the quantile level of 0.5. The hyperparameter λ controls the deviation of the upper or lower quantile from the median, and is taken as λ∝σ. range / 2,σ range =maxZ N,K -minZ N,K .

[0051] Based on the network architecture, the output of ψ(τ) can always provide a non-crossing quantile prediction interval because the upper limit of the interval is always greater than the upper limit of the median, while the lower limit is always less than the lower limit of the median. To obtain a 100(1-α)% prediction interval, an estimate of the median is first needed. And use it to calculate and ψ(τ,x) is used sequentially as the activation of the output layer. Therefore, the final predictor of the neural network is located at the unobserved position s0. in

[0052] A key advantage of the STKriging method is that the model can be tuned in terms of the number of neurons, activation functions, and loss functions to suit different data types and model interpretations. For example, for predicting continuous variables in regression problems, M can be chosen... L =1, ψ(·) is the identity function, and the loss function is MSE. To predict categorical variables in a classification problem, we can choose M... L Let denoted as the number of classes, ψ(·) be the softmax function, and the loss function be the cross-entropy loss. For the activation functions in the hidden layers, the model defaults to the Rectified Linear Unit (ReLU), which allows the model to maintain the linearity in the KL expansion but adds some deactivated neurons to select the optimal number of basis functions. The STKriging structure also allows covariate effects to vary spatially.

[0053] Regularization of the STKriging network structure includes adding dropout layers to mitigate overfitting, adding batch normalization to regularize covariates and basis functions to the same scale, and removing all-zero columns in the basis matrix D due to the compact support structure of the basis functions. The time complexity of the STKriging method is approximately O(Nneuron), where Nneuron is the number of neurons in the network. The computational cost depends on the width and depth of the network, and the computation is highly parallelizable, which can be significantly accelerated by CPUs and GPUs.

[0054] In the STKriging framework, the default training epoch is 200. For regression tasks, the loss function is set to MSE, while for classification tasks, the loss function is set to cross-entropy loss. For optimization, the Adam optimizer is introduced, whose efficient gradient descent algorithm helps the model converge quickly. Furthermore, key hyperparameters in the network can be tuned, including but not limited to the K value, the number of network layers, the number of neurons per layer (width), the dropout rate, the batch size, and the number of training epochs, thereby exhibiting higher prediction accuracy and robustness in complex data modeling tasks.

[0055] Example 2

[0056] A spatiotemporal data modeling method is proposed, which performs spatiotemporal interpolation on spatial data based on the spatiotemporal interpolation neural network model described in Example 1. This will be illustrated below with specific experiments.

[0057] PM2.5 (fine particulate matter) is a crucial indicator of air quality and is of great significance for low-altitude air quality management and urban governance. Real-time and accurate assessment of PM2.5 concentration distribution across different spatial scales is essential. However, PM2.5 monitoring data typically originates from sparsely distributed monitoring stations, making it challenging to directly estimate the PM2.5 concentration for an entire region using data from these points. Employing efficient spatial interpolation methods to reconstruct monitoring station data can provide high-resolution PM2.5 concentration maps for large areas. This embodiment utilizes a spatially dependent deep learning spatiotemporal interpolation model to reconstruct PM2.5 concentration data and analyzes the effectiveness of this algorithm.

[0058] The data source is the China High-Resolution PM2.5 dataset (ChinaHighPM2.5), which includes daily, monthly, and annual average concentration distributions from 2000 to 2018. Data was preprocessed using a data gridding method, with a resolution of 1 kilometer, covering the Xi'an area. Figure 3 As shown, it contains a total of 9989 grid cells.

[0059] To comprehensively evaluate model performance, we selected three comparison methods: Inverse Distance Weighted Method (IDW), Kriging, and the classic Deep Neural Network (DNN) model. We used 10x cross-validation to compare the accuracy of these methods. Evaluation metrics included the mean and standard deviation (STD) of the Root Mean Square Error (RMSE) and Mean Absolute Error (MAE). RMSE is particularly sensitive to large errors, tending to penalize predictions with significant errors. A smaller RMSE indicates that the model's predictions are closer to the true values. Unlike RMSE, MAE is less sensitive to errors, assigning equal weight to each error value, and can measure the overall accuracy of the model without being overly affected by outliers. A smaller MAE indicates higher overall prediction accuracy.

[0060] As shown in Table 1, STKriging demonstrates significant advantages in interpolation of PM2.5 concentration data, particularly in prediction accuracy and stability. Compared to other interpolation methods (such as IDW, Kriging, and classic DNN), STKriging exhibits lower values ​​for both root mean square error (RMSE) and mean absolute error (MAE). Specifically, the mean RMSE for STKriging is 1.326, while IDW is 4.978, Kriging is 2.492, and classic DNN is 3.532, indicating that STKriging provides more accurate predictions, especially under conditions of low data density or large regional spans. For MAE, the mean value for STKriging is 0.946, significantly lower than IDW (4.297), Kriging (1.807), and classic DNN (2.548), further demonstrating its advantage in error control. In addition, STKriging's standard deviation is significantly lower than other methods, especially for classic DNN (10.949) and IDW (8.563) methods. STKriging not only provides more accurate prediction results, but also has high stability and consistency.

[0061] Table 1

[0062]

[0063] STKriging, by combining the advantages of deep learning and Kriging, can effectively model complex dependencies in spatiotemporal data (see...). Figure 4 (a)~4(d)). In the interpolation of PM2.5 concentration, STKriging can not only capture the geographical correlation of data spatially, but also handle the impact of seasonality, weather changes, and other factors on PM2.5 concentration temporally. This spatiotemporal modeling capability makes STKriging superior to traditional methods when dealing with sparse data, complex spatial structures, and dynamically changing data. Especially in applications such as large-scale environmental monitoring and low-altitude airspace management, STKriging can provide more accurate and efficient predictions, ensuring the accuracy of flight safety and environmental quality assessment. Overall, STKriging's excellent performance in PM2.5 concentration interpolation indicates its broad application prospects in large-scale data analysis and spatial interpolation applications.

[0064] Figure 5A time efficiency analysis of several interpolation methods is presented. It can be seen that while STKriging is not as fast as IDW in terms of PM2.5 data interpolation time, it offers a better balance compared to Kriging and the classic DNN method. IDW's time is 16.408, significantly lower than other methods, demonstrating its advantage in computational efficiency. However, IDW typically does not consider spatial and temporal dependencies, thus its performance is weaker in complex data environments. In contrast, STKriging's time is 172.813, higher than IDW and the classic DNN, but significantly lower than Kriging's 329.137. Especially when Kriging's computation time is much longer than other methods, STKriging provides a better trade-off between accuracy and efficiency, delivering more accurate PM2.5 interpolation results in a shorter time.

[0065] In summary, the STKriging model provided by this invention can offer more efficient performance than traditional methods in high-precision spatiotemporal data interpolation applications, and is especially suitable for complex environments that require spatiotemporal modeling.

[0066] Obviously, the described embodiments are only a part of the embodiments of the present invention, and not all of them. All other embodiments obtained by those skilled in the art and related fields based on the embodiments of the present invention without inventive effort should fall within the scope of protection of the present invention.

Claims

1. A spatiotemporal interpolation neural network model, characterized in that, An embedding layer is generated using the spatiotemporal basis functions of a spatiotemporal process, where a spatiotemporal process γ(s,t) is approximated as a finite sum. Where ω p It is an independent random variable, Φ p (s,t) are pairwise orthogonal spacetime basis functions, derived from the space basis function φ. i (s), i = 1, 2, ..., G, and the time basis function χ j The stacking of (t), j=1,2,…,H yields Φ(s,t)={φ1(s),…,φ G (s),χ1(t),…,χ H (t)} T ; X Φ (s,t) are used as the input to the neural network, X Φ (s,t)=(Φ(s,t) T ,X vec (s,t) T ) T X vec (s,t) covariates, and the neural network layer is as follows: First layer, h1(s,t)=W1X Φ (s,t)+b1,a1(s,t)=ψ1(h1(X Φ (s,t))), In the second layer, h2(s,t)=W2a1(s,t)+b2, a2(s,t)=ψ2(h2(a1(s,t))), … Layer L, h L (s,t)=W L a L-1 (s,t)+b L , For the output of the model, for models containing M l The l-th layer of neurons, where l = 1, 2, ..., L, W l For M l ×M l-1 Weight matrix, b l For a length of M l The bias vector, a l For a length of M l The neuron vector, ψ l (·) is the activation function, and ψ(τ,·) is the activation function of the last layer.

2. The spatiotemporal interpolation neural network model according to claim 1, characterized in that, The spatial basis functions employ multi-resolution compactly supported Wendland radial basis functions, defined as follows: in u i Let G be a point in a rectangular grid, i = 1, 2, ..., G. This is the scale parameter.

3. The spatiotemporal interpolation neural network model according to claim 1, characterized in that, The time basis function uses a Gaussian radial basis function and is defined as χ. j (t)=exp(-0.5(tv j ) 2 / κ 2 ), where the time anchor point v∈{v1,v2,...,v... j ,···,v H The ratio κ = |v1 - v2|.

4. The spatiotemporal interpolation neural network model according to claim 1, characterized in that, Using nonlinear functions To perform quantile prediction, an empirical version of the risk function R1(·,·) is minimized using stochastic gradient descent. Let the parameter set of the spatiotemporal interpolation neural network model be θ={W l ,b l }, then we have Where, ρ τ (v) = v(τ-I(v<0)), where v<0 is the check loss function.

5. The spatiotemporal interpolation neural network model according to claim 4, characterized in that, For different τ values, an appropriate activation function ψ is selected to control... Output: in This is the model's estimate at the quantile level of 0.

5. The hyperparameter λ controls the deviation of the upper or lower quantile from the median, and is taken as λ∝σ. range / 2,σ range =maxZ N,K -minZ N,K .

6. A spatiotemporal data modeling method, characterized in that, Spatial data is spatiotemporally interpolated based on the spatiotemporal interpolation neural network model described in any one of claims 1-5.

Images