Space-time sequence interpolation method and device for heterogeneous deletion

Abstract

The invention discloses a space-time sequence interpolation method and device oriented to heterogeneous deletion, and belongs to the technical field of space-time data processing. Aiming at random or continuous loss of the sensor network caused by faults and communication interruption, the method comprises the following steps: setting static space experts, dynamic space experts, short-term experts and long-term period experts in parallel in the same framework, and respectively capturing fixed geographical adjacency, time-varying space correlation, local continuous trend and long-period rules; spatial features are extracted through high-order diffusion diagram convolution and bidirectional gating circulation, time features are extracted through multi-layer space-time attention, a memory attention gating network is introduced to dynamically weight and fuse output of experts according to reconstruction errors, and node-level and time-step-level self-adaptive interpolation is achieved. Experiments show that compared with the prior art, the method has the advantages that under various real data sets and heterogeneous missing scenes, the precision and robustness are remarkably improved, and the method can be widely applied to scenes needing high-integrity spatio-temporal data, such as intelligent transportation, air quality monitoring and energy internet of things.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of spatiotemporal sequence data processing, and particularly relates to a spatiotemporal sequence interpolation method and device for heterogeneous missing. BACKGROUND

[0002] In new-type infrastructures such as smart cities, double-carbon monitoring, and smart power grids, a large number of sensor nodes continuously collect observation data with geographical locations and timestamps, forming so-called “spatiotemporal sequence data”. These data are widely used in intelligent transportation, air quality monitoring, weather observation, and energy Internet of Things, and support key tasks such as city management, environmental monitoring, and resource scheduling. Each spatiotemporal sequence not only records the dynamic process of a certain location over time, but also implies the spatial correlation between different locations. However, due to factors such as sensor failure, communication interruption, and environmental interference, the actual collected spatiotemporal data often has a large number of missing values.

[0003] These missing values are not random or single-patterned, but highly heterogeneous. In the time dimension, there are both short-term random missing and long-term continuous missing; in the spatial dimension, there can be local anomalies of individual nodes or regional missing of multiple nodes simultaneously offline. This complex and variable missing pattern poses a serious challenge to the integrity of the data and subsequent analysis, prediction, and decision-making. Therefore, how to accurately and robustly recover the missing data has become one of the core problems in spatiotemporal data processing.

[0004] To solve this problem, researchers have carried out a lot of research around the “spatiotemporal dependency structure”, trying to infer the true values of missing positions by modeling the spatial correlation and temporal evolution law. Spatial dependence refers to the fact that adjacent or structurally similar nodes tend to show similar observation values at the same time; temporal dependence refers to the fact that the same node has trend, periodicity, and other evolution laws between different time steps. Based on these two types of dependencies, existing spatiotemporal sequence interpolation methods can be roughly divided into three categories:

[0005] The first category is the interpolation method that only models temporal dependence. This type of method treats each spatial node as an independent one-dimensional time series, models its temporal evolution law using historical observation data, and commonly uses techniques such as autoregressive models (such as ARIMA), exponential smoothing, Kalman filters, and recurrent neural networks (such as LSTM, GRU). These methods perform well in capturing short-term trends and periodicity, and are suitable for scenarios with low missing proportions and strong temporal continuity. However, they ignore the correlation in the spatial dimension and cannot utilize the information of neighboring nodes, leading to biased interpolation results when facing large-scale missing or event propagation, and even poor performance in areas with sparse node distribution.

[0006] The second type is an interpolation method only modeling spatial dependence. This type of method assumes that nodes with similar spatial positions or structures have similar observation values at the same time section, and commonly uses spatial interpolation algorithms (such as inverse distance weighting, Kriging) or graph structure-based learning methods (such as graph convolution network GCN) for modeling. They can effectively utilize the local structure in space and have good recovery ability for short-term missing in node-dense areas. However, this type of method usually ignores the dynamic evolution in the time dimension and cannot capture the historical trends and periodic changes of the nodes themselves, resulting in a lack of continuity in the interpolation results in time, especially when there is a long-term missing or evolution of system state, the recovery effect is poor.

[0007] The third type is an interpolation method jointly modeling spatial and temporal dependence. In recent years, with the development of deep learning, researchers have proposed various modeling frameworks that integrate spatial and temporal information, such as ST-GCN, ASTGCN, STGAT, etc. These methods combine spatial convolution based on graph structure and time series modeling modules (such as RNN, TCN, Transformer, etc.) to try to capture the dependence relationship in both spatial and temporal dimensions, improving the accuracy and robustness of interpolation. In addition, there are some methods based on tensor decomposition (such as HaLRTC, BTTF) and Transformer models based on attention mechanism (such as ImputeFormer), which further explore the potential structure of spatio-temporal data through low-rank reconstruction or cross-dimensional attention mechanism. SUMMARY

[0008] The present application proposes a spatio-temporal sequence interpolation method and device for heterogeneous missing, which explicitly decouples and models the implicit spatio-temporal dependence of spatio-temporal sequences through structured expert division. The specific technical solutions are as follows:

[0009] A spatio-temporal sequence interpolation method for heterogeneous missing, comprising the following steps:

[0010] Standardize the original spatio-temporal sequence obtained by the sensor network, and synchronize the observation tensor and the missing mask according to the mask tensor;

[0011] Start the static spatial expert and dynamic spatial expert sub-modules in parallel, the static spatial expert sub-module drives the graph convolution to generate hidden states and spatial interpolation results with a fixed geographical adjacency matrix, and the dynamic spatial expert sub-module generates a time-varying adjacency matrix through self-attention after fusing historical observations, node embeddings and periodic time embeddings at each time step, then obtains a multi-order support set through high-order diffusion, extracts hidden states using the support set, and outputs the spatial interpolation results through structure perception decoding and gated fusion of the hidden states;

[0012] The short-term time expert and the long-term period expert submodule are started in parallel, and the short-term time expert and the long-term period expert submodule each include: fusing the input observation and the mask, and then mapping the original channel to a hidden dimension through a multilayer perception to obtain a preliminary representation The preliminary representation is recovered by designing a learnable mask vector, the periodic time embedding is projected into a hidden space, and is broadcasted on a node dimension, next, fixed position encoding is superimposed on a time dimension to obtain a representation with position encoding, modeling is performed through stacked space-time attention, and finally, a hidden state and a time interpolation result are obtained;

[0013] The observation tensor, the missing mask and the expert hidden state are spliced into a query embedding, the query embedding is refined through memory attention, and a cosine similarity is calculated with a state after self-attention transformation of each expert to generate normalized expert weights, and the interpolation features of each expert are weighted and summed to obtain a final interpolation result; the best selection and the worst avoidance label are dynamically calculated according to the expert reconstruction error.

[0014] A space-time sequence interpolation device for heterogeneous missing, comprising the following steps:

[0015] The sequence acquisition module standardizes the original space-time sequence obtained by the sensor network, and synchronizes the observation tensor and the missing mask according to the mask tensor;

[0016] The space dependence modeling module starts the static space expert and the dynamic space expert submodule in parallel, the static space expert submodule drives the graph convolution to generate a hidden state and a space interpolation result by using a fixed geographical adjacency matrix, the dynamic space expert submodule generates a time-varying adjacency matrix through self-attention after fusing historical observation, node embedding and periodic time embedding at each time step, and then obtains a multi-order support set through high-order diffusion, and the hidden state is extracted by using the support set, and the hidden state is output through structure perception decoding and gate fusion to output the space interpolation result;

[0017] The time dependence modeling module starts the short-term time expert and the long-term period expert submodule in parallel, and the short-term time expert and the long-term period expert submodule each include: fusing the input observation and the mask, and then mapping the original channel to a hidden dimension through a multilayer perception to obtain a preliminary representation The preliminary representation is recovered by designing a learnable mask vector, the periodic time embedding is projected into a hidden space, and is broadcasted on a node dimension, next, fixed position encoding is superimposed on a time dimension to obtain a representation with position encoding, modeling is performed through stacked space-time attention, and finally, a hidden state and a time interpolation result are obtained;

[0018] The result output module splices the observation tensor, the missing mask and each expert hidden state into a query embedding, calculates cosine similarity between the query embedding after refinement by memory attention and state after transformation by each expert self-attention, generates normalized expert weights, and obtains a final interpolation result by weighting and summing interpolation features of each expert according to the weights; and dynamically calculates the best selection and worst avoidance labels according to the reconstruction error of each expert.

[0019] An electronic device includes one or more processors and memory storing one or more programs, wherein the one or more programs, when executed by the one or more processors, cause the one or more processors to implement the method.

[0020] A computer-readable storage medium has stored thereon executable instructions that, when executed by a processor, cause the processor to perform the method.

[0021] The present application has the following beneficial effects:

[0022] The present application decouples the display of spatio-temporal dependence by four types of structured experts. A static space expert (SE), a dynamic space expert (DE), a short-term time expert (RTE) and a long-term time expert (LTE) are set in parallel in the same interpolation framework, and expert weights are generated by a gating network at the node level and the time step level, and the interpolation results are obtained by adaptively weighting and fusing the expert outputs.

[0023] The present application is based on the dynamic graph construction (DE) of "priori x attention". The dynamic space expert fuses the historical observation, the time embedding and the node embedding at each time step to calculate the attention relevance; and element-wise multiplies the priori similarity matrix obtained from the node embedding / geography / relevance to form a time-varying adjacency matrix for subsequent graph convolution / recurrent propagation.

[0024] The present application adopts memory attention gating based on dynamic supervision of expert error. The gating network generates a query representation by memory attention, matches and scores four expert hidden representations, and introduces gating supervision labels (best selection and worst avoidance) to explicitly constrain the routing distribution, improve the selection stability and interpretability.

[0025] The present application proposes a structured expert hybrid framework, which realizes dynamic adaptive modeling of complex dependence structure in spatio-temporal sequence interpolation tasks. By explicitly dividing and coordinating four types of expert modules, static space, dynamic space, short-term time and long-term cycle and other multi-level spatio-temporal dependence can be captured, and the structural adaptation to heterogeneous missing patterns can be realized through the gating mechanism. Through system experiments on multiple real-world public datasets and various missing scenarios, the results show that the present application is significantly superior to existing mainstream methods in accuracy, robustness and generalization ability. Attached Figure Description

[0026] Figure 1 This is a flowchart of the method of the present invention. Detailed Implementation

[0027] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention. Furthermore, the technical features involved in the various embodiments of this invention described below can be combined with each other as long as they do not conflict with each other. To achieve the above objectives, this invention adopts the following technical solution.

[0028] like Figure 1 As shown, a spatiotemporal sequence interpolation method for heterogeneous missing sequences according to the present invention includes the following steps:

[0029] Step 1: Data preprocessing. The data acquired by the sensor network is converted into an observation tensor, and the observation tensor is recovered based on the mask tensor.

[0030] Data from a source Acquisition is obtained from a sensor network consisting of spatial nodes, with each node continuously... Data collected at each time step Dimensional observations form an observation tensor. However, due to sensor malfunctions, transmission delays, or other external factors, some values ​​are missing in the tensor.

[0031] Introduce a mask tensor of the same shape. ,in Indicates the location The observation tensor on Observations on the tensor are available if they are not, otherwise they are missing. The goal is to use the existing observation tensor... and mask tensor Learn an interpolation function To recover the complete observation tensor and make The model aims to approximate the true value as closely as possible to the missing values. The input observation tensor of the model... It has been standardized by standard deviation for the purpose of gradient stabilization.

[0032] Step 2, spatial dependency modeling, introduces static spatial expert submodule and dynamic spatial expert submodule, and models the two types of spatial dependency structures respectively.

[0033] The application introduces two types of structured space expert modules, including a static space expert submodule and a dynamic space expert submodule, to model static and dynamic space dependency structures respectively, so as to enhance the adaptability and discrimination ability of the model to different types of space dependency patterns. The two types of experts are basically the same in structure except for the difference in the design of the adjacency matrix.

[0034] In the static space expert submodule (SE), a priori given static adjacency matrix is used as the basis for space structure information. In the application, the static adjacency matrix is calculated by the distance between sensors. In the actual model, all graph-related operations (such as high-order graph convolution, gated recurrent unit, etc.) of the static space expert submodule are based on the fixed adjacency matrix.

[0035] In contrast, the space dependency structure in the real world is often not constant, but changes dynamically with factors such as environment and time period. To depict this time-varying space relationship, the dynamic space expert submodule (DE) adaptively learns the adjacency matrix at each time, and the main process is as follows:

[0036] At each time step , the node historical observation, time context and node semantic information are fused to dynamically model the space structure. The specific steps are as follows:

[0037] First, the historical observation sequence of each node is extracted, where is the historical window length, is the number of nodes, is the number of channels. After splicing the observations of different channels, the basic dynamic features are obtained through two layers of nonlinear fully connected networks . At the same time, time embedding (intra-day and intra-week period respectively) and two groups of node embedding matrices are introduced, where the embedding matrix is used to represent the embedding representation of each node in the process of information outward transmission, and the embedding matrix is used to represent the embedding representation of each node in the process of information inward reception. Accordingly, the forward and reverse node dynamic feature matrices are constructed respectively:

[0038] ;

[0039] ;

[0040] wherein represents the splicing of the historical observation of each node on all channels, wherein represents the splicing operation in the channel dimension C, is the observation data of the historical observation sequence X in C channels.

[0041] Subsequently, a self-attention mechanism is used to model the dynamic correlation between node pairs, specifically:

[0042] ;

[0043] ;

[0044] in, These are the static prior adjacency matrices, calculated from node embedding similarity. For self-attention parameters, This represents the Hadamard product (element-wise multiplication), with the Softmax operation normalizing each row. For the final feature dimension, , These are obtained by calculating the node embedding similarity from the static prior adjacency matrix, respectively, corresponding to the static graph structure priors in the upstream and downstream directions; , These represent the dynamic adjacency matrix learned at time t, the dynamic correlation between node pairs calculated using a self-attention mechanism, and then compared with the static prior adjacency matrix. , We obtain it by multiplying element by element.

[0045] final, and Each as a time The forward and reverse dynamic adjacency matrices, where The forward dynamic adjacency matrix (upstream) at time t is used to model the dependency of information flowing from upstream nodes to downstream nodes; The downstream dynamic adjacency matrix at time t is used to model the dependency of information feedback from downstream nodes to upstream nodes. These two types of matrices differ in directionality and are used for information propagation in subsequent dynamic graph convolutions and recursive structures, enabling adaptive modeling of time-varying spatial dependencies.

[0046] To more comprehensively capture the multi-level spatial dependencies between nodes, this invention introduces a higher-order diffusion support set when constructing the graph structure. Specifically, for any time t, regardless of whether a static adjacency matrix is ​​used... Or a dynamic graph adjacency matrix All of them generate multi-order support matrices through high-order matrix multiplication. Each support matrix is ​​defined as follows:

[0047] ;

[0048] in, denotes the adjacency matrix at time t, k denotes the diffusion order, K denotes the maximum diffusion order, i.e., the highest order in the considered support matrix set, denotes the k-th order support matrix at time t, which is obtained by power operation of the adjacency matrix. This high-order diffusion mechanism can significantly enhance the modeling ability of the model for long-range spatial dependencies and high-order structural relationships, enabling nodes to perceive a wider neighborhood range when aggregating information. Finally, the obtained support set will be used in subsequent spatial dependency modeling modules to achieve more rich and flexible spatial feature interactions.

[0049] To fully model the complex spatial topology in the spatio-temporal sequence, the invention proposes an interpolation method with a bidirectional graph convolution recurrent network as the backbone. This module not only integrates forward and backward spatio-temporal context, but also realizes adaptive interpolation of heterogeneous missing distribution and spatial relationship through structure-aware spatial decoding and gating fusion mechanism.

[0050] At each time step t, the node observation is input, combined with the hidden state at the previous time and the current spatial support matrix , and the spatio-temporal hidden state of the node is recursively updated through the graph convolution gated recurrent unit (GCGRU). Specifically, it includes:

[0051] Forward recursion (Forward Pass): recursively generate forward hidden state based on the previous observation and historical state;

[0052] Backward recursion (Backward Pass): recursively generate backward hidden state based on the subsequent observation.

[0053] The bidirectional recursive design can explicitly utilize historical and future information, and alleviate the error accumulation problem of one-way inference for long missing segments.

[0054] After obtaining the forward and backward hidden states and , a structure-aware spatial decoder is further designed to refine the interpolation result. For each time t, the observation , the missing mask , the node embedding and hidden state , are spliced, and high-order neighbor features and structural context information are extracted through multi-order spatial convolution and spatial attention mechanism. The spatial decoder outputs the bidirectional refined interpolation value and the refined interpolation result representation , effectively improving the interpolation robustness under complex spatial relationships. The node embedding refers to and the average.

[0055] To adapt to node heterogeneity and spatial dependence diversity, the model introduces a gating fusion network to adaptively weight the structural representations of different directions and different expert branches. Specifically, the forward and reverse spatial representation vectors are concatenated with the node embedding and the missing mask to serve as the hidden state of the corresponding spatial expert sub-module to fill in the output results. The missing mask indicates the missing mask of this historical sequence, and the concatenated result is sent to a multi-layer perception (MLP) to obtain the final interpolation output :

[0056] ;

[0057] Step 3, time dependence modeling, introduces two parallel sub-modules of short-term time experts and long-term cycle experts to model local continuity and global periodicity.

[0058] The time-dependent structure plays an indispensable role in multivariate spatio-temporal sequence interpolation. For different scales of time characteristics, the invention introduces two parallel sub-modules of short-term time experts (Recent Temporal Expert) and long-term cycle experts (Long Temporal Expert) to fully model local continuity and global periodicity, thereby improving the accuracy and generalization ability of missing value recovery.

[0059] The two types of experts are consistent in model structure and only differ in input. Specifically, the short-term time expert sub-module focuses on modeling the dependence between adjacent time steps in the observed sequence, focusing on the flow of information in the local context. The core idea is to use the strong correlation between consecutive time steps in the sequence to efficiently interpolate missing data within a short window. Therefore, the input of the short-term time expert includes the observed , time embedding , and the missing mask m of the current window, where the time embedding is calculated by the time index, specifically, the input time index sequence , where T is the time step. Let the set of cycle units be . For any unit , define its standard cycle length as , where the standard cycle length of day is 24 hours, and the standard cycle length of week is 7 24 hours. Then, convert the timestamp t to a unified nanosecond count , the phase angle is calculated: Then for each period unit u, the sine and cosine embeddings are generated by the sine and cosine functions respectively, and the sine and cosine embeddings of each u are spliced to obtain the time embedding .

[0060] To capture more extensive global trends, long-term dependencies and periodic patterns, the long-term period expert sub-module takes an extended history window and period auxiliary features as input, where the period auxiliary features refer to the features of the same window before and after the extended history window as day period and week period auxiliary features, focusing on modeling long-term time dependencies across periods and seasons. The long-term period expert sub-module can effectively provide completion capability for global missing and long section breaks that are difficult to interpolate by local information.

[0061] The specific short-term time expert and long-term period expert sub-module model structure is as follows. Before entering the Transformer structure, the input observation and mask are first fused and processed to retain only the true observation part:

[0062] ;

[0063] wherein, is the Hadamard product;

[0064] Then the original channel is uniformly mapped to the hidden dimension by a multi-layer perceptron to obtain the preliminary representation ;

[0065] In addition, to avoid no representation at the missing position, a learnable mask vector is designed to replace the missing features:

[0066] ;

[0067] Next, the periodic time embedding is projected into the hidden space and broadcasted in the node dimension to realize time conditioning:

[0068] ;

[0069] ;

[0070] wherein, represents the broadcast operation.

[0071] In addition, in order to enhance the time sequence information, a fixed position encoding is superimposed in the time dimension to obtain the representation with position encoding .

[0072] Subsequently The stacked L-layer Spatio-Temporal Transformer is modeled. Each layer first performs attention computation in the time dimension, then performs attention computation in the node dimension, and finally further maps through a feed-forward network (FFN) to obtain the final representation of the current Transformer layer After L-layer stacking, the final hidden state is obtained , The final interpolation result is obtained through a multi-layer perceptron. The mask mechanism ensures that the model only propagates effective information based on the true observed values.

[0073] Step 4, the gating mechanism is introduced, and the expert output is dynamically weighted and combined according to the input context;

[0074] To adaptively select the most suitable expert at each spatial node and time step, the present application proposes a gating network based on memory attention. Unlike traditional MoE models, the gating mechanism of the present application explicitly uses expert errors for dynamic supervision, thereby realizing fine-grained, spatio-temporally aware routing decisions.

[0075] Given the original input , the mask , and the hidden state of each expert , first fuse the input and the mask into a unified query embedding:

[0076] ;

[0077] wherein is a learnable linear transformation matrix representing X, used to project the input feature into the query space dimension; MLP refers to a multi-layer perceptron (MLP) whose input is the mask , and the output is a representation of the same dimension as , which serves to nonlinearly encode the missing patterns, is a learnable fusion weight. Subsequently, the learned memory unit is used to refine the attention of the query:

[0078] ;

[0079] Self-attention transformation is applied to the hidden state of each expert:

[0080] ;

[0081] , , respectively correspond to the query (Query) projection matrix of the e-th expert, used to project the hidden state Mapping to query, key, value vector space, expert Gating probability Calculated by cosine similarity:

[0082] ;

[0083] And select the expert with the highest probability, that is The final interpolation result is the combination of the results of each expert corresponding to each node and each time step.

[0084] For dynamic guidance gating selection, define two types of supervision labels for each expert according to the reconstruction error of each expert And Specifically as follows:

[0085] If the error of the expert exceeds the high quantile threshold , let ; Otherwise .

[0086] If the error of the expert is lower than the low quantile threshold And the corresponding true value exceeds the threshold , let ; Otherwise .

[0087] These dynamically generated labels drive the gating network to preferentially select more accurate experts and suppress less accurate experts. Formally, the gating loss is defined as:

[0088] ;

[0089] Where, Control the relative strength of the "best selection" encouragement item and the "worst avoidance" penalty item.

[0090] Another aspect of the present application provides a spatiotemporal sequence interpolation device for heterogeneous missing, comprising the following steps:

[0091] The sequence acquisition module standardizes the original spatiotemporal sequence obtained by the sensor network, and synchronizes the observation tensor and the missing mask according to the mask tensor;

[0092] The space-dependent modeling module starts a static space expert and a dynamic space expert submodule in parallel, the static space expert submodule generates a hidden state and a space interpolation result by driving graph convolution with a fixed geographical adjacency matrix, and the dynamic space expert submodule generates a time-varying adjacency matrix by self-attention after fusing historical observations, node embedding and periodic time embedding at each time step, and then obtains a multi-order support set through high-order diffusion, extracts a hidden state by using the support set, and outputs a space interpolation result through structure perception decoding and gated fusion of the hidden state;

[0093] The time-dependent modeling module starts a short-term time expert and a long-term period expert submodule in parallel, and the short-term time expert and the long-term period expert submodule both include: fusing the input observation and the mask, and then mapping the original channel to a hidden dimension through a multilayer perception machine to obtain a preliminary representation The design of the learnable mask vector restores the preliminary representation, projects the periodic time embedding into the hidden space, and broadcasts it in the node dimension, then superimposes the fixed position coding in the time dimension to obtain a representation with position coding, models through stacked space-time attention, and finally obtains a hidden state and a time interpolation result.

[0094] The result output module splices the observation tensor, the missing mask and the hidden state of each expert into a query embedding, calculates the cosine similarity between the query embedding after refinement by memory attention and the state after transformation by each expert self-attention, generates normalized expert weights, and sums the interpolation features of each expert weighted by the weights to obtain the final interpolation result; and dynamically calculates the best selection and the worst avoidance label according to the reconstruction error of each expert.

[0095] In another aspect, the application provides an electronic device, including: one or more processors; a memory for storing one or more programs, wherein when the one or more programs are executed by the one or more processors, the one or more processors implement the method.

[0096] In another aspect, the application provides a computer-readable storage medium having stored executable instructions, which are executed by a processor to enable the processor to implement the method.

[0097] Those skilled in the art will appreciate that embodiments of the present application can be readily used as software, hardware, or a combination of software and hardware. In a software embodiment, the methods can be tangibly embodied in a machine-readable storage medium having stored thereon instructions that can be used to program a computer to perform any of the methods. The software implementation can be initialized by loading and executing a set of instructions arranged to perform one of the methods into the computer's memory. Alternatively, hard-wired circuitry can be used in place of, or in combination with, software instructions. Thus, the

[0098] The present application is described in reference to the flowchart and / or block diagrams of the method, apparatus (system) and computer program product according to embodiments of the application. It will be understood that each block of the flowchart and / or block diagrams, and combinations of blocks in the flowchart and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general purpose computer, special purpose computer, embedded processing device, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in the flowchart and / or block diagram block or blocks. Figure 1 one or more functions specified in one or more of the flowchart and / or block diagram block or blocks. Figure 1 one or more functions specified in one or more of the flowchart and / or block diagram block or blocks.

[0099] These computer program instructions can also be stored in a computer- readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instructions which implement the flowchart and / or block diagram block or blocks. Figure 1 one or more functions specified in one or more of the flowchart and / or block diagram block or blocks. Figure 1 one or more functions specified in one or more of the flowchart and / or block diagram block or blocks.

[0100] These computer program instructions can also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer-implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the flowchart and / or block diagram block or blocks. Figure 1 one or more functions specified in one or more of the flowchart and / or block diagram block or blocks. Figure 1 one or more functions specified in one or more of the flowchart and / or block diagram block or blocks.

[0101] While preferred embodiments of the application have been described, modifications and alterations thereto will occur to those skilled in the art upon reading the preceding description. In particular, it will be apparent to those skilled in the art that parts can be added to, or substituted for, parts of the described embodiment. It is therefore desired to be secured to the appended claims as they follow.

[0102] It will be apparent to those skilled in the art that various modifications and variations can be made to the present application without departing from the spirit or scope of the application. Thus, it is intended that the present application cover modifications and variations of this application provided they come within the scope of the appended claims and their equivalents.

Claims

1. A spatiotemporal sequence interpolation method for heterogeneous missing sequences, characterized in that, Includes the following steps: The original spatiotemporal sequences acquired by the sensor network are standardized, and the observation tensor is synchronized with the missing mask based on the mask tensor; The static spatial expert submodule and the dynamic spatial expert submodule are launched in parallel. The static spatial expert submodule uses a fixed geographic adjacency matrix to drive graph convolution to generate hidden states and spatial interpolation results. The dynamic spatial expert submodule fuses historical observations, node embeddings and periodic time embeddings at each time step and generates a time-varying adjacency matrix through self-attention. Then, it obtains a multi-order support set through high-order diffusion. The hidden state is extracted using the support set. The hidden state is then fused with structure-aware decoding and gating to output the spatial interpolation result. The short-term and long-term expert submodules are launched in parallel. Both the short-term and long-term expert submodules include: fusing the input observations and the mask, and then using a multilayer perceptron to uniformly map the original channels to the hidden dimensions to obtain a preliminary representation. The design recovers the initial representation by learningable mask vectors, projects periodic temporal embeddings into the hidden space and broadcasts them in the node dimension. Then, fixed positional codes are superimposed in the temporal dimension to obtain a representation with positional codes. The model is then performed by stacked spatiotemporal attention to finally obtain the hidden state and temporal interpolation results. The observation tensor, missing mask, and hidden states of each expert are concatenated to form a query embedding. The query embedding is refined by memory attention and its cosine similarity to the states of each expert after self-attention transformation is calculated to generate normalized expert weights. The imputation features of each expert are weighted and summed according to the weights to obtain the final imputation result. The best choice and worst avoidance labels are dynamically calculated based on the reconstruction error of each expert.

2. The method according to claim 1, characterized in that, The dynamic spatial expert submodule concatenates intraday phase embeddings and intraweek phase embeddings of upstream and downstream nodes, and then obtains the query and key through linear mapping. After Softmax normalization, it is multiplied element-wise with the static prior adjacency matrix to form a bidirectional time-varying adjacency matrix that simultaneously reflects prior geographical relevance and real-time data relevance.

3. The method according to claim 1, characterized in that, Higher-order diffusion generates a set of support matrices covering first-order to multi-order neighbors by continuously exponentially expanding the adjacency matrix to a preset maximum order.

4. The method according to claim 1, characterized in that, The short-term expert submodule takes observations from the most recent historical window as input, while the long-term period expert submodule takes observations from the same window extended to seven days before and after as input.

5. The method according to claim 1, characterized in that, The learnable mask vector replaces the all-zero vector at the masked position with the same dimension in the input layer, and is then broadcast and added to the real observation vector to generate the restored initial representation.

6. The method according to claim 1, characterized in that, Memory attention is specifically calculated by taking a scaled dot product of the learnable memory unit matrix and the query embedding, and then summing the results with weights.

7. The method according to claim 1 or 6, characterized in that, Dynamic supervision sets the best choice label to 1 only when the actual observed value is higher than the preset threshold and the expert error is in the low quantile. At the same time, when the high error exceeds the high quantile, the worst avoidance label is set to 0.

5. Adaptive reward and punishment for expert routing is achieved through backpropagation of gating loss.

8. A spatiotemporal sequence interpolation device for heterogeneous missing sequences, characterized in that, Includes the following steps: The sequence acquisition module standardizes the raw spatiotemporal sequence acquired by the sensor network and synchronizes the observation tensor with the missing mask based on the mask tensor. The spatial dependency modeling module starts the static spatial expert submodule and the dynamic spatial expert submodule in parallel. The static spatial expert submodule uses a fixed geographic adjacency matrix to drive graph convolution to generate hidden states and spatial interpolation results. The dynamic spatial expert submodule fuses historical observations, node embeddings and periodic time embeddings at each time step and generates a time-varying adjacency matrix through self-attention. Then, it obtains a multi-order support set through high-order diffusion. The hidden state is extracted using the support set. The hidden state is then fused with structure-aware decoding and gating to output the spatial interpolation result. The time-dependent modeling module initiates short-term time expert submodules and long-term period expert submodules in parallel. Both the short-term and long-term period expert submodules include: fusing input observations and masks, and then using a multilayer perceptron to uniformly map the original channels to the hidden dimensions to obtain a preliminary representation. The design recovers the initial representation by learningable mask vectors, projects periodic temporal embeddings into the hidden space and broadcasts them in the node dimension. Then, fixed positional codes are superimposed in the temporal dimension to obtain a representation with positional codes. The model is then performed by stacked spatiotemporal attention to finally obtain the hidden state and temporal interpolation results. The output module concatenates the observation tensor, missing mask, and hidden states of each expert into a query embedding. After the query embedding is refined by memory attention, the cosine similarity is calculated with the state after the self-attention transformation of each expert to generate normalized expert weights. The imputation features of each expert are weighted and summed according to the weights to obtain the final imputation result. The best choice and worst avoidance labels are dynamically calculated based on the reconstruction error of each expert.

9. An electronic device, characterized in that, include: One or more processors; A memory for storing one or more programs, wherein when the one or more programs are executed by the one or more processors, the one or more processors cause the one or more processors to implement the method of any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, It stores executable instructions that, when executed by a processor, cause the processor to perform the method described in any one of claims 1 to 7.

Images