Self-supervised spatiotemporal data completion method and system based on temporal semantic attention

Abstract

The present invention discloses a self-supervised spatiotemporal data completion method and system based on temporal semantic attention. The method obtains spatiotemporal data and divides incomplete time series into semantic segment units through temporal segment modeling. The method utilizes a cross-attention mechanism to mine potential temporal dependency patterns between segments and obtains segmented attention calculation results. A temporal comparison model is constructed based on the segmented attention calculation results, and a strong generalized embedding for temporal modeling is obtained through a self-supervised method. The embedding obtained through temporal modeling is used as the temporal feature of each node, and spatial domain modeling is performed through a graph convolutional network to obtain spatial domain modeling results. This method breaks through the traditional model's strong reliance on data integrity and can effectively capture nonlinear spatiotemporal correlations in data-missing scenarios, providing a highly robust temporal embedding foundation for subsequent spatiotemporal node-level tasks.

Description

Technical Field

[0001] The present invention belongs to the technical field of urban spatiotemporal data completion, and the present invention relates to a self-supervised spatiotemporal data completion method and system based on temporal semantic attention. Background Art

[0002] In urban traffic monitoring, sensors may lose data due to malfunctions, communication interruptions, power failures, etc., which has a negative impact on the effectiveness of subsequent applications.

[0003] Complete spatiotemporal data is fundamental to tasks such as traffic management and environmental monitoring in smart cities. For example, PM2.5 concentration interpolation requires modeling both long-term temporal trends and cross-regional spatial correlations, while traffic data completion requires processing the dynamically changing spatiotemporal semantic relationships within large-scale road networks.

[0004] Urban spatiotemporal data (such as traffic flow, environmental monitoring, and sensor networks) often have multidimensionality (e.g., temporal, spatial, and semantic attributes), high missingness rates (e.g., sensor failures, transmission interruptions), and complex spatiotemporal dependencies (e.g., the spatiotemporal propagation of traffic flow and dynamic inter-regional correlations). For example, the coexistence of random missingness and non-random missingness (e.g., sensor failures for multiple consecutive days) in traffic data makes it difficult for traditional interpolation methods to capture nonlinear spatiotemporal correlations. Summary of the Invention

[0005] The purpose of the present invention is to provide a self-supervised spatiotemporal data completion method and system based on temporal semantic attention. This method breaks through the traditional model's strong dependence on data integrity, can effectively capture nonlinear spatiotemporal correlations in data missing scenarios, and provides a highly robust temporal embedding foundation for subsequent spatiotemporal node-level tasks.

[0006] The technical solutions for achieving the purpose of the present invention are:

[0007] A self-supervised spatiotemporal data completion method based on temporal semantic attention includes the following steps:

[0008] S01: Obtain spatiotemporal data and divide the incomplete time series into semantic segment units through time series segmentation modeling;

[0009] S02: Use the cross-attention mechanism to mine the potential temporal dependency patterns between segments and obtain the segmented attention calculation results;

[0010] S03: Build a temporal contrast model based on the segmented attention calculation results, and obtain a strong generalization embedding for temporal modeling through self-supervision methods;

[0011] S04: The embedding obtained through time series modeling is used as the time series feature of each node, and spatial domain modeling is performed through the graph convolutional network to obtain the spatial domain modeling result.

[0012] In the preferred technical solution, after step S04, the method further includes designing a segment-to-segment reconstruction loss function to constrain the consistency between local segment features and global temporal semantics.

[0013] In a preferred technical solution, the method of dividing the incomplete time series into semantic segment units by time series segment modeling in step S01 includes:

[0014] S11: Perform long-short time dependency encoding. The encoding result contains global time information. For the input time series data ,in, is a real number, Represents the number of nodes, and Represent the dimension and length of the time series features respectively. After splicing the time information, we get data containing global time information. , Greater than , represents the new time series feature dimension:

[0015] ;

[0016] in, Represents a splicing operation, Indicates time information including global time information;

[0017] S12: Perform short-term time-dependent mapping coding. The mapping coding only considers the time information of a day and uses position coding to input the time information of each time series.

[0018] S13: In the timing segmentation stage, each input channel First, it is normalized to zero mean and unit standard deviation through the reversible instance; then, Split into consecutive non-overlapping blocks ,in is the number of blocks, is the original dimension of each block, and then they are embedded as , the embedding method uses a simple linear layer as a block embedder with the dimension of .

[0019] In the preferred technical solution, step S02 utilizes the cross-attention mechanism to mine potential temporal dependency patterns between segments, including:

[0020] S21: Inter-segment attention uses the temporal segment as the basic computational unit and represents the input of each segment-related module as ,in and are the length and dimension of the input, the initial hidden state , for single-head attention, the input sequence Projected by three projection matrices to obtain the query ,key Sum ;

[0021] Query segment and key segments Correlation measure between for:

[0022] ;

[0023] in, is the dot product operation between two matrices of the same size;

[0024] Each query segment Relevance measure to all key segments will pass Function is normalized to obtain the aggregate weight : ;

[0025] No. Output of segment positions All value segments The weighted sum of is calculated as follows:

[0026] ;

[0027] By splicing all outputs To obtain the output of the inter-segment attention module, the calculation process is:

[0028] ;

[0029] After the linear layer, the final output is , realizing multi-scale segment attention modeling;

[0030] S22: Calculate the segmented temporal reconstruction loss function for:

[0031] ;

[0032] in, represents the total number of segments reconstructed, represents the true value of the reconstructed segment, represents the temporal cross-modeling results of the reconstructed segments;

[0033] The goal of the segmented temporal reconstruction loss function is to minimize the overall loss of each segment.

[0034] In the preferred technical solution, the method of obtaining a strong generalization embedding for time series modeling through a self-supervisory method in step S03 includes:

[0035] Modeling the intermediate representation of segment cross Spatiotemporal data as input , strong samples and weak samples are generated by the enhancement strategy in contrastive learning, that is, its strong enhancement view is expressed as , and its weakly enhanced view is expressed as ; The enhanced view is then encoded, and the encoder first maps it to a high-dimensional latent representation , A layer of neural network is used for encoding, and the encoding result is expressed as , and get the corresponding two views and Strong Enhancement Coding and weak enhancement coding ;

[0036] Through the autoregressive model All the The time series data before the moment is encoded and summarized as a context vector , , ,in, yes The hidden dimension of

[0037] Using a log-bilinear model Preserve Input and The mutual information between them is ,in, is an exponential function, is the linear transformation parameter.

[0038] In the preferred technical solution, step S03 further includes:

[0039] Autoregressive Model Use Transformer, stack L identical layers to generate the final features, and add a token to the input , whose states serve as representative context vectors in the output;

[0040] The features are applied to a linear projection layer which maps the features to a hidden dimension, i.e. ; Then, the output of this linear projection is sent to Transformer to get the output result , the input feature contains tokens and becomes , where subscript 0 represents the input of the first layer; next, Passed to the Transformer layer, as shown in the following formula:

[0041] ;

[0042] ;

[0043] in, , Indicates the The result of the layer, Is the output of the attention calculation, an intermediate result, represents multi-head attention calculation, Indicates the standardization of values. represents a multilayer perceptron;

[0044] Reappend the token value from the final output so that , as the input of the time series contrast model.

[0045] In the preferred technical solution, performing spatial domain modeling by using a graph convolutional network in step S04 includes:

[0046] Compute the learned adaptive adjacency matrix :

[0047] ;

[0048] in, is the activation function;

[0049] Get a predefined adjacency matrix , the adaptive adjacency matrix and Combined together, we define multi-directional multi-graph spatial convolution, Including the forward space adjacency matrix and the backward adjacency matrix , the calculation process of airspace modeling is: ;

[0050] in, yes The hidden state of the layer, is the result of airspace modeling, 、 as well as is the linear transformation parameter;

[0051] Use residual connections and batch normalization to obtain the final enhanced results ;

[0052] Loss function in the spatial modeling phase for:

[0053] ;

[0054] in, Represents the number of points involved in the airspace loss calculation, is the output of airspace modeling, Model the corresponding label value for the spatial domain.

[0055] In the preferred technical solution, the segment-to-segment reconstruction loss function includes: a segment reconstruction loss function, a time domain cross loss function, and a maximum similarity loss function;

[0056] In the time domain cross loss function, a strong enhanced context vector is used / Weakly enhanced context vector Cross-predict future time steps for weak / strong reinforcement The time domain cross loss function minimizes the dot product between the predicted result and the true result of the same sample, while maximizing the dot product with other samples in the same enhancement batch. The dot product of two loss functions and The formula is:

[0057] ;

[0058] ;

[0059] in, Representative time step, represents the total number of time steps in the future, Representative A moment, represents a linear transformation, represents the exponential function, Indicates a weakly enhanced future moment The prediction results, Indicates a future moment of strong reinforcement The prediction results;

[0060] For the maximum similarity loss function, under the time series comparison model, given input samples, obtained from two augmented views context, for context ,Will Expressed as The positive sample of as positive pairs, while the rest of the context from other inputs in the same batch is considered negative samples, derive a context contrast loss to maximize the similarity between positive pairs and minimize the similarity between negative pairs, and obtain the context contrast loss function : ;

[0061] in, , Represents 2 vectors and Normalization of the dot product between is an indicator function if and only if When the value is 1, is the temperature parameter;

[0062] The total loss function of the temporal cross-contrast learning module is:

[0063] ;

[0064] in, , and is a hyperparameter that represents the relative weight of each loss, is the piecewise reconstruction loss function, and is the time domain cross loss, is the maximum similarity loss function.

[0065] The present invention also discloses a self-supervised spatiotemporal data completion system based on temporal semantic attention, comprising:

[0066] The time series segmentation modeling module obtains spatiotemporal data and divides the incomplete time series into semantic segment units through time series segmentation modeling;

[0067] The segmented attention calculation module uses the cross-attention mechanism to mine the potential temporal dependency patterns between segments and obtain the segmented attention calculation results;

[0068] The temporal contrast model construction module builds a temporal contrast model based on the segmented attention calculation results and obtains a strong generalization embedding for temporal modeling through self-supervision methods;

[0069] The spatial domain modeling enhancement module uses the embedding obtained through temporal modeling as the feature of each node, performs spatial domain modeling through a graph convolutional network, and obtains enhanced results.

[0070] The present invention further discloses a computer storage medium on which a computer program is stored. When the computer program is executed, the above-mentioned self-supervised spatiotemporal data completion method based on temporal semantic attention is implemented.

[0071] Compared with the prior art, the present invention has the following significant advantages:

[0072] The present invention adopts node-level temporal cross-attention and self-supervision modeling. The time series data is first segmented. Compared with a single time point, segmentation not only makes the modeling time series semantics more complete, but also makes block time series modeling more reasonable for data completion problems. Single time point reconstruction can achieve good results using surrounding mean completion, and segment time series reconstruction can fully consider long time series relationships. Time series segment modeling is performed through segmented cross-attention, and different time series periods are taken into account in time series modeling. In the attention calculation process, the original attention calculation between points is replaced by calculation between segments, which greatly saves the amount of calculation. And through the self-supervision of time series data between nodes, the generalization of time series modeling is enhanced.

[0073] This paper proposes a robust feature embedding framework for time series dependency failures, aiming to achieve robust representation of missing data through segmented modeling and self-supervised learning. This approach overcomes the traditional model's strong reliance on data integrity and provides a highly robust time series embedding foundation for subsequent spatiotemporal node-level tasks. This approach has significant application value in scenarios where data missingness is common, such as smart cities and the Internet of Things. BRIEF DESCRIPTION OF THE DRAWINGS

[0074] Figure 1 Flowchart of the self-supervised spatiotemporal data completion method based on temporal semantic attention of the present invention;

[0075] Figure 2 This is a schematic diagram of the node-level temporal cross-attention self-supervision principle of an embodiment of the present invention. DETAILED DESCRIPTION

[0076] The principle of the present invention is as follows: the present invention divides incomplete time series into semantic segment units through temporal segment modeling, and uses the cross-attention mechanism to explore the potential temporal dependency patterns between segments to alleviate the impact of data fragmentation on feature integrity; secondly, based on the segmentation results, a multi-scale self-supervised task is constructed, and the model's ability to capture complex temporal laws is enhanced by explicitly modeling sudden events, periodic trends and long-term dependencies; finally, a segment-to-segment reconstruction loss function is designed to simultaneously improve the feature reconstruction accuracy and model generalization performance by constraining the consistency of local segment features with the global temporal semantics.

[0077] Example 1:

[0078] A self-supervised spatiotemporal data completion method based on temporal semantic attention includes the following steps:

[0079] S01: Obtain spatiotemporal data and divide the incomplete time series into semantic segment units through time series segmentation modeling;

[0080] S02: Use the cross-attention mechanism to mine the potential temporal dependency patterns between segments and obtain the segmented attention calculation results;

[0081] S03: Build a temporal contrast model based on the segmented attention calculation results, and obtain a strong generalization embedding for temporal modeling through self-supervision methods;

[0082] S04: The embedding obtained through temporal modeling is used as the feature of each node, and spatial domain modeling is performed through the graph convolutional network to obtain enhanced results.

[0083] The following describes in detail the self-supervised spatiotemporal data completion method based on temporal semantic attention, taking a preferred embodiment as an example. The method includes the following steps: Figure 1 、 Figure 2 As shown:

[0084] S1: The spatiotemporal data with missing data are input into the model. First, the time series data is segmented and divided into semantic segment units according to the time series to ensure semantic integrity during the modeling process.

[0085] S2: After the time series data is segmented, the cross-attention mechanism is used to mine the potential temporal dependency patterns between segments.

[0086] S3: Build a temporal contrast model based on the segmented attention calculation results, and obtain a strong generalization embedding for temporal modeling through self-supervision.

[0087] S4: The embedding obtained through temporal modeling is the feature of each node, and spatial domain modeling is further realized through graph convolutional networks.

[0088] S5: Design a segment-to-segment reconstruction loss function to constrain the consistency between local segment features and global temporal semantics.

[0089] Specifically, S1: First, the node timing information is embedded. This process mainly includes three stages.

[0090] S11: In the long-short time dependency encoding stage, most of the existing deep learning methods split the spatiotemporal data through batch processing technology, destroying the long-term time dependency in each time series. In order to regain the long-term dependency longer than one sequence length, a new encoding mechanism is designed, and the encoding result contains global time information (the time dependency is longer than one sequence length). For example, "hour of the day" and "day of the week" are encoded to enhance the ability to capture long-term time dependencies. Unlike the position encoding in the transformer that only processes the time dependency in each sequence, the proposed module processes the global time dependency. For the input traffic data ,in Represents the number of nodes, and Represent the dimension and length of the time series features respectively. The splicing process is shown in Formula 1. After splicing the time information, we get , Greater than , represents the new time series feature dimension.

[0091] (1)

[0092] in Represents a splicing operation, Represents time information, such as "hour of the day", "day of the week", and other periodic information.

[0093] S12: During the short-term time-dependent mapping encoding phase, this invention proposes to use mapping encoding to encode time to ensure sequence agnosticism. Mapping time encoding only considers time information within a day. This paper uses position encoding to input the time information of each time series. The calculation process is shown in Equation 2.

[0094] (2)

[0095] in, It is the first The index of a time point, It is a time mapping of day units. represents the sine function, Represents the cosine function. Represents a splicing operation. is the sinusoidal encoding of the timing, It is the cosine coding of the time series. and As the final time code Timing Dependencies The calculation of is shown in formula 3.

[0096] (3)

[0097] S13: In the timing segmentation stage, each input channel First, it is normalized to zero mean and unit standard deviation via a reversible instance. Then, Split into consecutive non-overlapping blocks ,in is the number of blocks, is the original dimension of each block, and the time series length is , the block length is , so the total number of input blocks is , which is calculated as shown in Formula 4, where Indicates the horizontal sliding step size.

[0098] (4)

[0099] The present invention will Embed as , a simple linear layer is used as a block embedder to create the dimension Because the complexity of attention is quadratic with the number of time segments, if each segment represents an attention calculation point, rather than each time point, this obviously reduces the number of calculation points and the overall amount of attention calculation.

[0100] S2: The inter-segment cross attention mainly includes two stages: first, the temporal inter-segment attention calculation, and then the calculation of the segment temporal reconstruction loss function.

[0101] S21: Inter-segment cross attention is a key module in temporal modeling. Inter-segment attention uses temporal segments as the basic computational unit, rather than a single time point, which makes modeling semantics more complete. This paper proposes to express the input of each segment-related module as ,in and are the length and dimension of the input, the initial hidden state Formally, for the single-head attention case, the input sequence will be projected by three projection matrices to obtain query, key and value, i.e. , , , 、 、 is the projection matrix.

[0102] Query segment and key segments Correlation measure between It can be calculated by formula 5.

[0103] (5)

[0104] in, is the dot product operation between two matrices of the same size.

[0105] For each query segment , its correlation with all key segments is measured will pass Function is normalized to obtain the aggregate weight , the calculation process is shown in Formula 6.

[0106] (6)

[0107] No. Output of segment positions All value segments The weighted sum of is calculated as shown in Formula 7.

[0108] (7)

[0109] Finally, by splicing all outputs The output of the inter-segment attention module is obtained. The calculation process is shown in Formula 8.

[0110] (8)

[0111] After the linear layer, the final output is The segment length can be customized according to the temporal modeling semantics, thus achieving multi-scale segment attention modeling.

[0112] S22: In the segmented time series reconstruction loss function, for the entire time series data of segmented cross modeling, the present invention intends to use reconstruction loss. The calculation process of is shown in Formula 9, and its goal is to minimize the overall loss of each segment.

[0113] (9)

[0114] in, represents the total number of reconstructed segments, represents the true value of the reconstructed segment, Represents the temporal cross-modeling results of the reconstructed segments.

[0115] S3: During the temporal contrastive learning phase, although the present invention employs segmented cross-modeling and segment reconstruction, the supervised information in the absence of data still results in a lack of generalization in the model. The present invention proposes to add temporal contrastive learning to enable the model to model macroscopic information. Specifically, this approach involves generating strong and weak samples from the segmented cross-modeling results, and then training the model through cross-contrast learning. This collaborative training improves the generalization of temporal modeling.

[0116] Modeling the intermediate representation of segment cross Spatiotemporal data as input . Generate strong samples and weak samples through the enhancement strategy in contrastive learning. And express its strong enhancement view as , and its weakly enhanced view is expressed as For input spatiotemporal data , the encoder will Mapping to high-dimensional latent representation . Use a layer of neural network for encoding and define , and get the corresponding two views and Strong Enhancement Coding and weak enhancement coding .

[0117] Context vector generation stage: through autoregressive model All the The time series data before the moment is encoded and summarized as a context vector , , ,in yes The hidden dimension of .

[0118] The mutual information stage relies on the context vector generated in the previous step. In order to predict the future time steps, a log-bilinear model is used. To preserve input and The mutual information between .in, is an exponential function, is the linear transformation parameter.

[0119] This paper proposes to use Transformer as an autoregressive model and stack L identical layers to generate the final features. Add a token to the input , whose state serves as a representative context vector in the output. First, the feature is applied to a linear projection layer that maps features to a hidden dimension, i.e. The output of this linear projection is then sent to the Transformer to get the output The input features contain tokens and become , where the subscript 0 indicates that it is the input of the first layer. Next, Passed to the Transformer layer, as shown in Equation 13.

[0120] (13)

[0121] Finally, the token value is re-appended from the final output so that This context vector will become the input of the contrast module.

[0122] S4: In the spatial domain modeling phase, the learned adaptive adjacency matrix is ​​calculated :

[0123] ;

[0124] in, is the activation function;

[0125] Since the predefined adjacency matrix It is constructed based on expert experience (topology-based or distance-based) and cannot fully reflect the complex correlations between nodes. Therefore, the learned adaptive adjacency matrix and Combined together, we define multi-directional multi-graph spatial convolution, Including the forward space adjacency matrix and the backward adjacency matrix , the airspace modeling calculation process is shown in Formula 10.

[0126] (10)

[0127] in, yes The hidden state of the layer, is the result of airspace modeling, 、 and is the linear transformation parameter. Then residual connection and batch normalization are used to obtain the final enhanced result .by and the true value The mean absolute error between As a loss function for spatiotemporal augmentation tasks.

[0128] Multi-directional multi-graph spatial convolution includes forward, backward, and graph convolutional networks of multiple spatial graphs.

[0129] Loss function in the spatial modeling phase for:

[0130] ;

[0131] in, Represents the number of points involved in the airspace loss calculation, is the output of airspace modeling, Model the corresponding label value for the spatial domain.

[0132] S5: In the loss function calculation process, relying on strong enhancement , weak enhancement , the present invention uses strong enhancement context to predict the future time steps of weak reinforcement , and vice versa. The contrast loss attempts to minimize the dot product between the predicted result and the true result of the same sample, while maximizing the dot product with other samples in the same augmentation batch. Therefore, the two loss functions and As shown in Formula 11 and Formula 12.

[0133] (11)

[0134] (12)

[0135] in, Representative Step, Representative A moment. Representatives The linear transformation of Represents the exponential function.

[0136] Under the temporal cross-comparison model, a context comparison module is designed. input samples, will be obtained from its two augmented views context. ,Will Expressed as Therefore, are considered as positive pairs. Meanwhile, the rest of the context from other inputs in the same batch is considered Therefore, a contextual contrastive loss can be derived to maximize the similarity between positive pairs and minimize the similarity between negative pairs. The context contrast loss function is defined, and the calculation process is shown in Formula 14. Given a context , and compare it with the positive sample Divide the similarity between it and all other samples (including positive and negative pairs) to normalize the loss.

[0137] (14)

[0138] in , Represents 2 vectors and Normalization of the dot product between (i.e. cosine similarity), is an indicator function if and only if When the value is 1, is the temperature parameter. The overall self-supervised loss is a combination of two temporal contrastive losses and a contextual contrastive loss. The total loss function of the temporal cross-contrastive learning module is shown in Equation 15.

[0139] (15)

[0140] in, , and is a hyperparameter that represents the relative weight of each loss. is the piecewise reconstruction loss function, and is the time domain cross loss, is the maximum similarity loss function.

[0141] In another embodiment, a computer storage medium stores a computer program that, when executed, implements the aforementioned method for self-supervised spatiotemporal data completion based on temporal semantic attention. The specific implementation is the same as the above method and will not be repeated here.

[0142] In another embodiment, a self-supervised spatiotemporal data completion system based on temporal semantic attention includes:

[0143] The time series segmentation modeling module obtains spatiotemporal data and divides the incomplete time series into semantic segment units through time series segmentation modeling;

[0144] The segmented attention calculation module uses the cross-attention mechanism to mine the potential temporal dependency patterns between segments and obtain the segmented attention calculation results;

[0145] The temporal contrast model construction module builds a temporal contrast model based on the segmented attention calculation results and obtains a strong generalization embedding for temporal modeling through self-supervision methods;

[0146] The spatial domain modeling enhancement module uses the embedding obtained through temporal modeling as the feature of each node, performs spatial domain modeling through a graph convolutional network, and obtains enhanced results.

[0147] The specific implementation is the same as the above method and will not be repeated here.

[0148] The accuracy of the missing data completion method for urban spatiotemporal data based on temporal segmented attention and spatiotemporal contrast learning was verified under different datasets and different missing rates. The dataset mainly contains fixed-point monitoring data of the road network, and the monitoring data is somewhat sparse and missing. The datasets used include: PEMS-Bay, which consists of traffic speed time series from 325 sensors in the Bay Area from January 1, 2017, to May 31, 2017, and contains 52,116 time points; METR-LA, a four-month traffic speed dataset from 207 sensors in Los Angeles, has 1,515 edges, covers 119 days, and contains 34,272 (= 12 x 24 x 119) time points, with an inherent missingness rate of 8.1%; PEMS04, which includes 307 sensors, each collecting data with three dimensional features, for a total of 16,992 time points, or 59 x 24 x 12 (data is collected every 5 minutes, resulting in 12 data points per hour. Since there are 24 hours in a day, data is collected over 59 days, resulting in 59 x 24 x 12 = 16,992). In addition, there are the PEMS03, PEMS07, and PEMS08 datasets. Table 1 shows examples and data characteristics of each dataset.

[0149] Table 1 Dataset related information statistics

[0150]

[0151] For project evaluation, this project intends to use MSE, RMSE, and R² for reconstruction loss functions; and maximum similarity as the loss metric for contrastive learning. The baseline comparison methods primarily include mainstream methods such as M-RNN, GAIN, E2GAN, NAOMI, KCN, and IGNNK.

[0152] The experimental results are shown in Table 2:

[0153] Table 2 Experimental results

[0154]

[0155] The experimental results verify the overall performance of the present invention under various conditions of continuous missing data; the present invention is superior to relevant mainstream methods in terms of the accuracy of repairing continuous missing data.

[0156] The above embodiments are preferred implementation modes of the present invention, but the implementation modes of the present invention are not limited to the above embodiments. Any other changes, modifications, substitutions, combinations, and simplifications that do not deviate from the spirit and principles of the present invention should be considered as equivalent replacement methods and are included in the scope of protection of the present invention.

Claims

1. A self-supervised spatiotemporal data completion method based on temporal semantic attention, characterized by: The following steps are involved: S01: Acquire spatiotemporal data, wherein the spatiotemporal data includes road network fixed point monitoring data, wherein the road network fixed point monitoring data includes data collected by multiple sensors and containing certain timestamps, and divide the incomplete time series into semantic segment units through time series segmentation modeling; S02: Use the cross-attention mechanism to mine the potential temporal dependency patterns between segments and obtain the segmented attention calculation results; Using the cross-attention mechanism to mine potential temporal dependency patterns between segments includes: S21: Inter-segment attention uses the temporal segment as the basic computational unit and represents the input of each segment-related module as ,in, is a real number, and are the length and dimension of the input, the initial hidden state , For the temporal feature block, for single-head attention, the input sequence Projected by three projection matrices to obtain the query ,key Sum ; Query segment and key segments Correlation measure between for: ， in, is the dot product operation between two matrices of the same size; Each query segment Relevance measure to all key segments will pass Function is normalized to obtain the aggregate weight : ， No. Output of segment positions All value segments The weighted sum of is calculated as follows: ， By splicing all outputs To obtain the output of the inter-segment attention module, the calculation process is: ， After the linear layer, the final output is , realizing multi-scale segment attention modeling; S22: Calculate the segmented temporal reconstruction loss function for: ， in, represents the total number of reconstructed segments, represents the true value of the reconstructed segment, represents the temporal cross-modeling results of the reconstructed segments; The goal of the segmented temporal reconstruction loss function is to minimize the overall loss of each segment; S03: Build a temporal contrast model based on the segmented attention calculation results, and obtain a strong generalization embedding for temporal modeling through self-supervision methods; S04: The embedding obtained through time series modeling is used as the time series feature of each sensor node, and spatial domain modeling is performed through the graph convolutional network to obtain the spatial domain modeling result.

2. The self-supervised spatiotemporal data completion method based on temporal semantic attention according to claim 1 is characterized in that After step S04, the process also includes designing a segment-to-segment reconstruction loss function to constrain the consistency between local segment features and global temporal semantics.

3. The self-supervised spatiotemporal data completion method based on temporal semantic attention according to claim 1 is characterized in that: The method for dividing the incomplete time series into semantic segment units by time series segment modeling in step S01 includes: S11: Perform long-short time dependency encoding. The encoding result contains global time information. For the input time series data ,in, is a real number, Represents the number of nodes, and Represent the dimension and length of the time series features respectively. After splicing the time information, we get data containing global time information. , Greater than , represents the new time series feature dimension: ， in, Represents a splicing operation, Indicates time information including global time information; S12: Perform short-term time-dependent mapping coding. The mapping coding only considers the time information of a day and uses position coding to input the time information of each time series. S13: In the timing segmentation stage, each input channel First, it is normalized to zero mean and unit standard deviation through the reversible instance; then, Split into consecutive non-overlapping blocks ,in is the number of blocks, is the original dimension of each block, and then they are embedded as , the embedding method uses a simple linear layer as a block embedder with the dimension of .

4. The self-supervised spatiotemporal data completion method based on temporal semantic attention according to claim 1 is characterized in that Step S03: The method for obtaining a strong generalization embedding for time series modeling through a self-supervisory method includes: Modeling the intermediate representation of segment cross Spatiotemporal data as input , strong samples and weak samples are generated by the enhancement strategy in contrastive learning, that is, its strong enhancement view is expressed as , and its weakly enhanced view is expressed as ; The enhanced view is then encoded, and the encoder first maps it to a high-dimensional latent representation , A layer of neural network is used for encoding, and the encoding result is expressed as , and get the corresponding two views and Strong Enhancement Coding and weak enhancement coding ; Through the autoregressive model All the The time series data before the moment is encoded and summarized as a context vector , , ,in, yes The hidden dimension of Using a log-bilinear model Preserve Input and The mutual information between them is ,in, is an exponential function, is the linear transformation parameter.

5. The self-supervised spatiotemporal data completion method based on temporal semantic attention according to claim 4 is characterized in that: Step S03 also includes: Autoregressive Model Use Transformer, stack L identical layers to generate the final features, and add a token to the input , whose states serve as representative context vectors in the output; The features are applied to a linear projection layer which maps the features to a hidden dimension, i.e. ; Then, the output of this linear projection is sent to Transformer to get the output result , the input feature contains tokens and becomes , where subscript 0 represents the input of the first layer; next, Passed to the Transformer layer, as shown in the following formula: ， ， in, , Indicates the The result of the layer, Is the output of the attention calculation, an intermediate result, represents multi-head attention calculation, Indicates the standardization of values. represents a multilayer perceptron; Reappend the token value from the final output so that , as the input of the time series contrast model.

6. The self-supervised spatiotemporal data completion method based on temporal semantic attention according to claim 4 is characterized in that In step S04, spatial domain modeling is performed using a graph convolutional network, including: Compute the learned adaptive adjacency matrix : ， in, is the activation function; Get a predefined adjacency matrix , the adaptive adjacency matrix and Combined together, we define multi-directional multi-graph spatial convolution, Including the forward space adjacency matrix and the backward adjacency matrix , the calculation process of airspace modeling is: ， in, yes The hidden state of the layer, is the result of airspace modeling, 、 as well as is the linear transformation parameter; Use residual connections and batch normalization to obtain the final enhanced results ; Loss function in the spatial modeling phase for: ， in, Represents the number of points involved in the airspace loss calculation, is the output of airspace modeling, Model the corresponding label value for the spatial domain.

7. The self-supervised spatiotemporal data completion method based on temporal semantic attention according to claim 2 is characterized in that The segment-to-segment reconstruction loss functions include: segment reconstruction loss function, time domain intersection loss function, and maximum similarity loss function; In the time domain cross loss function, a strong enhanced context vector is used / Weakly enhanced context vector Cross-predict future time steps for weak / strong reinforcement The time domain cross loss function minimizes the dot product between the predicted result and the true result of the same sample, while maximizing the dot product with other samples in the same enhancement batch. The dot product of two loss functions and The formula is: ， ， in, Representative time step, represents the total number of future time steps, Representative A moment, represents a linear transformation, represents the exponential function, Indicates a weakly enhanced future moment The prediction results, Indicates a future moment of strong reinforcement The prediction results; For the maximum similarity loss function, under the time series comparison model, given input samples, obtained from two augmented views context, for context ,Will Expressed as The positive sample of as positive pairs, and the rest of the context from other inputs in the same batch is considered negative samples, derive a context contrast loss to maximize the similarity between positive pairs and minimize the similarity between negative pairs, and obtain the context contrast loss function : ， in, , Represents 2 vectors and Normalization of the dot product between is an indicator function if and only if When the value is 1, is the temperature parameter; The total loss function of the temporal cross-contrast learning module is: ， in, , and is a hyperparameter that represents the relative weight of each loss, is the piecewise reconstruction loss function, and is the time domain cross loss, is the maximum similarity loss function.

8. A self-supervised spatiotemporal data completion system based on temporal semantic attention, characterized by: include: A time series segmentation modeling module acquires spatiotemporal data, including road network fixed point monitoring data, including data collected by multiple sensors containing certain timestamps, and divides the incomplete time series into semantic segment units through time series segmentation modeling; The segmented attention calculation module uses the cross-attention mechanism to mine the potential temporal dependency patterns between segments and obtain the segmented attention calculation results; Using the cross-attention mechanism to mine potential temporal dependency patterns between segments includes: S21: Inter-segment attention uses the temporal segment as the basic computational unit and represents the input of each segment-related module as ,in, is a real number, and are the length and dimension of the input, the initial hidden state , For the temporal feature block, for single-head attention, the input sequence Projected by three projection matrices to obtain the query ,key Sum ; Query segment and key segments Correlation measure between for: ， in, is the dot product operation between two matrices of the same size; Each query segment Relevance measure to all key segments will pass Function is normalized to obtain the aggregate weight : ， No. Output of segment positions All value segments The weighted sum of is calculated as follows: ， By splicing all outputs To obtain the output of the inter-segment attention module, the calculation process is: ， After the linear layer, the final output is , realizing multi-scale segment attention modeling; S22: Calculate the segmented temporal reconstruction loss function for: ， in, represents the total number of reconstructed segments, represents the true value of the reconstructed segment, represents the temporal cross-modeling results of the reconstructed segments; The goal of the segmented temporal reconstruction loss function is to minimize the overall loss of each segment; The temporal contrast model construction module builds a temporal contrast model based on the segmented attention calculation results and obtains a strong generalization embedding for temporal modeling through self-supervision methods; The spatial domain modeling enhancement module uses the embedding obtained through temporal modeling as the feature of each sensor node, and performs spatial domain modeling through a graph convolutional network to obtain enhanced results.

9. A computer storage medium having a computer program stored thereon, characterized in that: When the computer program is executed, the self-supervised spatiotemporal data completion method based on temporal semantic attention is implemented as described in any one of claims 1-7.

Images