Human mobility prediction and uncertainty estimation method based on sparse space-time trajectory

By adopting dual-graph convolution and global timestamp embedding in human mobility prediction, combining autocorrelation mechanisms and relative time self-attention sequence encoder, the problem of dealing with sparseness and periodicity in the prior art is solved, more accurate and interpretable mobility prediction is achieved, and the model's uncertainty estimation ability is enhanced through the trajectory entropy coordination loss function.

CN120105223APending Publication Date: 2025-06-06BEIJING UNIV OF POSTS & TELECOMM
View PDF 0 Cites 2 Cited by

Patent Information

Application Number
CN202411674509.4
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-11-21
Publication Date
2025-06-06

AI Technical Summary

Technical Problem

Existing human mobility prediction methods are difficult to effectively deal with the sparsity of location access, the multi-level periodicity of human movement, the difference in time intervals between continuous space-time points, and the regularity and randomness of human movement behavior.

Method used

The mobility prediction method based on sparse spatiotemporal trajectories is adopted to obtain space-time vectors through double-graph convolution embedding and global timestamp embedding. Combining the autocorrelation mechanism and relative time self-attention sequence encoder, a discrete and continuous time trajectory prediction model is constructed, and the trajectory entropy coordination loss function is defined to estimate the uncertainty of the model.

Benefits of technology

It effectively alleviates the prediction difficulty caused by location access sparseness and time sparseness, improves the accuracy and interpretability of human mobility prediction, and enhances the generalization ability of the model.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN120105223A_ABST
    Figure CN120105223A_ABST
Patent Text Reader

Abstract

The invention discloses a human mobility prediction and uncertainty estimation method and system based on a sparse spatio-temporal trajectory, and the method comprises the steps: obtaining a spatio-temporal trajectory, and carrying out the double-graph convolution embedding and global timestamp embedding of the spatio-temporal trajectory, and obtaining a spatio-temporal vector; for the discrete time trajectory, predicting a place of the discrete time trajectory at the next moment based on a sequence encoder of an autocorrelation mechanism, evaluating a place prediction effect based on an evaluation index, and optimizing a discrete time trajectory prediction model; for the continuous time track, the location and time of the next time-space point of the continuous time track are predicted based on a relative time self-attention sequence encoder, the location prediction effect is evaluated based on an evaluation index, the accuracy of time is measured based on a mean square error, and a continuous time track prediction model is optimized; and for an uncertain trajectory, defining the data uncertainty as trajectory entropy based on the spatio-temporal trajectory, determining a trajectory entropy coordination loss function based on the trajectory entropy, and estimating the uncertainty of the model.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence technology, and more specifically, to a method and system for predicting and estimating human mobility based on sparse spatiotemporal trajectories. Background Art

[0002] The direct goal of human mobility prediction is to accurately predict the user's future geographic location, usually focusing on the user's next location prediction. Human mobility prediction technology is crucial for many location-based application scenarios. In the field of urban planning, by deeply analyzing and predicting people's mobility patterns at different times and places, it can provide a scientific basis for urban traffic layout and public facilities configuration, effectively alleviate traffic congestion, and improve the convenience of living and traveling. In the field of mobile communications, human mobility prediction can assist mobile communication networks in actively adjusting according to real-time changes in network demand and optimizing network control and resource allocation mechanisms. In addition, in the commercial field, accurately predicting users' mobility trends plays an important role in providing location-sensitive services or product recommendations, as well as improving the accuracy of targeted advertising.

[0003] There are many problems with current human mobility prediction methods:

[0004] (1) Sparsity of location visits. The frequency of users visiting different locations varies greatly. Many locations are difficult to accurately predict because they have been visited only a few times or have never been visited. This sparsity is even higher than the sparsity of user-item ratings in traditional recommendation systems.

[0005] (2) Multi-layered periodicity of human mobility. Periodicity is an important factor governing human mobility. Human mobility exhibits complex multi-layered periodicity, involving daily activities, weekend leisure, annual holidays, and even personal periodic activities.

[0006] Their interactions make predictions even more difficult.

[0007] Existing mobility prediction models are mainly sequence models, which focus on capturing complex sequential mobility patterns, but often cannot effectively handle the above-mentioned location visit sparsity and complex periodicity.

[0008] (3) Differences in time intervals between consecutive spatiotemporal points. The time intervals between adjacent spatiotemporal points in a trajectory may be extremely short, such as only a few minutes; or they may be quite long, reaching several hours. This difference in time intervals means that it is difficult to fully understand the user's mobility pattern by relying solely on the user's sequential location transfer information. Existing sequence prediction models (such as RNN and Transformer) all assume that the contextual importance between adjacent time points is the same, making it difficult to accurately identify the user's mobility pattern.

[0009] (4) The regularity and randomness of human mobility behavior. Users may have some non-routine and temporary mobility behaviors, such as traveling. These behaviors are often sporadic and unlikely to occur repeatedly. Therefore, these fragments in the trajectory data may interfere with the learning of regular mobility patterns. Summary of the invention

[0010] According to the present invention, a method and system for predicting and estimating human mobility based on sparse spatiotemporal trajectories are provided to solve the technical problems existing in current human mobility prediction methods.

[0011] According to a first aspect of the present invention, a method for predicting and estimating human mobility based on sparse spatiotemporal trajectories is provided, comprising:

[0012] Obtain the user's open information dynamic data and preprocess it to obtain the spatiotemporal trajectory, and perform dual graph convolution embedding and global timestamp embedding on the spatiotemporal trajectory to obtain the spatiotemporal vector;

[0013] For discrete time trajectories, the sequence encoder based on the autocorrelation mechanism takes the space-time vector as input, determines the discrete time trajectory prediction model, predicts the location of the discrete time trajectory at the next moment, and evaluates the location prediction effect based on the evaluation index to optimize the discrete time trajectory prediction model;

[0014] For continuous time trajectories, based on the relative time self-attention sequence encoder, the spatiotemporal vector is used as input to determine the continuous time trajectory prediction model, predict the location and time of the next spatiotemporal point of the continuous time trajectory, and evaluate the location prediction effect based on the evaluation index. The accuracy of time is measured based on the mean square error, and the continuous time trajectory prediction model is optimized;

[0015] For uncertain trajectories, data uncertainty is defined as trajectory entropy based on the spatiotemporal trajectory, and based on the trajectory entropy, a trajectory entropy coordination loss function is determined to estimate the uncertainty of the model.

[0016] According to another aspect of the present invention, a human mobility prediction and uncertainty estimation system based on sparse spatiotemporal trajectories is provided, comprising:

[0017] The space-time vector module is used to obtain the user's open information dynamic data and perform preprocessing to obtain the space-time trajectory, and perform dual-graph convolution embedding and global timestamp embedding on the space-time trajectory to obtain the space-time vector;

[0018] The discrete trajectory mobility prediction module is used for discrete time trajectories. Based on the sequence encoder of the autocorrelation mechanism, the space-time vector is used as input to determine the discrete time trajectory prediction model, predict the location of the discrete time trajectory at the next moment, and evaluate the location prediction effect based on the evaluation index to optimize the discrete time trajectory prediction model.

[0019] The continuous trajectory mobility prediction module is used for continuous time trajectories. Based on the relative time self-attention sequence encoder, it takes the spatiotemporal vector as input, determines the continuous time trajectory prediction model, predicts the location and time of the next spatiotemporal point of the continuous time trajectory, evaluates the location prediction effect based on the evaluation index, measures the accuracy of time based on the mean square error, and optimizes the continuous time trajectory prediction model.

[0020] The uncertain mobility prediction module is used to define data uncertainty as trajectory entropy based on the spatiotemporal trajectory for uncertain trajectories, determine a trajectory entropy coordination loss function based on the trajectory entropy, and estimate the uncertainty of the model.

[0021] Therefore, for the mobility prediction of discrete time trajectories, the time intervals of the trajectories are the same. The present invention provides a dual-graph convolution autocorrelation network. The model embeds the location through dual-graph convolution to fuse the geographic spatial relationship of the location and the collective transfer flow, which alleviates the sparsity of the location access; uses timestamp layering for time embedding, and captures the multiple periodicities in the trajectory through the sequence encoder of the autocorrelation mechanism to improve the performance of the model. Secondly, for the prediction of human mobility in continuous time, the time intervals of the trajectory are different, and a Hawkes process model based on the relative time self-attention mechanism is provided. The model embeds the time difference between spatiotemporal points into the self-attention mechanism, and constructs the trajectory into a Hawkes process. By defining the conditional intensity function related to time to predict the probability of the next spatiotemporal point, the problem of unclear prediction meaning and low accuracy caused by time sparsity is alleviated. In addition, the present invention also estimates and calibrates the uncertainty of the mobility prediction task. By defining the trajectory entropy as the uncertainty of the data, and designing the trajectory entropy coordination loss function to reduce the impact of sample difficulty on the model; estimating the model uncertainty, the generalization ability of the model is enhanced. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] A more complete understanding of exemplary embodiments of the present invention may be obtained by referring to the following drawings:

[0023] Figure 1 A flowchart of a method for predicting and estimating human mobility based on sparse spatiotemporal trajectories according to this embodiment;

[0024] Figure 2 This is an architecture diagram of a method for predicting and estimating human mobility based on sparse spatiotemporal trajectories described in this embodiment;

[0025] Figure 3 This is a core flow diagram of a method for predicting and estimating uncertainty of human mobility based on sparse spatiotemporal trajectories described in this embodiment;

[0026] Figure 4Schematic diagram of the overall framework of DGCAN described in this embodiment;

[0027] Figure 5 Schematic diagram of the overall framework of RTA-HP described in this embodiment;

[0028] Figure 6 A schematic diagram of the uncertainty of the MC-Dropout algorithm estimating model described in this embodiment;

[0029] Figure 7 Schematic diagram of a human mobility prediction and uncertainty estimation system based on sparse spatiotemporal trajectories described in this embodiment. DETAILED DESCRIPTION

[0030] Now, exemplary embodiments of the present invention are described with reference to the accompanying drawings. However, the present invention can be implemented in many different forms and is not limited to the embodiments described herein. These embodiments are provided to disclose the present invention in detail and completely and to fully convey the scope of the present invention to those skilled in the art. The terms used in the exemplary embodiments shown in the accompanying drawings are not intended to limit the present invention. In the accompanying drawings, the same units / elements are marked with the same reference numerals.

[0031] Unless otherwise specified, the terms (including technical terms) used herein have the commonly understood meanings to those skilled in the art. In addition, it is understood that the terms defined in commonly used dictionaries should be understood to have the same meanings as those in the context of the relevant fields, and should not be understood as idealized or overly formal meanings.

[0032] According to a first aspect of the present invention, a flowchart 100 of a method for predicting and estimating human mobility based on sparse spatiotemporal trajectories is provided, referring to Figure 1 As shown, the method 100 includes:

[0033] S101: Obtain the user's open information dynamic data and perform preprocessing to obtain the spatiotemporal trajectory, and perform dual-graph convolution embedding and global timestamp embedding on the spatiotemporal trajectory to obtain the spatiotemporal vector;

[0034] S102: For discrete time trajectories, a sequence encoder based on an autocorrelation mechanism takes the space-time vector as input, determines a discrete time trajectory prediction model, predicts the location of the discrete time trajectory at the next moment, and evaluates the location prediction effect based on an evaluation index, thereby optimizing the discrete time trajectory prediction model;

[0035] S103: For the continuous time trajectory, based on the relative time self-attention sequence encoder, the spatiotemporal vector is used as input to determine the continuous time trajectory prediction model, predict the location and time of the next spatiotemporal point of the continuous time trajectory, and evaluate the location prediction effect based on the evaluation index, measure the accuracy of the time based on the mean square error, and optimize the continuous time trajectory prediction model;

[0036] S104: For the uncertain trajectory, define the data uncertainty as trajectory entropy based on the spatiotemporal trajectory, determine the trajectory entropy coordination loss function based on the trajectory entropy, and estimate the uncertainty of the model.

[0037] Specifically, refer to Figure 2 As shown in the architecture diagram, this implementation can be divided into three parts: discrete-time human mobility prediction, continuous-time human mobility prediction, and uncertainty.

[0038] The discrete-time human mobility prediction part is divided into three modules, namely the location embedding module, the time embedding module, and the trajectory prediction module. The location embedding module models the longitude and latitude coordinates that appear in the trajectory dataset, converts them into dense vectors, and mines out potential geographic features and correlations. The time embedding module implements multi-level periodic display encoding of timestamps to obtain a vector representation with the same dimension as the location embedding. The trajectory prediction module uses the spatiotemporal point vector sequence obtained by the location embedding module and the time embedding module as input to predict the location of the next moment of the discrete trajectory.

[0039] The continuous-time human mobility prediction part mainly includes trajectory embedding module, relative time self-attention module and Hawkes process inference module. The trajectory embedding module adopts the same location embedding module and time embedding module as the discrete-time human mobility prediction. The relative time self-attention module realizes the representation of relative time intervals in the trajectory sequence, and thereby affects the attention distribution of the context. The Hawkes process inference module obtains the most likely location of the next spatiotemporal point under uncertain time conditions, fully considering various time conditions, and improving the interpretability and accuracy of the prediction.

[0040] Uncertainty estimation is divided into two categories: data uncertainty estimation and model uncertainty estimation. By calculating the data uncertainty in the trajectory as an evaluation indicator of data quality, the uncertainty of the trajectory is included in the loss function as a quality weight to control the impact of low-quality data on the update of model parameters, and to a certain extent improve the quality of the data set. By estimating the model uncertainty, the model's understanding of the data is judged, and then the loss weights of each spatiotemporal point in the trajectory are adjusted, so that the model can make full use of reliable information as much as possible. Uncertainty estimation does not depend on a specific model and can be directly used in the training of other models, while improving performance without incurring additional computational overhead.

[0041] refer to Figure 3 The detailed description of each step is as follows:

[0042] 1) Embed the spatiotemporal trajectory to obtain the spatiotemporal vector

[0043] Two graphs, spatial graph and mobility flow graph, are introduced to model the relationship between locations, representing the geographical distance relationship and collective mobility pattern of locations respectively. Finally, the vectors learned from these two relationships are fused as the location embedding representation.

[0044] The period information of the timestamp is obtained from different period features to obtain a vector representation with the same dimension as the location embedding.

[0045] 2) For discrete-time trajectories, the autocorrelation sequence encoder is used to obtain location prediction results.

[0046] A sequence encoder based on autocorrelation mechanism is used to capture the cycles in the sequence and focus on similar subsequences in different cycles in the sequence. The spatiotemporal point vector sequence obtained by the location embedding module and the time embedding module is used as input to predict the location of the next moment of the discrete trajectory.

[0047] 3) For continuous-time trajectories, a relative temporal self-attention sequence encoder is used to model the trajectory sequence.

[0048] 4) Construct the Hawkes process and define a time-dependent conditional intensity function to represent the probability of the next spacetime point occurring at this time.

[0049] 5) Uncertainty estimation

[0050] Uncertainty estimation and calibration are performed from two aspects: data uncertainty and model uncertainty. Data uncertainty is defined as trajectory entropy, and a trajectory entropy coordination loss function is designed based on its distribution law, aiming to reduce the impact of trajectory samples with too large or too small entropy on the model. The MC-Dropout method is used to estimate the uncertainty of the model, and the expected prediction result of the model is approximated as the mean of multiple outputs of MC-Dropout. The uncertainty of the model is approximated as the sampled prediction entropy of multiple outputs of MC-Dropout, making the prediction effect of the model robust and reliable.

[0051] Optionally, the user's open information dynamic data is obtained and preprocessed to obtain the spatiotemporal trajectory, and the spatiotemporal trajectory is embedded by dual graph convolution and global timestamp to obtain the spatiotemporal vector, including:

[0052] Acquire the user's open information dynamic data, pre-process the open information dynamic data, and obtain a spatiotemporal trajectory, including an initial location vector and a timestamp;

[0053] Determine the spatial graph and the mobile flow graph, perform dual-graph convolution location feature embedding on the initial location vector, and obtain the location embedding vector:

[0054] The spatial graph G s =(L s , E s , A s ), L represents the set of locations, E s represents the set of edges, A s ∈R |L|×|L| represents the adjacency matrix, A s (i, j) represents l i , l j The spatial proximity relationship between two locations is defined as:

[0055]

[0056] where dist(l i , l j ) is the Euclidean distance between two locations, Δ is the distance threshold, indicating l i , l j The spatial relationship between them decreases as the distance between the two locations increases. When the distance exceeds Δ, there is no edge connection;

[0057] Mobile flow graph G f =(L f , E f , A f ), L represents the set of locations, E f represents the set of edges, A f ∈R |L|×|L| represents the adjacency matrix, A f (i, j) represents l i , l j The spatial proximity relationship between two locations is defined as:

[0058] A f (i, j) = count(l i , l j )

[0059] Where count(l i , l j ) means that in all user trajectories, i , l j The number of times two locations are visited consecutively, if (l i , l j ) subsequence appears in a trajectory of a certain user, then l i , l j There are edges between them, and the mobility flow graph simulates the movement pattern of the crowd;

[0060] Performing global timestamp feature embedding on the timestamp to obtain a time embedding vector;

[0061] The graph neural network GCN based on the spatial graph and the mobile flow graph learns in parallel. After the node is updated, the location vectors es and ef are output respectively. The aggregation layer is applied to generate the final location embedding vector:

[0062] e l =Agg(e s , e f )=Φe s +(1-Φ)e f

[0063] where Φ∈R C It is a trainable parameter vector used to control the contribution of the topological information of the two graphs to different final representations. By fusing the two relationship graphs, the inadequacy of the geographical distance in expressing the complex relationship between places can be compensated by the mobility tendency of all users. At the same time, the information gap caused by the sparsity of place visits can be filled by the relationship between places with closer distances, so as to more fully represent the places.

[0064] Aggregate the time embedding vector and the location embedding vector to obtain the space-time vector:

[0065]

[0066] The information selected from the timestamp includes the calendar timestamp minute, hour, day, week, month, and year, which represent the relative offset of the timestamp in this periodic layer, and the special timestamp holidays and events, which are used to distinguish the different mobility patterns of users on weekdays and holidays. Each global timestamp scalar is normalized and mapped to a vector representation with the same dimension as the location embedding through a learnable timestamp embedding layer SE(*). The time embedding vector and the location embedding vector are combined to obtain a spatiotemporal point vector.

[0067] Optionally, for discrete time trajectories, a sequence encoder based on an autocorrelation mechanism takes the space-time vector as input, determines a discrete time trajectory prediction model, and predicts the location of the discrete time trajectory at the next moment, including:

[0068] Given the current trajectory and historical trajectory of user u, the long trajectory sequence is concatenated in time, and the input vector of the first layer encoder is obtained after location embedding and time embedding:

[0069] X (0) =e p1 e p2 ...e pm

[0070] Where m is the length of the trajectory;

[0071] For the lth layer of the encoder, the input sequence vector X (l-1) After mapping the trainable parameter matrix, three vector sequence query vectors can be obtained respectively

[0072] Q=X (l-1) W Q , K = X (l-1) W K , V = X (l-1) W V

[0073] Apply the autocorrelation mechanism to get each head output:

[0074] Head i =AutoCoelation(Q i , K i , V i )

[0075]

[0076] in Roll(X,τ i ) represents a rolling operation with a time delay τ on the sequence X, during which the element at the first position of the sequence will be moved to the last position, aligning and integrating similar subsequences at the same phase position of the estimation cycle, R Q,K (τ) represents the cross-correlation between sequences Q and K, i.e., the autocorrelation of time series X, L is the length of the time series, and k represents the k time delays τ with the largest autocorrelation function. 1 ,…,τ k As the k-layer cycle dependency of the trajectory;

[0077] The final output of the multi-head autocorrelation module is:

[0078]

[0079] In each encoder layer, a feedforward neural network (FC) is connected after the autocorrelation module, which is calculated as:

[0080]

[0081] Where W 1 , W 2 is a trainable weight matrix, b 1 、b 2 is the deviation, and finally, the output of the l-th encoder layer is:

[0082]

[0083] In order to predict the user's next action, a multi-layer perceptron MLP is used to output the final probability distribution of the location at the next moment:

[0084]

[0085] Where W out is a trainable weight matrix, b out It is a deviation;

[0086] The cross entropy loss is used as the loss function for mini-batch training, expressed as:

[0087]

[0088] Among them, the model outputs the corresponding next moment location for each space-time point in the sequence, and uses a mask method to cover the information after the current space-time point.

[0089] Specifically, the application in discrete time scenarios is also called Dual Graph Convolution Auto-Correlation Network (DGCAN). The overall framework of DGCAN is as follows: Figure 4 shown.

[0090] Get the data and preprocess it:

[0091] This embodiment uses OIDD (Open Information Dynamic Data) of telecommunications users from Shanghai from January 14 to January 20, 2018. Each OIDD record contains information such as mobile phone number timestamp, longitude, and latitude. The mobile phone number can be used as the unique identifier of the user, and the longitude and latitude indicate the base station to which the user was connected when this record was generated, thereby approximating the actual location of the user.

[0092] The dataset was screened and only user data with no less than 50 records were retained. In order to enable the data to be applied to discrete-time human mobility prediction tasks, the trajectories were discretized and a day was divided into 48 equal-length time slots, each of which was 30 minutes. For time slots containing multiple records, the coordinate average method was used for aggregation, and the base station closest to the average coordinate point was taken as the user position of the time slot; in time slots where no records were generated, in order to maintain data continuity, it was assumed that the user remained at the last known position.

[0093] Double-image convolution location embedding obtains the location embedding vector:

[0094] At the same time, two spatial graphs and mobile flow graphs with the same nodes and different edge structures are used to represent the straight-line distance relationship of locations and the global view of all user movements, respectively, to deeply represent locations. Two GCNs are used to learn two different relationships between locations, and finally an aggregation layer is designed to fuse the node vectors output by the two GCNs into the final location dense vector.

[0095] The spatial graph G s =(L s , E s , A s ). L represents the set of locations, E s represents the set of edges, A s ∈R |L|×|L| represents the adjacency matrix, A s (i, j) represents l i , l j The spatial proximity relationship between two locations is defined as:

[0096]

[0097] where dist(l i , l j ) is the Euclidean distance between two locations, Δ is the distance threshold, indicating l i , l j The spatial relationship between the two locations decreases as the distance between them increases. When the distance exceeds Δ, there is no edge connection.

[0098] Mobile flow graph G f =(L f , E f , A f ). L represents the set of locations, E f represents the set of edges, A f ∈R |L|×|L| represents the adjacency matrix, A f (i, j) represents l i , l j The spatial proximity relationship between two locations is defined as:

[0099] A f (i, j) = count(l i , l j )

[0100] Where count(l i , l j ) means that in all user trajectories, i , l j The number of times two locations are visited consecutively, if (l i , l j) subsequence appears in a trajectory of a certain user, then l i , l j There are edges between them, and the mobility flow graph simulates the movement pattern of the crowd.

[0101] Two GCNs based on the spatial graph and the mobile flow graph learn in parallel. After updating the nodes, the location vectors es and ef are output respectively. Next, the aggregation layer is applied to generate the final location embedding vector:

[0102] e l =Agg(e s , e f )=Φe s +(1-Φ)e f

[0103] where Φ∈R C is a trainable parameter vector used to control the contribution of the topological information of the two graphs to different final representations. By fusing these two relationship graphs, the inadequacy of the geographical distance in expressing the complex relationship between places can be compensated by the mobility tendency of all users. At the same time, the information gap caused by the sparsity of place visits can be filled by the relationship between places with closer distances, so as to more fully represent the places.

[0104] The global timestamp is embedded into the space-time point vector:

[0105] Global Time Stamp Embeddings (GTSE) implements multi-level periodic display encoding of timestamps. The information that can be selected from the timestamp includes calendar timestamp minutes, hours, days, weeks, months, years, etc., which represent the relative offset of the timestamp in this periodic layer, and special timestamps such as holidays and events, which are used to distinguish different mobility patterns of users on weekdays and holidays. Each global timestamp scalar is normalized and mapped to a vector representation with the same dimension as the location embedding through a learnable timestamp embedding layer SE(*). The time embedding vector and the location embedding vector are combined to obtain a spatiotemporal point vector:

[0106]

[0107] The trajectory prediction module predicts the location at the next moment

[0108] The Transformer sequence encoder structure is used to model the sequence, followed by a multi-layer perceptron (MLP) to complete the classification prediction of the location.

[0109] The sequence encoder is implemented by stacking N standard Transformer encoding layers. Each layer is mainly composed of a multi-head autocorrelation module and a feedforward neural network. In this structure, the multi-head autocorrelation module adopts a standard multi-head self-attention structure, replacing the point-to-point self-attention mechanism with a hierarchical sequence connection autocorrelation mechanism. In addition, both modules apply residual connection and hierarchical normalization technology to promote the stability and efficiency of the model training process.

[0110] Given the current trajectory and historical trajectory of user u, the long trajectory sequence is concatenated in time, and the input vector of the first layer encoder is obtained through the location embedding module and the time embedding module:

[0111] X (0) =e p1 e p2 ...e pm

[0112] Where m is the length of the trajectory.

[0113] For the lth layer of the encoder, the input sequence vector X (l-1) After mapping the trainable parameter matrix, three vector sequence query vectors can be obtained respectively

[0114] Q=X (l-1) W Q , K = X (l-1) W K , V = X (l-1) W V

[0115] Applying the autocorrelation mechanism yields the following output for each head:

[0116] Head i =AutoCoelation(Q i , K i , V i )

[0117]

[0118] in Roll(X,τ i ) represents a rolling operation with a time delay τ on the sequence X, during which the element at the first position of the sequence will be moved to the last position, so that similar subsequences at the same phase position of the estimation period can be aligned and integrated. Q,K (τ) represents the cross-correlation between the sequences Q and K (i.e., the autocorrelation of the time series X), L is the length of the time series, and k represents the k time delays τ with the largest autocorrelation function. 1 ,…,τ kk-layer cyclic dependencies as trajectories.

[0119] The final output of the multi-head autocorrelation module is:

[0120]

[0121] In each encoder layer, a feedforward neural network (FC) is connected after the autocorrelation module, which is calculated as:

[0122]

[0123] Where W 1 , W 2 is a trainable weight matrix, b 1 、b 2 is the deviation. Finally, the output of the l-th encoder layer is:

[0124]

[0125] In order to predict the user's next action, a multi-layer perceptron (MLP) is finally used to output the final probability distribution of the location at the next moment:

[0126]

[0127] Where W out is a trainable weight matrix, b out It's a deviation.

[0128] The cross entropy loss, which is the most commonly used loss function for classification problems, is used as the loss function for mini-batch training, expressed as:

[0129]

[0130] In order to make full use of trajectory information, the model outputs the next location of each spatiotemporal point in the sequence. In order to prevent learning future information, a mask is used to cover the information after the current spatiotemporal point.

[0131] Optionally, the location prediction effect is evaluated based on the evaluation index, and the discrete time trajectory prediction model is optimized, including:

[0132] Set the model hidden layer dimension to 128. During the training process, use the gradient descent method, set the training batch size to 32, and optimize the model parameters based on the Adam optimizer. The evaluation indicators include accuracy and average reciprocal ranking.

[0133] Accuracy, Acc: measures the degree of match between the predicted location and the actual next access location. For a given test data set, it is defined as the ratio of the number of samples with the correct next access location to the total number of samples. Acc@k indicates whether the true location is in the top k most likely locations predicted by the model, and is calculated as:

[0134]

[0135] Among them, N represents the number of test samples, a is a discriminant function, which returns 1 if the condition is true, otherwise it returns 0, and rank i is the i-th test sample The ranking of the real position in;

[0136] Mean reciprocal rank (MRR): It calculates the average reciprocal rank of the first relevant entry retrieved in the prediction vector, calculated as:

[0137]

[0138] Specifically, the model hidden layer dimension is set to 128, and the dropout technology is used to reduce overfitting, where the dropout ratio is set to 0.2. During the training process, the gradient descent method is used, the training batch size is set to 32, and the Adam optimizer is used to optimize the model parameters.

[0139] Evaluation Metrics: Accuracy (Acc): It measures the degree of match between the predicted location and the actual next access location. For a given test dataset, it is defined as the ratio of the number of samples with the correct next access location to the total number of samples. Acc@k indicates whether the true location is in the top k most likely locations predicted by the model, and is calculated as:

[0140]

[0141] Where N is the number of test samples, a is a discriminant function that returns 1 if the condition is true, otherwise it returns 0, and rank i is the i-th test sample The ranking of the real position in.

[0142] Mean reciprocal rank (MRR): It calculates the mean reciprocal rank of the first relevant entry retrieved in the prediction vector, calculated as:

[0143]

[0144] The DGCAN model is compared with several state-of-the-art mobility prediction methods:

[0145] Markov: A classic sequence modeling machine learning model that assumes that individual location visits have Markov properties.

[0146] LSTM: A deep learning model for classic sequence modeling that retains the hidden state to perform sequential operations on the input, thereby effectively capturing sequential relationships and being able to learn long-term dependencies by introducing a gating mechanism.

[0147] DeepMove: A RNN-based model that introduces an attention mechanism to calculate the similarity between the current trajectory and the historical trajectory, uses RNN to process the historical and current trajectories to learn user preferences, and captures the spatiotemporal correlation of the trajectory.

[0148] Mobcast: A Transformer encoder-based model that exploits temporal, semantic, social, and geographic context in historical POI sequences. In addition, an auxiliary task is introduced to encourage prediction of geographically nearby POIs.

[0149] GCDANDang: A model based on the Transformer encoder-decoder framework that uses graph convolution to learn spatiotemporal representations and designs a dual attention mechanism to effectively extract long-range sequence correlations within a trajectory and correlations between different trajectories for prediction.

[0150] GETNext: A Transformer encoder-based model that uses the user global trajectory flow graph to generate POI embeddings through GCN, fuses time-aware category context embeddings, and uses transition probability graphs based on self-attention to optimize the next moment position prediction.

[0151] The test results are shown in Table 1.

[0152] Table 1 Experimental results of Example 1

[0153]

[0154] By learning two relationship graphs between locations, the geospatial relationships and collective flow characteristics between locations are fully utilized to reduce the impact of location sparsity.

[0155] By capturing the periodic meaning in the timestamp and explicitly extracting multiple period lengths in the trajectory sequence to model the periodicity of the sequence, the user's movement pattern is captured and the prediction accuracy is improved.

[0156] Optionally, for a continuous time trajectory, based on a relative time self-attention sequence encoder, the spatiotemporal vector is used as input to determine a continuous time trajectory prediction model, and the location and time of the next spatiotemporal point of the continuous time trajectory are predicted, including:

[0157] Extract the open information dynamic data of continuous time trajectory for preprocessing, filter repeated locations, and obtain the continuous time and space trajectory;

[0158] Perform dual graph convolution embedding and global timestamp embedding on the spatiotemporal trajectory to obtain the spatiotemporal vector;

[0159] Based on the relative time self-attention mechanism, the trajectory sequence is modeled and the continuous time trajectory prediction model is determined:

[0160] Construct a relative time difference matrix to represent the time difference in the trajectory sequence. Each time difference corresponds to a learnable vector representation and affects the attention score of two spatiotemporal points with corresponding time differences in the sequence. The trajectory is divided into historical trajectory and recent trajectory. The former only considers periodicity, while the latter also considers relative time decay.

[0161] In order to express the relative time difference between space-time points, the time difference is used to represent the distance relationship between different positions. First, a relative time vector is designed. Represents the corresponding vectors in each relative time case, and globally shares the data in the model, where k is the maximum relative time in the sequence. In addition, a time difference matrix δt is defined, with elements δ t (i, j)∈[0,k] represents the time difference from the i-th space-time point to the j-th space-time point:

[0162]

[0163] In the self-attention mechanism, the content information and position information in the sequence are considered at the same time. For the i-th point in the sequence, two vectors {H i} and {P i|j} to represent the content and the position information relative to the jth point, then the calculation of the cross attention score between i and j is:

[0164]

[0165] In the self-attention mechanism, the input content sequence vector H is a fusion vector including location embedding and cycle time embedding. After mapping the trainable parameter matrix, three vectors content query vector Q can be obtained respectively. c , content key vector K c , content value vector V c :

[0166] Q c =HWq,c |, K c =HW k,c , V c =HW v,c

[0167] In addition, the relative time query vector Q is additionally calculated r and the relative time key vector K r :

[0168] Q r =PW q,r , K r =PW k,r

[0169] The self-attention matrix based on relative time is Represents the attention score from the i-th point to the j-th point:

[0170]

[0171] Respectively represent content-content, content-time, and time-content information, where The matrix Q c The i-th row vector of is the matrix K c The j-th row vector of is the δ(i, j)th row vector of the matrix Kr, The matrix Q r The δ(j, i)th row vector of ;

[0172] Query vector Q by relative time r And the relative time key vector Kr, the time difference in the sequence is mapped into the corresponding vector, which is integrated into the self-attention matrix. For the space-time points that differ from the current space-time point by more than k, they will all be mapped to P t In the last vector of , that is, for these spacetime points, The last two terms of the attention score are the same, which has nothing to do with the time difference and is only related to the representation of the spatiotemporal point, shielding the influence of time decay and playing the role of masking;

[0173] The output of self-attention is:

[0174]

[0175] in Divide by a scaling factor to stabilize the model training process;

[0176] Relative temporal encoding can not only capture the time difference in short-term dependencies, but also learn the periodicity of long-term dependencies.

[0177] Optionally, for a continuous time trajectory, based on a relative time self-attention sequence encoder, taking the spatiotemporal vector as input, determining a continuous time trajectory prediction model, predicting the location and time of the next spatiotemporal point of the continuous time trajectory, and further comprising:

[0178] set up is the conditional strength function of the model, where is the historical trajectory up to time t. The definition of the intensity function is different for different location categories, that is, for each l∈{1, 2, ..., L}, is defined as the conditional intensity function of location l, and the conditional intensity function of the entire trajectory is defined as:

[0179]

[0180] The conditional intensity function of each location category is expressed as:

[0181]

[0182] Time is defined in the interval t∈[t i ,t],f l (x) = β l log(1+exp(x / β l )) is the parameter β i Softplus function, in the conditional intensity function, the current part affects the timestamps t of the two space-time points i and t i+1 The interpolation between l Used to adjust the importance of interpolation. The history part is cut off to t i The hidden state h(t i ); the base part indicates the probability of an event occurring without considering historical information;

[0183] Using a neural network to represent the Hawkes process, define N(t) = |h:t h ≤t| is the count of spacetime points before time t, given the past history (t i , the number of events in t) is expressed as:

[0184] ΔN(t i , t) = N(t) - N(t i )

[0185] Let T i >t i Let L be the random variable of the next time. i+1 is the random variable of the next location, T i The cumulative distribution function (in As a condition) is:

[0186]

[0187] set up but It is expressed as:

[0188]

[0189] The corresponding probability density function is:

[0190]

[0191] Given the past history and the time t of the next spacetime point i+1 , Location i+1 The distribution of is:

[0192]

[0193] Predict the time of the next spacetime point:

[0194]

[0195] Given the next time t i+1 , the corresponding next location is:

[0196]

[0197] In the unknown i In this case, it is necessary to consider t i The next location is:

[0198]

[0199] where the same value of t is used for pairwise comparisons for each k in argmax, sharing λ(t) and p across all k i (t) calculation;

[0200] For the RTA-HP model, given its conditional strength function For any sequence S in the observation interval [0,T], the likelihood function is:

[0201]

[0202] Correspondingly, the log-likelihood is defined as:

[0203]

[0204] The model parameters are learned by maximizing the log-likelihood of all sequences. Specifically, suppose there are N trajectories S in the dataset1 , S 2 , ..., S N , in order to find the solution parameters, the goal of model training is:

[0205]

[0206] Each Added to explain why at time t i User arrival location i ,and Reduced to explain why no location l∈{l 1 , ..., l |L|}.

[0207] Specifically, refer to Figure 5 As shown in Figure 2, the application in continuous time scenarios is also called the Relative Time Attention Hawkes Process (RTA-HP). The overall framework of RTA-HP is as follows: Figure 5 Specifically including:

[0208] Get data and preprocess it

[0209] Four location-based service datasets are used for experimental analysis.

[0210] Foursquare is a location-based social networking service that allows users to check in at specific locations and share location information with friends via mobile devices. Foursquare provides two data subsets, which are user check-in records collected in Tokyo and New York from April 12, 2012 to February 16, 2013, labeled FourSquare-TKY and FourSquare-NYC respectively. Each check-in record records the user information, timestamp, longitude and latitude, and location category in detail.

[0211] Gowalla is also a location-based social networking service, and its dataset covers users' check-in activities in the United States from March 2008 to October 2010, with a total of 123,129 check-in locations. Each record also includes user information, timestamp, latitude and longitude, and the unique identifier of the location.

[0212] Continuous time trajectories were extracted from the OIDD dataset for experiments. In order to eliminate the impact of users staying in the same place for a long time on the prediction accuracy, repeated places were first filtered out, and the static period was defined as the user's continuous stay in the same place. Only the first record in this period was retained to remove redundant data.

[0213] Dual graph convolution embedding and global timestamp embedding are used to obtain spatiotemporal point vectors.

[0214] The relative time attention mechanism (Relative TimeAttention, RTA) realizes the modeling of trajectory sequences.

[0215] Improve the self-attention mechanism in Transformer. Construct a relative time difference matrix to represent the time difference in the trajectory sequence. Each time difference corresponds to a learnable vector representation and affects the attention score of two spatiotemporal points with corresponding time differences in the sequence. In addition, due to the limited representation of relative time differences, the trajectory is naturally divided into historical trajectory and recent trajectory. The former only considers periodicity, while the latter also considers relative time decay.

[0216] In order to express the relative time difference between space-time points, the time difference is used to represent the distance relationship between different positions. First, a relative time vector is designed represents the corresponding vector in each relative time case and is globally shared for the data in the model, where k is the maximum relative time in the sequence. In addition, a time difference matrix δt is defined, with elements δ t (i, j)∈[0,k] represents the time difference from the i-th space-time point to the j-th space-time point:

[0217]

[0218] In the self-attention mechanism, the content information and position information in the sequence are considered at the same time. For the i-th point in the sequence, two vectors {H i} and {P i|j} to represent the content and the position information relative to the jth point. Then, the calculation of the cross attention score for i and j is:

[0219]

[0220] In the self-attention mechanism, the input content sequence vector H is a fusion vector including location embedding and cycle time embedding. After mapping the trainable parameter matrix, three vectors content query vector Q can be obtained respectively. c , content key vector K c , content value vector V c :

[0221] Q c =HW q,c |, K c =HW k,c , V c =HW v,c

[0222] In addition, compared with the traditional self-attention structure, the relative time query vector Q is additionally calculated r and the relative time key vector K r :

[0223] Q r =PW q,r , K r =PW k,r

[0224] The self-attention matrix based on relative time is Represents the attention score from the i-th point to the j-th point:

[0225]

[0226] Respectively represent content-content, content-time, and time-content information, where The matrix Q c The i-th row vector of is the matrix K c The j-th row vector of is the δ(i, j)th row vector of the matrix Kr, The matrix Q r The δ(j, i)th row vector of .

[0227] Query vector Q by relative time r And the relative time key vector Kr, the time difference in the sequence is mapped into the corresponding vector and integrated into the self-attention matrix. For the time and space points that differ from the current time and space point by more than k, they will all be mapped to P t In the last vector of , that is, for these spacetime points, The last two terms of the attention score are the same, which has nothing to do with the time difference and is only related to the representation of the spatiotemporal point. It shields the influence of time attenuation and plays the role of a mask. The meaning is consistent with the traditional self-attention mechanism.

[0228] Finally, the output of self-attention is:

[0229]

[0230] in Divide by a scaling factor to stabilize the model training process.

[0231] Relative temporal encoding can not only capture the time difference in short-term dependencies, but also learn the periodicity of long-term dependencies.

[0232] Hawkes process inference predicts the location and time of the next space-time point

[0233] The Hawkes process is a time-point model in which the conditional intensity function at any time t depends linearly on the influence of events that happened before t. Relative temporal self-attention only generates hidden representations of discrete timestamps, and the associated intensity is also discrete, requiring an interpolated continuous-time intensity function.

[0234] set up is the conditional strength function of the model, where is the historical trajectory up to time t. The definition of the intensity function is different for different location categories, that is, for each l∈{1, 2, ..., L}, is defined as the conditional intensity function of location l, and the conditional intensity function of the entire trajectory is defined as:

[0235]

[0236] The conditional intensity function of each location category is expressed as:

[0237]

[0238] Time is defined in the interval t∈[t i ,t],f l (x) = β l log(1+exp(x / β l )) is the parameter β l In the conditional intensity function, the current part affects the timestamps t of the two space-time points. i and t i+1 The interpolation between l Used to adjust the importance of interpolation. The history part is the end point of t i The hidden state h(t i ); the base part indicates the probability of an event occurring without considering historical information.

[0239] The Hawkes process is represented by a neural network. Define N(t) = |h:t h ≤t| is the count of spacetime points before time t. Therefore, given the past history (t i , the number of events in t) is expressed as:

[0240] ΔN(t i , t) = N(t) - N(t i )

[0241] Let T i >t i Let L be the random variable of the next time. i+1 is the random variable of the next location, Ti The cumulative distribution function (in As a condition) is:

[0242]

[0243] set up but It can be expressed as:

[0244]

[0245] The corresponding probability density function is:

[0246]

[0247] Given the past history and the time t of the next spacetime point i+1 , Location i+1 The distribution of is:

[0248]

[0249] Predict the time of the next spacetime point:

[0250]

[0251] Given the next time t i+1 , the next location is most likely to be:

[0252]

[0253] In the unknown i In this case, it is necessary to consider t i The most likely next location is:

[0254]

[0255] where the same value of t is used for each k in argmax for pairwise comparisons, so that λ(t) and p can be shared across all k. i (t) Calculation.

[0256] Model Training

[0257] For the RTA-HP model, given its conditional strength function For any sequence S in the observation interval [0,T], the likelihood function is:

[0258]

[0259] Correspondingly, the log-likelihood is defined as:

[0260]

[0261] The model parameters are learned by maximizing the log-likelihood of all sequences. Specifically, assuming that the dataset

[0262] There are N trajectories S in 1 , S 2 , ..., S N , then in order to find the solution parameters, the goal of model training is:

[0263]

[0264] Intuitively, each Added to explain why at time t i User arrival location i ,and Reduced to explain why no location l∈{l 1 , ..., l |L|}.

[0265] Optionally, the location prediction effect is evaluated based on the evaluation index, the time accuracy is measured based on the mean square error, and the continuous time trajectory prediction model is optimized, including:

[0266] Evaluate the location prediction effect based on the evaluation index;

[0267] The root mean square error (RMSE) is quantified in minutes. The root mean square error is used to calculate the average of the squares of the difference between the time predicted by the model and the actual time. This is used to measure the accuracy of time prediction, eliminate the influence of the time interval scale, and normalize the prediction error:

[0268]

[0269] in Represents the predicted timestamp, t i Represents the real timestamp, and the corresponding mean square error of time is:

[0270]

[0271] Specifically, the model hidden layer dimension is set to 128, and the dropout technology is used to reduce overfitting, where the dropout ratio is set to 0.2. During the training process, the gradient descent method is used, the training batch size is set to 32, and the Adam optimizer is used to optimize the model parameters.

[0272] The effect of location prediction is still evaluated by two indicators: accuracy and mean reciprocal rank (MRR).

[0273] The effect of time prediction is quantified by the root mean square error (RMSE) in minutes. The root mean square error is used to calculate the average of the squares of the difference between the time predicted by the model and the actual time, so as to measure the accuracy of time prediction. In order to eliminate the influence of the time interval scale, the prediction error is normalized:

[0274]

[0275] in Represents the predicted timestamp, t i Represents the real timestamp. The corresponding mean square error of time is:

[0276]

[0277] RTA-HP is compared with several advanced mobility prediction methods including Markov, LSTM, DeepMove and GCDAN:

[0278] ST-RNN: An RNN-based model that models the local spatiotemporal context at each layer using time-specific transition matrices for different time intervals and distance-specific transition matrices for different geographic distances.

[0279] LSTPM: A LSTM-based model that develops a context-aware non-local network structure to model users' historical patterns; a geographically expanded RNN is proposed to model users' short-term behavioral intentions.

[0280] The location prediction results are shown in Table 2.

[0281] Table 2-2 Location prediction effect of Example 2

[0282]

[0283]

[0284] The time prediction effect is shown in Table 3.

[0285] Table 3 Time prediction effect

[0286]

[0287] Therefore, the relative temporal self-attention mechanism can flexibly capture and distinguish the historical periodicity and short-term mobility dependencies in the trajectory, thereby improving the accuracy of the prediction.

[0288] By constructing the trajectory into a Hawkes process, the most likely location of the next space-time point under uncertain time conditions is obtained, fully considering various time conditions and improving the interpretability and accuracy of the prediction.

[0289] Optionally, for an uncertain trajectory, defining data uncertainty as trajectory entropy based on the spatiotemporal trajectory, determining a trajectory entropy coordination loss function based on the trajectory entropy, and estimating the uncertainty of the model include:

[0290] Filter duplicate locations in the open information dynamic data of spatiotemporal trajectories, exclude records of abnormally fast movement speed or frequent back-and-forth movement, remove unpopular locations with fewer than a predetermined number of visits and inactive users with fewer than a certain number of activity records, and obtain uncertain spatiotemporal trajectories;

[0291] The data uncertainty is defined as trajectory entropy based on the spatiotemporal trajectory of the uncertainty:

[0292] The user's trajectory sequence is regarded as a random process, and the uncertainty of the trajectory sequence is calculated based on the entropy of the random process. Given a trajectory sequence S u =p 1 p 2 …p n , a space-time point p is a tuple of a timestamp t and a location identifier l, i.e., p = (t, l), and its position sequence is defined as: L u = l 1 l 2 …l n ;

[0293] Trajectory entropy is defined to represent the uncertainty of the location a user will visit at the next moment. It depends not only on the frequency of the user's visits to each location, but also on the order of the visited locations and the length of time the user stays at each location, thereby fully capturing the spatiotemporal structure of the individual flow pattern, and is calculated as

[0294]

[0295] Among them, P(L′ u ) is on the trajectory S u Find a specific time ordered subsequence L′ u The probability of using the Lempel-Ziv estimation method to quickly converge to the actual entropy value, the simplified trajectory entropy calculation formula is:

[0296]

[0297] Among them, Λi represents the length of the shortest subsequence starting from position i that appears for the first time in the entire sequence of positions up to position i-1;

[0298] Taking the probability density distribution of trajectory entropy as the weight, define the trajectory entropy density function:

[0299]

[0300] where e k is the trajectory entropy probability density of the kth sample, δ ε (e k , e) represents the number of samples whose trajectory entropy distribution is in the range of (e-ε / 2, e+ε / 2). If it is in the range, δ ε (e k , e) = 1, otherwise, δ ε (e k , e)=0,l ε (e) represents the length of the interval (e-ε / 2, e+ε / 2). According to the kernel density estimation method, the trajectory entropy density corresponding to each trajectory sample is calculated and the cross entropy loss L is used. CE As the loss function, the TEH loss function is defined as:

[0301]

[0302] where p k and Represent the true value and predicted value of the kth sample respectively;

[0303] In order to avoid the problem that the weighted differentiation of trajectory entropy density is not large due to the similar trajectory entropy of samples in the same batch, we further calculate the ED (e k ) after the normalization function f w (*) is a standardized transformation, defined as:

[0304]

[0305] Where B is the batch size set during model training, U b is the set of trajectories in this batch in the training set. In this way, each time the parameters are updated, there must be some sample instance weights f W (ED(e k )) will be greater than 1, and the weight f of some sample instances W (ED(e k )) will be less than 1, achieving the purpose of weight allocation. Through the weight allocation strategy, the parameters of the final model are updated:

[0306]

[0307] Where lr is the learning rate, represents the gradient operator, W t and U t They respectively represent the model trainable parameters and the corresponding trajectory set in the tth training batch.

[0308] Optionally, a trajectory entropy coordination loss function is determined based on the trajectory entropy to estimate the uncertainty of the model, including:

[0309] Let q(w) be the distribution of a matrix with columns randomly set to zero according to a Bernoulli distribution:

[0310]

[0311] where z i,j ~Benoulli(pi), where i = 1, ..., L, and j = 1, ..., K i-1 The dimension of the W matrix here is K i ×K i-1 , p i Indicates the probability of a node being blocked during the Dropout process, M i Represents the parameters of the variation matrix.

[0312] When the Dropout layer is turned on during the test, the neural network is randomly forward-inferred T times, which is equivalent to sampling T times from the Bernoulli distribution to obtain a set of network parameters. Based on this, the model for a given input x * The expected value of the prediction is:

[0313]

[0314] where y * Represents the input x under the current neural network parameters * The prediction results;

[0315] The output is presented as a probability distribution after multiple random Dropout operations. These probability distributions can be regarded as samples of the distribution inferred from the Bayesian neural network. For the classification category c, the predicted expected value is:

[0316]

[0317] Model uncertainty is quantified as prediction entropy, which represents the average amount of information contained in the prediction distribution. Results with low entropy values ​​correspond to confident predictions. The sampled prediction entropy is used as the uncertainty of the model through the MC-dropout method, which is expressed as:

[0318]

[0319] Specifically, scenarios where uncertainty estimates are directly used in other model training include:

[0320] Get the data and preprocess it:

[0321] This embodiment first filters duplicate locations on the OIDD data set, and only retains the first record of the time period for the continuous stay trajectory at the same location. Then, cell oscillation is removed, and records of abnormally fast movement speed or frequent back-and-forth movement are excluded. Finally, users and locations are filtered, and unpopular locations with less than 10 visits are removed, and inactive users with less than 5 activity records are removed.

[0322] Data uncertainty driven loss function optimization: Trajectory Entropy Harmonizing (TEH) loss function:

[0323] The user's trajectory sequence can be regarded as a random process, and the entropy of the random process can be used to calculate the uncertainty of the trajectory sequence. Given a trajectory sequence S u =p 1 p 2 …p n , a space-time point p is a tuple of a timestamp t and a location identifier l, i.e., p = (t, l), and its position sequence is defined as: L u = l 1 l 2 …l n .

[0324] Trajectory entropy is defined to represent the uncertainty of the location a user will visit at the next moment. It depends not only on the frequency of the user's visits to each location, but also on the order of the visited locations and the length of time the user stays at each location, thereby fully capturing the spatiotemporal structure in the individual flow pattern, and is calculated as

[0325]

[0326] Among them, P(L′ u ) is on the trajectory S u Find a specific time ordered subsequence L′ u The Lempel-Ziv estimation method is used to quickly converge to the actual entropy value. The simplified trajectory entropy calculation formula is:

[0327]

[0328] Among them, Λ i represents the length of the shortest subsequence starting at position i that occurs for the first time in the entire sequence of positions up to position i-1.

[0329] Taking the probability density distribution of trajectory entropy as weight, trajectories with large probability density are given more weight, and the entropy values ​​of these trajectories are at a medium level; trajectories with low probability density are given less weight, and the entropy values ​​of these trajectories are either extremely low, belonging to easy-to-learn samples with low training efficiency, or extremely high, belonging to difficult-to-learn samples, which can be regarded as noise.

[0330] The core of the trajectory entropy coordination loss function is to define the trajectory entropy density function:

[0331]

[0332] where e k is the trajectory entropy probability density of the kth sample, δ ε (e k , e) represents the number of samples whose trajectory entropy distribution is in the range of (e-ε / 2, e+ε / 2). If it is in the range, δ ε (e k , e) = 1, otherwise, δ ε (e k , e)=0,l ∈ (e) represents the length of the interval (e-ε / 2, e+ε / 2). According to the kernel density estimation method, the trajectory entropy density corresponding to each trajectory sample is calculated. The cross entropy loss L CE As the loss function, the TEH loss function is defined as:

[0333]

[0334] where p k and They represent the true value and predicted value of the kth sample respectively.

[0335] In order to avoid the problem that the weighted differentiation of trajectory entropy density is not large due to the similar trajectory entropy of samples in the same batch, we further calculate the ED (e k ) after the normalization function f w (*) is a standardized transformation, defined as:

[0336]

[0337] Where B is the batch size set during model training, U b is the set of trajectories in this batch in the training set. In this way, each time the parameters are updated, some sample instance weights f W (ED(e k )) will be greater than 1, and the weight f of some sample instances W (ED(e k )) will be less than 1, achieving the purpose of weight allocation. Through the weight allocation strategy, the parameters of the final model are updated:

[0338]

[0339] Where lr is the learning rate, represents the gradient operator, W t and U t They represent the model trainable parameters and the corresponding trajectory set in the tth training batch. Training samples with high trajectory entropy density have a more significant impact on parameter updating, and most trajectories with medium trajectory entropy have a greater impact on model parameter updating.

[0340] Model uncertainty estimation

[0341] The Dropout algorithm uses the Dropout layer to implement random sampling during the inference process, effectively reducing the implementation complexity. In the training phase, the Dropout layer reduces the overfitting of the model by randomly discarding some neurons, thereby enhancing the generalization ability of the model. MC Dropout further extends the application of Dropout from the training phase to the testing phase, and implements a mechanism to randomly shut down some neurons each time the model predicts, which is equivalent to using different network architectures and parameter configurations in each training batch.

[0342] refer to Figure 6 As shown, Figure 6 This paper proposes a method framework for model uncertainty estimation using the MC-dropout method. An input sample can obtain multiple outputs after passing through the MC-Dropout layer, and the prediction results and model uncertainty are finally calculated based on these outputs.

[0343] The MC Dropout algorithm sets q(w) to be the distribution of a matrix with columns randomly set to zero according to a Bernoulli distribution:

[0344]

[0345] where z i,j ~Benoulli(pi), where i = 1, ..., L, and j = 1, ..., K i-1 The dimension of the W matrix here is K i ×K i-1 , p i Indicates the probability of a node being blocked during the Dropout process, M i Represents the parameters of the variation matrix.

[0346] When the Dropout layer is turned on during the test, the neural network is randomly forward-inferred T times, which is equivalent to sampling T times from the Bernoulli distribution to obtain a set of network parameters. Based on this, the model for a given input x* The expected value of the prediction is:

[0347]

[0348] where y * Represents the input x under the current neural network parameters * prediction results.

[0349] Using the MC-dropout method, the output results are presented as probability distributions after multiple random dropout operations, which can be regarded as samples of the distribution inferred from the Bayesian neural network. For the classification category c, the predicted expected value is:

[0350]

[0351] Model uncertainty can be quantified as prediction entropy, which represents the average amount of information contained in the prediction distribution. Results with low entropy values ​​correspond to confident predictions, while results with high entropy values ​​have great uncertainty. The MC-dropout method can be used to treat the sample prediction entropy as model uncertainty, expressed as:

[0352]

[0353] Select 3 models as basic models:

[0354] (1) ST-RNN: A model based on RNN sequence modeling.

[0355] (2) DeepMove: A model based on RNN sequence modeling and assisted by the attention mechanism

[0356] (3)RTA: A model based on self-attention mechanism sequence modeling (RTA-HP). In order to ensure the simplicity of the loss function and the inference process, the Hawkes process inference part is removed.

[0357] “+TEH” means that the basic model uses the TEH loss function, “+MC” means that the basic model is tested using the MCDropout method, and “+TEH+MC” means that the TEH loss function is used and the MC Dropout method is used for testing.

[0358] The effects of this embodiment are shown in Table 4.

[0359] Table 2-4 Effects of TEH loss function and MC Dropout method

[0360]

[0361] Therefore, by calculating the data uncertainty in the trajectory as an evaluation indicator of data quality, and then reducing the impact of low-quality data on the model by reducing sample weights, the quality of the collected data set can be improved to a certain extent.

[0362] By estimating model uncertainty, we can determine the model's understanding of the data and adjust the loss weights of each spatiotemporal point in the trajectory, so that the model can make the best use of reliable information.

[0363] For the discrete-time human mobility prediction problem, the sparsity of place visits is addressed by learning two relationship graphs between places, making full use of the geographic spatial relationship and collective mobility characteristics between places to reduce the impact of place sparsity. For the multi-level periodicity of human mobility, the periodicity of the sequence is modeled by capturing the periodic meaning in the timestamp and explicitly extracting multiple period lengths in the trajectory sequence, thereby capturing the user's mobility pattern and improving the accuracy of prediction.

[0364] For the problem of human mobility prediction in continuous time, the temporal sparsity of trajectories makes it difficult to capture user mobility patterns. Through the relative time self-attention mechanism, the time interval information of two spatiotemporal points is embedded into the calculation of spatiotemporal point similarity, and the trajectories are naturally divided into historical trajectories and recent trajectories. The periodicity of the former and the sequentiality of the time decay of the latter are considered, so that the model can simultaneously learn the time interval information and periodic information of sparse trajectories. For the problem of unclear next location prediction caused by the temporal sparsity of trajectories, the most likely location of the next spatiotemporal point under uncertain time conditions is obtained by constructing the trajectories into a Hawkes process, which fully considers various time conditions and improves the interpretability and accuracy of the prediction.

[0365] The temporal sparsity of trajectory data and the noise factors in the trajectory bring a lot of uncertainty to the prediction. By calculating the data uncertainty in the trajectory as an evaluation indicator of data quality, the impact of low-quality data on the model can be reduced by reducing the sample weight and other methods, thus improving the quality of the collected data set to a certain extent. By estimating the model uncertainty, the model's understanding of the data can be judged, and then the loss weight of each spatiotemporal point in the trajectory can be adjusted, so that the model can make full use of reliable information as much as possible.

[0366] According to another aspect of the present invention, a human mobility prediction and uncertainty estimation system 700 based on sparse spatiotemporal trajectories is also provided, comprising:

[0367] A space-time vector acquisition module 710 is used to acquire and pre-process the user's open information dynamic data to obtain the space-time trajectory, and perform dual-graph convolution embedding and global timestamp embedding on the space-time trajectory to obtain the space-time vector;

[0368] The discrete trajectory mobility prediction module 720 is used for determining the discrete time trajectory prediction model based on the sequence encoder of the autocorrelation mechanism for the discrete time trajectory, taking the space-time vector as input, predicting the location of the discrete time trajectory at the next moment, and evaluating the location prediction effect based on the evaluation index, and optimizing the discrete time trajectory prediction model;

[0369] The continuous trajectory mobility prediction module 730 is used for determining the continuous time trajectory prediction model for the continuous time trajectory based on the relative time self-attention sequence encoder, taking the spatiotemporal vector as input, predicting the location and time of the next spatiotemporal point of the continuous time trajectory, evaluating the location prediction effect based on the evaluation index, measuring the accuracy of the time based on the mean square error, and optimizing the continuous time trajectory prediction model;

[0370] The uncertain mobility prediction module 740 is used to define data uncertainty as trajectory entropy based on the spatiotemporal trajectory for the uncertain trajectory, determine the trajectory entropy coordination loss function based on the trajectory entropy, and estimate the uncertainty of the model.

[0371] A human mobility prediction and uncertainty estimation system 700 based on sparse spatiotemporal trajectories in an embodiment of the present invention corresponds to a human mobility prediction and uncertainty estimation method 100 based on sparse spatiotemporal trajectories in another embodiment of the present invention, which will not be repeated here.

[0372] Those skilled in the art will appreciate that the embodiments of the present application can be provided as methods, systems, or computer program products. Therefore, the present application can adopt the form of complete hardware embodiments, complete software embodiments, or embodiments in combination with software and hardware. Moreover, the present application can adopt the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that contain computer-usable program code. The scheme in the embodiments of the present application can be implemented in various computer languages, for example, object-oriented programming language Java and literal scripting language JavaScript, etc.

[0373] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1A device that provides the functions specified in a block or multiple blocks.

[0374] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0375] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0376] Although the preferred embodiments of the present application have been described, those skilled in the art may make other changes and modifications to these embodiments once they have learned the basic creative concept. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications falling within the scope of the present application.

[0377] Obviously, those skilled in the art can make various changes and modifications to the present application without departing from the spirit and scope of the present application. Thus, if these modifications and variations of the present application fall within the scope of the claims of the present application and their equivalents, the present application is also intended to include these modifications and variations.

Claims

1. A method for predicting and estimating human mobility based on sparse spatiotemporal trajectories, characterized in that: include: Obtain the user's open information dynamic data and preprocess it to obtain the spatiotemporal trajectory, and perform dual graph convolution embedding and global timestamp embedding on the spatiotemporal trajectory to obtain the spatiotemporal vector; For discrete time trajectories, the sequence encoder based on the autocorrelation mechanism takes the space-time vector as input, determines the discrete time trajectory prediction model, predicts the location of the discrete time trajectory at the next moment, and evaluates the location prediction effect based on the evaluation index to optimize the discrete time trajectory prediction model; For continuous time trajectories, based on the relative time self-attention sequence encoder, the spatiotemporal vector is used as input to determine the continuous time trajectory prediction model, predict the location and time of the next spatiotemporal point of the continuous time trajectory, and evaluate the location prediction effect based on the evaluation index. The accuracy of time is measured based on the mean square error, and the continuous time trajectory prediction model is optimized; For uncertain trajectories, data uncertainty is defined as trajectory entropy based on the spatiotemporal trajectory, and based on the trajectory entropy, a trajectory entropy coordination loss function is determined to estimate the uncertainty of the model.

2. The method according to claim 1, characterized in that Obtain the user's open information dynamic data and preprocess it to obtain the spatiotemporal trajectory, perform dual graph convolution embedding and global timestamp embedding on the spatiotemporal trajectory to obtain the spatiotemporal vector, including: Acquire the user's open information dynamic data, pre-process the open information dynamic data, and obtain a spatiotemporal trajectory, including an initial location vector and a timestamp; Determine the spatial graph and the mobile flow graph, perform dual-graph convolution location feature embedding on the initial location vector, and obtain the location embedding vector: The spatial graph G s =(L s , E s , A s ), L represents the set of locations, E s represents the set of edges, A s ∈R |L|×|L| represents the adjacency matrix, A s (i, j) represents l i , l j The spatial proximity relationship between two locations is defined as: where dist(l i , l j ) is the Euclidean distance between two locations, Δ is the distance threshold, indicating l i , l j The spatial relationship between them decreases as the distance between the two locations increases. When the distance exceeds Δ, there is no edge connection; Mobile flow graph G f =(L f , E f , A f ), L represents the set of locations, E f represents the set of edges, A f ∈R |L|×|L| represents the adjacency matrix, A f (i, j) represents l i , l j The spatial proximity relationship between two locations is defined as: A f (i,j)=count(l i ,l j ) Where count(l i , l j ) means that in all user trajectories, i , l j The number of times two locations are visited consecutively, if (l i , l j ) subsequence appears in a trajectory of a certain user, then l i , l j There are edges between them, and the mobility flow graph simulates the movement pattern of the crowd; Performing global timestamp feature embedding on the timestamp to obtain a time embedding vector; The graph neural network GCN based on the spatial graph and the mobile flow graph learns in parallel. After the node is updated, the location vectors es and ef are output respectively. The aggregation layer is applied to generate the final location embedding vector: And l =Add(e) s ,And f )=Φe s +(1-Φ)e f where Φ∈R C It is a trainable parameter vector used to control the contribution of the topological information of the two graphs to different final representations. By fusing the two relationship graphs, the inadequacy of the geographical distance in expressing the complex relationship between places can be compensated by the mobility tendency of all users. At the same time, the information gap caused by the sparsity of place visits can be filled by the relationship between places with closer distances, so as to more fully represent the places. Aggregate the time embedding vector and the location embedding vector to obtain the space-time vector: The information selected from the timestamp includes the calendar timestamp minute, hour, day, week, month, and year, which represent the relative offset of the timestamp in this periodic layer, and the special timestamp holidays and events, which are used to distinguish the different mobility patterns of users on weekdays and holidays. Each global timestamp scalar is normalized and mapped to a vector representation with the same dimension as the location embedding through a learnable timestamp embedding layer SE(*). The time embedding vector and the location embedding vector are combined to obtain a spatiotemporal point vector.

3. The method according to claim 2, characterized in that For discrete time trajectories, the sequence encoder based on the autocorrelation mechanism takes the space-time vector as input, determines the discrete time trajectory prediction model, and predicts the location of the next moment of the discrete time trajectory, including: Given the current trajectory and historical trajectory of user u, the long trajectory sequence is concatenated in time, and the input vector of the first layer encoder is obtained after location embedding and time embedding: X (0) =and p1 And p2 ...And pm Where m is the length of the trajectory; For the lth layer of the encoder, the input sequence vector X (l-1) After mapping the trainable parameter matrix, three vector sequence query vectors can be obtained respectively Q=X (l-1) W Q ,K=X (l-1) W K ,V=X (l-1) W V Apply the autocorrelation mechanism to get each head output: Head i =AutoCoelation(Q i ,K i ,V i ) in Roll(X,τ i ) represents a rolling operation with a time delay τ on the sequence X, during which the element at the first position of the sequence will be moved to the last position, aligning and integrating similar subsequences at the same phase position of the estimation cycle, R Q,K (τ) represents the cross-correlation between the sequences Q and K, i.e., the autocorrelation of the time series X, L is the length of the time series, and k represents the k time delays τ1,…,τ with the largest autocorrelation function. k As the k-layer cycle dependency of the trajectory; The final output of the multi-head autocorrelation module is: In each encoder layer, a feedforward neural network (FC) is connected after the autocorrelation module, which is calculated as: Where W1 and W2 are trainable weight matrices, b1 and b2 are biases, and finally, the output of the lth encoder layer is: In order to predict the user's next action, a multi-layer perceptron MLP is used to output the final probability distribution of the location at the next moment: Where W out is a trainable weight matrix, b out It is a deviation; The cross entropy loss is used as the loss function for mini-batch training, expressed as: Among them, the model outputs the corresponding next moment location for each space-time point in the sequence, and uses a mask method to cover the information after the current space-time point.

4. The method according to claim 1, characterized in that: Evaluate the location prediction effect based on evaluation indicators and optimize the discrete time trajectory prediction model, including: Set the model hidden layer dimension to 128. During the training process, use the gradient descent method, set the training batch size to 32, and optimize the model parameters based on the Adam optimizer. The evaluation indicators include accuracy and average reciprocal ranking. Accuracy, Acc: measures the degree of match between the predicted location and the actual next access location. For a given test data set, it is defined as the ratio of the number of samples with the correct next access location to the total number of samples. Acc@k indicates whether the true location is in the top k most likely locations predicted by the model, and is calculated as: Among them, N represents the number of test samples, a is a discriminant function, which returns 1 if the condition is true, otherwise it returns 0, and rank i is the i-th test sample The ranking of the real position in; Mean reciprocal rank (MRR): It calculates the average reciprocal rank of the first relevant entry retrieved in the prediction vector, calculated as:

5. The method according to claim 1, characterized in that For continuous time trajectories, based on the relative time self-attention sequence encoder, the spatiotemporal vector is used as input to determine the continuous time trajectory prediction model, and the location and time of the next spatiotemporal point of the continuous time trajectory are predicted, including: Extract the open information dynamic data of continuous time trajectory for preprocessing, filter repeated locations, and obtain the continuous time and space trajectory; Perform dual graph convolution embedding and global timestamp embedding on the spatiotemporal trajectory to obtain the spatiotemporal vector; Based on the relative time self-attention mechanism, the trajectory sequence is modeled and the continuous time trajectory prediction model is determined: Construct a relative time difference matrix to represent the time difference in the trajectory sequence. Each time difference corresponds to a learnable vector representation and affects the attention score of two spatiotemporal points with corresponding time differences in the sequence. The trajectory is divided into historical trajectory and recent trajectory. The former only considers periodicity, while the latter also considers relative time decay. In order to express the relative time difference between space-time points, the time difference is used to represent the distance relationship between different positions. First, a relative time vector is designed. Represents the corresponding vectors in each relative time case, and globally shares the data in the model, where k is the maximum relative time in the sequence. In addition, a time difference matrix δt is defined, with elements δ t (i, j)∈[0,k] represents the time difference from the i-th space-time point to the j-th space-time point: In the self-attention mechanism, the content information and position information in the sequence are considered at the same time. For the i-th point in the sequence, two vectors {H i } and {P i|j } to represent the content and the position information relative to the jth point, then the calculation of the cross attention score between i and j is: In the self-attention mechanism, the input content sequence vector H is a fusion vector including location embedding and cycle time embedding. After mapping the trainable parameter matrix, three vectors content query vector Q can be obtained respectively. c , content key vector K c , content value vector V c : Q c =HW q,c| ,K c =HW k,c ,V c =HW v,c In addition, the relative time query vector Q is additionally calculated r and the relative time key vector K r : Q r =PW q,r ,K r =PW k,r The self-attention matrix based on relative time is Represents the attention score from the i-th point to the j-th point: Respectively represent content-content, content-time, and time-content information, where The matrix Q c The i-th row vector of is the matrix K c The j-th row vector of is the δ(i, j)th row vector of the matrix Kr, The matrix Q r The δ(j, i)th row vector of ; Query vector Q by relative time r And the relative time key vector Kr, the time difference in the sequence is mapped into the corresponding vector, which is integrated into the self-attention matrix. For the space-time points that differ from the current space-time point by more than k, they will all be mapped to P t In the last vector of , that is, for these spacetime points, The last two terms of the attention score are the same, which has nothing to do with the time difference and is only related to the representation of the spatiotemporal point, shielding the influence of time decay and playing the role of masking; The output of self-attention is: in Divide by a scaling factor to stabilize the model training process; Relative temporal encoding can not only capture the time difference in short-term dependencies, but also learn the periodicity of long-term dependencies.

6. The method according to claim 5, characterized in that For continuous time trajectories, based on the relative time self-attention sequence encoder, the spatiotemporal vector is used as input to determine the continuous time trajectory prediction model, predict the location and time of the next spatiotemporal point of the continuous time trajectory, and also include: set up is the conditional strength function of the model, where is the historical trajectory up to time t. The definition of the intensity function is different for different location categories, that is, for each l∈{1, 2, ..., L}, is defined as the conditional intensity function of location l, and the conditional intensity function of the entire trajectory is defined as: The conditional intensity function of each location category is expressed as: Time is defined in the interval t∈[t i ,t],f l (x) = β l log(1+exp(x / β l )) is the parameter β l Softplus function, in the conditional intensity function, the current part affects the timestamps t of the two space-time points i and t i+1 The interpolation between l Used to adjust the importance of interpolation. The history part is cut off to t i The hidden state h(t i ); the base part indicates the probability of an event occurring without considering historical information; Using a neural network to represent the Hawkes process, define N(t) = |h:t h ≤t| is the count of spacetime points before time t, given the past history (t i , the number of events in t) is expressed as: ΔN(t i ,t)=N(t)-N(t i ) Let T i >t i Let L be the random variable of the next time. i+1 is the random variable of the next location, T i The cumulative distribution function (in As a condition) is: set up but It is expressed as: The corresponding probability density function is: Given the past history and the time t of the next spacetime point i+1 , Location i+1 The distribution of is: Predict the time of the next spacetime point: Given the next time t i+1 , the corresponding next location is: In the unknown i In this case, it is necessary to consider t i The next location is: where the same value of t is used for pairwise comparisons for each k in argmax, sharing λ(t) and p across all k i (t) calculation; For the RTA-HP model, given its conditional strength function For any sequence S in the observation interval [0,T], the likelihood function is: Correspondingly, the log-likelihood is defined as: The model parameters are learned by maximizing the log-likelihood of all sequences. Specifically, suppose there are N trajectories S1, S2, ..., S N , in order to find the solution parameters, the goal of model training is: Each Added to explain why at time t i User arrival location i ,and Reduced to explain why no location l∈{l1,...,l |L| }.

7. The method according to claim 6, characterized in that The location prediction effect is evaluated based on the evaluation index, the time accuracy is measured based on the mean square error, and the continuous time trajectory prediction model is optimized, including: Evaluate the location prediction effect based on the evaluation index; The root mean square error (RMSE) is quantified in minutes. The root mean square error is used to calculate the average of the squares of the difference between the time predicted by the model and the actual time. This is used to measure the accuracy of time prediction, eliminate the influence of the time interval scale, and normalize the prediction error: in Represents the predicted timestamp, t i Represents the real timestamp, and the corresponding mean square error of time is:

8. The method according to claim 1, characterized in that For uncertain trajectories, data uncertainty is defined as trajectory entropy based on the spatiotemporal trajectory, a trajectory entropy coordination loss function is determined based on the trajectory entropy, and the uncertainty of the model is estimated, including: Filter duplicate locations in the open information dynamic data of spatiotemporal trajectories, exclude records of abnormally fast movement speed or frequent back-and-forth movement, remove unpopular locations with fewer than a predetermined number of visits and inactive users with fewer than a certain number of activity records, and obtain uncertain spatiotemporal trajectories; The data uncertainty is defined as the trajectory entropy based on the spatiotemporal trajectory of the uncertainty: The user's trajectory sequence is regarded as a random process, and the uncertainty of the trajectory sequence is calculated based on the entropy of the random process. Given a trajectory sequence S u =p1p2…p n , a space-time point p is a tuple of a timestamp t and a location identifier l, i.e., p = (t, l), and its position sequence is defined as: L u =l1l2…l n ; Trajectory entropy is defined to represent the uncertainty of the location a user will visit at the next moment. It depends not only on the frequency of the user's visits to each location, but also on the order of the visited locations and the length of time the user stays at each location, thereby fully capturing the spatiotemporal structure of the individual flow pattern, and is calculated as Among them, P(L′ u ) is on the trajectory S u Find a specific time ordered subsequence L′ u The probability of using the Lempel-Ziv estimation method to quickly converge to the actual entropy value, the simplified trajectory entropy calculation formula is: Among them, Λ i represents the length of the shortest subsequence starting from position i that appears for the first time in the entire sequence of positions up to position i-1; Taking the probability density distribution of trajectory entropy as the weight, define the trajectory entropy density function: where e k is the trajectory entropy probability density of the kth sample, δ ε (e k , e) represents the number of samples whose trajectory entropy distribution is in the range of (e-ε / 2, e+ε / 2). If it is in the range, δ ε (e k , e) = 1, otherwise, δ ε (e k , e)=0,l ∈ (e) represents the length of the interval (e-ε / 2, e+ε / 2). According to the kernel density estimation method, the trajectory entropy density corresponding to each trajectory sample is calculated and the cross entropy loss L is used. CE As the loss function, the TEH loss function is defined as: where p k and Represent the true value and predicted value of the kth sample respectively; Further analysis of the same batch of ED (e k ) after the normalization function f w (*) is a standardized transformation, defined as: Where B is the batch size set during model training, U b is the set of trajectories in this batch in the training set. In this way, each time the parameters are updated, there must be some sample instance weights f W (ED(e k )) will be greater than 1, and the weight of some sample instances f W (ED(e k )) will be less than 1, achieving the purpose of weight allocation. Through the weight allocation strategy, the parameters of the final model are updated: Where lr is the learning rate, represents the gradient operator, W t and U t They respectively represent the model trainable parameters and the corresponding trajectory set in the tth training batch.

9. The method according to claim 8, characterized in that The trajectory entropy coordination loss function is determined based on the trajectory entropy to estimate the uncertainty of the model, including: Let q(w) be the distribution of a matrix with columns randomly set to zero according to a Bernoulli distribution: where z i,j ~Benoulli(pi), where i = 1, ..., L, and j = 1, ..., K i-1 The dimension of the W matrix here is K i ×K i-1 , p i Indicates the probability of a node being blocked during the Dropout process, M i Represents the parameters of the variation matrix. When the Dropout layer is turned on during the test, the neural network is randomly forward-inferred T times, which is equivalent to sampling T times from the Bernoulli distribution to obtain a set of network parameters. Based on this, the model for a given input x * The expected value of the prediction is: where y * Represents the input x under the current neural network parameters * The prediction results; The output is presented as a probability distribution after multiple random Dropout operations. These probability distributions can be regarded as samples of the distribution inferred from the Bayesian neural network. For the classification category c, the predicted expected value is: Model uncertainty is quantified as prediction entropy, which represents the average amount of information contained in the prediction distribution. Results with low entropy values ​​correspond to confident predictions. The sampled prediction entropy is used as the uncertainty of the model through the MC-dropout method, which is expressed as:

10. A human mobility prediction and uncertainty estimation system based on sparse spatiotemporal trajectories, characterized in that: include: The space-time vector module is used to obtain the user's open information dynamic data and perform preprocessing to obtain the space-time trajectory, and perform dual-graph convolution embedding and global timestamp embedding on the space-time trajectory to obtain the space-time vector; The discrete trajectory mobility prediction module is used for discrete time trajectories. Based on the sequence encoder of the autocorrelation mechanism, the space-time vector is used as input to determine the discrete time trajectory prediction model, predict the location of the discrete time trajectory at the next moment, and evaluate the location prediction effect based on the evaluation index to optimize the discrete time trajectory prediction model. The continuous trajectory mobility prediction module is used for continuous time trajectories. Based on the relative time self-attention sequence encoder, it takes the spatiotemporal vector as input, determines the continuous time trajectory prediction model, predicts the location and time of the next spatiotemporal point of the continuous time trajectory, evaluates the location prediction effect based on the evaluation index, measures the accuracy of time based on the mean square error, and optimizes the continuous time trajectory prediction model. The uncertain mobility prediction module is used to define data uncertainty as trajectory entropy based on the spatiotemporal trajectory for uncertain trajectories, determine a trajectory entropy coordination loss function based on the trajectory entropy, and estimate the uncertainty of the model.

Citation Information

Cited By

  • Motion track prediction data processing method and device, electronic equipment and storage medium

    CN122219113A

  • Motion trajectory prediction data processing method and device, electronic equipment and storage medium

    CN122219113B