User position prediction method based on multi-dimensional historical behavior trajectory data
By adopting a self-supervised comparative learning model of multi-dimensional historical behavior trajectory data in user location prediction, combining location, network behavior and social relationship information, the problem of traditional methods failing to make full use of user behavior characteristics is solved, and more accurate and reliable is achieved user location prediction.
Patent Information
- Application Number
- CN202411100861.7
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Priority Date
- 2024-06-14
- Filing Date
- 2024-08-12
- Publication Date
- 2025-01-03
AI Technical Summary
Traditional user location prediction methods ignore the learning of users' more comprehensive behavioral trajectory and fail to make full use of location attributes, resulting in insufficient prediction accuracy.
A self-supervised comparative learning model based on multi-dimensional historical behavioral trajectory data is adopted. By embedding position coordinates, network behavior and social relationship information, the training model is designed to extract user behavior characteristics and realize user position prediction.
Effectively extracting deep characteristics of human behavior, improving the accuracy and reliability of user location prediction, and is suitable for multiple fields, including location prediction and abnormal behavior detection.
Smart Images
Figure CN119249308A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of spatiotemporal data mining, and in particular to a method for predicting user locations based on multi-dimensional historical behavior trajectory data. Background Art
[0002] The development and popularization of Internet technology has led to the mining of big data containing a large amount of potential information. This also includes the accumulation of a large amount of user behavior trajectory data, which facilitates the mining of user behavior patterns. Accurate prediction of user location information is a necessary foundation for many popular location-based applications. Accurately identifying restaurants, attractions, or any location-tagged content that users want to know about can greatly improve the user experience. Based on this demand, the user location prediction problem has attracted great attention from researchers, and issues such as point of interest recommendations and travel plans have become a research topic that needs to be solved urgently.
[0003] Traditional methods model location sequences based on the assumption of periodic similarity of users, but ignore the learning of more comprehensive user behavior trajectories and do not fully utilize location attributes. Recently, with the development of graph neural networks, research on user location prediction has focused on combining geographic influence with user historical location data. For example, DisenPOI uses an explicit modeled geographic area graph combined with user sequences to utilize sequential influence and geographic influence on the accuracy of location prediction, and achieves relatively good results. Summary of the invention
[0004] The present invention proposes a user location prediction method based on multi-dimensional historical behavior trajectory data. By embedding multi-dimensional information of location coordinates, network behavior, and social relationships, a self-supervised contrastive learning training model is designed for training and user behavior features are extracted to achieve user location prediction.
[0005] Based on the above objectives, the present invention provides a method for predicting user location based on multi-dimensional behavior trajectories, which is characterized by comprising:
[0006] Acquire historical behavior trajectory data and perform data access, data cleaning and data compression to obtain a preprocessed data set; the data access is used to integrate the historical behavior trajectory data into a data set form; the data cleaning is used to eliminate noise data; the data compression is used to reduce the resources consumed when storing data;
[0007] Performing data enhancement based on the preprocessed data set to obtain a training data set; the data enhancement is to expand the trainable content; the training data set includes a user data set and a data training data set;
[0008] The behavior sequence feature embedding vector and the spatial structure feature embedding vector are input into a self-supervised contrastive learning training model to obtain a user location prediction location; wherein the self-supervised contrastive training model is established by the following steps:
[0009] Position encoding is superimposed on the behavior sequence feature embedding vector and the spatial structure feature embedding vector, and combined with the social relationship embedding vector to obtain a user representation vector and a training representation vector;
[0010] The user representation vector is passed through the output layer to obtain the predicted location of the user's position.
[0011] Preferably, the historical behavior trajectory data includes: location information sequence, network behavior information sequence, and social relationship information sequence; the location information sequence includes location coordinates and timestamp; the network behavior information sequence includes visits to mobile phone software or websites and their specific content; the social relationship information sequence includes user relationships and interaction records.
[0012] Preferably, the data access method includes: screening and filtering the historical behavior trajectory data and integrating them into a data set format.
[0013] Preferably, the data cleaning method includes: processing positioning errors and filtering meaningless heartbeat connection signals.
[0014] Preferably, the data compression method includes: sampling and compressing the data set, processing possible data conflicts, and obtaining a preprocessed data set.
[0015] Preferably, the data enhancement method includes: creating a new sequence user trajectory based on the preprocessed data set, which is represented by T = {p1, p2, ..., p |T|}, where p i is the i-th trajectory point, and its structure is {lng i ,lat i ,behavior i ,time i}; Perform trajectory clipping and / or trajectory masking and / or stay point position shifting and / or stay point simplification and / or behavior transformation and / or behavior masking on the user trajectory, and integrate the data into a training data set.
[0016] Preferably, the method of establishing a behavior sequence feature embedding vector and a spatial structure feature embedding vector based on each trajectory point in the training data set, and establishing a social relationship embedding vector based on the social relationship information sequence includes:
[0017] Behavior sequence feature embedding: Based on the longitude and latitude from the location information of each trajectory point in the training dataset, network behavior, the distance between the current trajectory point and the previous trajectory point, and the timestamp, a five-tuple is established as the behavior sequence feature of each trajectory point; all behavior sequence features are integrated into a behavior sequence feature embedding vector;
[0018] Spatial structure feature embedding: Based on the longitude and latitude of each trajectory point in the training set, the number of the known location is obtained as a node; a spatial structure graph is established based on the node, and the spatial structure feature embedding vector is obtained through the embedding algorithm;
[0019] Social relationship embedding: A directed graph is obtained based on the user ID and user relationship in the training dataset, and the social relationship embedding vector is obtained through the embedding algorithm.
[0020] Preferably, the method for obtaining the user representation vector and the training representation vector includes:
[0021] The behavior sequence feature embedding vector and the spatial structure feature embedding vector are superimposed and positionally encoded to obtain two new vectors; the two new vectors are used as input and passed through a multi-head self-attention module to obtain two intermediate vectors; the intermediate vector is fused with the social relationship embedding vector and then the representation vector of the intermediate vector is generated through the self-attention layer, thereby obtaining the user representation vector and the data training representation vector.
[0022] The beneficial effects of the present invention are as follows:
[0023] 1. Through the self-supervised contrastive learning method, we can effectively extract the deep features of human behavior from massive unlabeled trajectory data and make similar users as close as possible in the representation space, thus providing a solid foundation for subsequent location prediction tasks.
[0024] 2. A data enhancement method is designed based on the characteristics of trajectory data, including trajectory clipping, trajectory masking, stop point simplification, stop point position offset, behavior transformation and behavior masking. These methods can adapt to different data missing and deviation situations, such as the adaptability of device activation on the road, the adaptability of trajectory missing, the adaptability of positioning deviation and the adaptability of partial Internet log missing, thereby improving the generalization ability and robustness of the model.
[0025] 3. The model makes full use of the multi-dimensional features of location information, network behavior and social relationships. The model's integration of multi-dimensional features can more comprehensively capture the user's behavior patterns, thereby improving the accuracy and reliability of the prediction.
[0026] 4. The present invention mines the value of a large amount of user historical behavior trajectory data, which is not only suitable for location prediction tasks, but can also be extended to multiple fields such as user alignment and abnormal behavior detection, providing a reliable method for many popular location-based applications. BRIEF DESCRIPTION OF THE DRAWINGS
[0027] Figure 1 .A flowchart of a user location prediction method based on multi-dimensional historical behavior trajectory data provided by the present invention.
[0028] Figure 2 .A schematic diagram of a user location prediction method based on multi-dimensional historical behavior trajectory data provided by the present invention.
[0029] Figure 3 .A schematic diagram of a data enhancement method provided by the present invention.
[0030] Figure 4 .A schematic diagram of a self-supervised contrastive learning model provided by the present invention. DETAILED DESCRIPTION
[0031] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, the present invention is further described in detail below through specific embodiments and drawings.
[0032] refer to Figure 1 , Figure 2 An embodiment of the present invention provides a method for predicting user location based on multi-dimensional historical behavior trajectory data, which specifically includes the following steps:
[0033] Acquire historical behavior trajectory data and perform data access, data cleaning and data compression to obtain a preprocessed data set; the data access is used to integrate the historical behavior trajectory data into a data set form; the data cleaning is used to eliminate noise data; the data compression is used to reduce the resources consumed when storing data;
[0034] Performing data enhancement based on the preprocessed data set to obtain a training data set; the data enhancement is to expand the trainable content; the training data set includes a user data set and a data training data set;
[0035] The behavior sequence feature embedding vector and the spatial structure feature embedding vector are input into a self-supervised contrastive learning training model to obtain a user location prediction location; wherein the self-supervised contrastive training model is established by the following steps:
[0036] Position encoding is superimposed on the behavior sequence feature embedding vector and the spatial structure feature embedding vector, and combined with the social relationship embedding vector to obtain a user representation vector and a training representation vector;
[0037] The user representation vector obtains the user location prediction location through the output layer;
[0038] During the model usage phase, the historical behavior trajectory data to be predicted is obtained and data access, data cleaning and data compression are performed to obtain a preprocessed data set, and the training data set is input into the self-supervised comparative learning training model to obtain the user location prediction location.
[0039] In the step, the historical behavior trajectory data includes the user's movement trajectory data, semantic behavior data and social relationship data.
[0040] The mobile trajectory data is a sequence of location information of the mobile device used by the user over a period of time, including location coordinates and timestamps, which are used to describe and analyze the user's mobile behavior and patterns. The semantic behavior data is a sequence of network behavior information of the user over a period of time, including visits to apps or websites and specific website content, which are used to describe and analyze detailed records of the user's interaction with the Internet. The social relationship data is a sequence of social relationship information formed by users on social platforms. This includes user relationships, follow lists, likes, comments, shares, and other forms of interaction records, reflecting the user's connection patterns and interaction habits in social networks.
[0041] Secondly, the present invention provides a hierarchical data preprocessing process, which includes: a data access module; a data cleaning module; and a data conflict processing module. Among them:
[0042] Data access module: adapts information from different data sources, especially various types of data generated by users in the process of using the application, including mobile trajectory data, semantic behavior data, and social relationship data, and integrates them into data sets. First, identify and read these data, and then perform preliminary screening and quality control according to the threshold set by the user. This includes detecting and recording fields with a data missing rate higher than 50%, and filtering out historical behavior trajectory data with insufficient data volume to ensure that the quality of the accessed data meets the requirements of subsequent analysis. The historical behavior trajectory data is processed by the data access module and used as input for subsequent data processing and analysis steps.
[0043] Data cleaning module: It can process and optimize the location information sequence, network behavior information sequence and social relationship information sequence in the data set in detail to improve the quality and availability of the data. In terms of specific operations, the data cleaning module processes the following two situations:
[0044] 1. Position information sequence correction: This is to handle the "simultaneous but different locations" phenomenon caused by manual positioning errors by the user or system positioning errors, as well as the abnormal situation of large-scale position drift in a short period of time.
[0045] 2. Abnormal record identification and filtering: Identify and filter out a large number of meaningless heartbeat connection signals. These records usually do not contain analyzable information and may interfere with the accuracy of data analysis.
[0046] After being processed by the data cleaning module, a batch of high-quality data sets that have been corrected and filtered will be obtained. These data sets eliminate the noise data in the original data and ensure the accuracy and consistency of the data.
[0047] Data compression module: To reduce the resources consumed by data storage, the data set output by the data cleaning module is sampled and compressed according to the time slice set by the user. However, there may be conflicts in which users are in different areas or have multiple network behaviors in the same time slice. Therefore, each behavior generated by the user can be called an activity, and the concept of information entropy can be used to calculate its activity entropy. Activity entropy can be calculated using the following formula:
[0048] H(X)=-∑P(x)*log2(P(x))
[0049] Where H(X) is the entropy of activity X and P(x) is the probability of activity X occurring.
[0050] The lower the activity entropy of an activity, the more representative the activity is in the time slice, and the activity is selected as the behavior generated by the user in the time slice to eliminate conflicts.
[0051] refer to Figure 3 , the position information sequence and network behavior information sequence in the preprocessed data set are cut and masked, the position information sequence is offset and simplified, and the behavior information sequence is transformed and masked. These six methods are used to enhance the data, so that the original data set of size N is expanded to 7N. Among them:
[0052] Based on the location information sequence and the network behavior information sequence, a new sequence is created: user trajectory T. The user trajectory T is a list type. If a user has |T| trajectory points, it is represented as T = {p1, p2, ..., p |T|}, where p i is the i-th trajectory point, and its structure is {lng i ,lat i ,behavior i ,time i}, where lng i Longitude of the floating point record for the ith action, lat i The latitude of the floating point record for the i-th behavior, behavior i The network behavior recorded for the i-th behavior string type, time iThe occurrence timestamp of the i-th integer type record, lng i and lat i Collectively referred to as location information.
[0053] 1) Trajectory cutting: Trajectory cutting removes the prefix and / or suffix from a given user trajectory T. The prefix of the trajectory refers to the initial part of the trajectory sequence, starting from the starting point and ending at a certain middle point; the suffix refers to the end part of the trajectory sequence, starting from a certain middle point and ending at the end point. The rest is retained as a variant. It aims to reveal partially overlapping trajectories for applications such as carpooling and partial companionship. Its formal definition is:
[0054]
[0055] Among them, i is a random number in [1,|T|), and k is a random number in (i,|T|-i).
[0056] 2) Trajectory masking: Trajectory masking randomly masks certain trajectory points from a given user trajectory T to generate variants To help learn similar trajectories with different sampling rates or incomplete records. Its formal definition is:
[0057]
[0058] Among them, i is one of the k points randomly selected
[0059] 3) Stay point position offset: The stay point position offset is randomly p from the given user trajectory T i ∈T, add an offset to the geolocation coordinates to generate the variant It aims to learn similar trajectories with small point position differences. Its formal definition is:
[0060]
[0061] lng i =lng i +ε1
[0062] lat i =lat i +ε2
[0063] Among them, ε1 and ε2 are offsets in [-0.5, 0.5].
[0064] 4) Stay point simplification: Stay point simplification removes points that are not important to the overall shape of T from a given user trajectory T to form a variant The purpose is to guide the trajectory encoder to focus on the key points of the trajectory. This method uses the DP algorithm (Douglas-Peuckeralgorithm) to judge the importance of trajectory points and perform simplified operations. Its formal definition is:
[0065]
[0066] 5) Behavior transformation: Behavior transformation finds similar behaviors for network behavior and location information from a given user trajectory T and replaces them. For network behavior, the same type of APP is replaced through the open source domain name-type knowledge base; for location information, the user's location type is obtained using the API provided by the map service provider, and randomly replaced with nearby (within 500 meters) location coordinates of the same type to generate variants. Aims to learn the importance of similar behavior information. Its formal definition is:
[0067]
[0068] behavior i =behavior j ,i≠j
[0069] lng i =lng k
[0070] lat i =lat k
[0071] Among them, behavior j Is related to behavior i The same type of online behavior, (lng k ,lat k ) is located at (lng i ,lat i )'s coordinates for nearby places of the same type.
[0072] 6) Behavior masking: Behavior masking masks the network behavior of certain points from a given user trajectory T to generate variants It aims to deal with similar behavior trajectories in the absence of network behavior. Its formal definition is:
[0073]
[0074] where p i `The structure is {lng i ,lat i ,None,time i}.
[0075] The present invention creates two types of features for each trajectory point in the user's trajectory, namely, behavior sequence features and spatial structure features, and uses an embedding vector to represent each feature, namely, a spatial structure feature embedding vector, a behavior sequence feature embedding vector, and adds a social relationship embedding vector. The behavior sequence features include location behavior features and usage behavior features. Due to the model used in the present invention, which improves efficiency through parallel computing, the input embedding vector lacks the front-to-back order feature, so it is necessary to construct a position code to reflect the time series feature. Among them:
[0076] 1) Behavior sequence feature embedding: By calculating the longitude and latitude in the trajectory, the fine-grained location information of the trajectory points in the user trajectory is further captured as the location behavior feature. The network behavior of the user at a certain location in the user trajectory is captured, including the access to the APP or website and the specific website content, as the usage behavior feature. All behavior features are trained through the word embedding model word2vec to obtain a multi-dimensional word vector set to obtain the behavior sequence feature embedding vector. Given a point p in the user trajectory T i , whose behavior sequence feature is a five-tuple (x i ,y i ,d i ,b i ,t i ), where x i and i is its position coordinate, d i is the distance between two locations, b i For network behavior, t i is the timestamp. To represent the behavior sequence feature embedding vector, the feature embedding dimension D is set to 4+D b .
[0077] 2) Spatial structure feature embedding: Use geographic coding to create a spatial structure map G g = {V, E, A}, where node V is the number of a known location, and edge E represents the connection between two nodes v i and v j The distance between two nodes is within the set threshold. A represents the attribute information of the edge between two nodes, including distance and angle spatial structural features. The self-supervised graph embedding algorithm node2vec is used to learn the structural information contained in the structural graph around a specific node to obtain the spatial structural feature embedding vector D S In the present invention, it is set to 256.
[0078] 3) Social relationship embedding: The user relationship is abstracted into a directed graph G = (V, E), where V is the user ID from the social relationship information sequence and E is the social relationship between users. The self-supervised graph embedding algorithm node2vec is used to learn the structural information contained in the directed graph to obtain the social relationship embedding vector U.
[0079] 4) Input to the self-supervised contrastive learning training model: Since the embedding of the Transformer structure does not contain time information, it cannot reflect the relationship between the spatial structure features and the behavior sequence features, so position encoding is required. This method uses sine and cosine functions to positionally encode the behavior sequence feature embedding vector and the spatial structure feature embedding vector. For the j-th dimension of the i-th point on the user trajectory T, S[i,j] and B[i,j] are used to represent it. i,j The formula for conversion is:
[0080]
[0081] Among them, d p Indicates the dimension size of feature embedding S or B. Both the behavior sequence feature embedding vector B and the spatial structure feature embedding vector S need to be encoded by superimposed position to obtain the final vectors B' and S' input to the self-attention encoder module. Figure 4 , Self-attention encoder module: Based on the Encoder part of the Transformer model, a dual-feature self-attention encoding is proposed to simultaneously capture the spatial structure features and behavior sequence features of the input trajectory points. The behavior sequence feature embedding vector B' and the spatial structure feature embedding vector S' are used as inputs, respectively, through a multi-head self-attention module, and the output is connected through residual connections and layer normalization to slow down model degradation. For the intermediate vector C S The intermediate vector H can be directly obtained S ; For the intermediate vector C B , and the social relationship embedding vector U is added to form H B , and then calculate the joint attention by learning the intermediate vector output by the multi-head self-attention module through cross-attention. The main processing process is:
[0082] C S =LayerNorm(S`+Dropout(MSM(S`)))
[0083] C B =LayerNorm(B`+Dropout(MSM(B`)))
[0084] H S =LayerNorm(C S +Dropout(MLP(CS )))
[0085] H B =LayerNorm(C B +Dropout(MLP(C B )))+MLP(U)
[0086] Among them, LayerNorm means layer normalization; Dropout means random dropout; MSM means multi-head self-attention module; MLP means multi-layer perceptron.
[0087] Generate H through multiple self-attention layers S and H B The representation vector, then let Q = H B W Q , K=V=H S W k , perform cross attention calculation to obtain the user representation vector H:
[0088]
[0089] in, H K The output parameters to be set.
[0090] Afterwards, the output layer is used to obtain the prediction result of the user's next position for the user representation vector H generated by the model:
[0091]
[0092] Use cross entropy as the loss function for its predicted location:
[0093]
[0094] Using the data-enhanced dataset, the user trajectory T of a user u is u and a random enhanced variant As a pair of positive samples, T u T with different users v and The user trajectory T u The calculated user representation vector is H, The calculated user representation vector is H`, and the training representation vector obtained by negative sample j is The loss function uses cosine distance to maximize the similarity of positive samples and minimize the similarity of negative samples. The loss function is defined as follows:
[0095]
[0096] Among them, the loss function is a positive sample and Q neg Negative samples are used to optimize the model, and more negative samples help produce more robust embeddings.
[0097] In order to improve the accuracy of user location prediction by using the behavior sequence feature embedding vector B and the spatial structure feature embedding vector S, this method adopts a multi-task learning strategy. By jointly optimizing the location prediction task and the self-supervised contrastive learning task, the final loss function is designed as:
[0098]
[0099] The specific embodiments of the present invention disclosed above are intended to help understand the content of the present invention and implement it accordingly. It can be understood by those skilled in the art that various replacements, changes and modifications are possible without departing from the spirit and scope of the present invention. The present invention should not be limited to the contents disclosed in the embodiments of this specification, and the scope of protection of the present invention shall be subject to the scope defined in the claims.
Claims
1. A user location prediction method based on multi-dimensional historical behavior trajectory data, comprising: Obtain historical behavior trajectory data and perform data access, data cleaning and data compression to obtain a preprocessed data set; The data access is used to integrate the historical behavior trajectory data into a data set form; The data cleaning is used to eliminate noise data; The data compression is used to reduce the resources consumed when storing data; Perform data enhancement based on the preprocessed data set to obtain a training data set; The data enhancement is used to expand the trainable content; The training data set includes a user data set and a data training data set; Based on each trajectory point in the training data set, a behavior sequence feature embedding vector and a spatial structure feature embedding vector are established, and based on the social relationship information sequence, a social relationship embedding vector is established; The behavior sequence feature embedding vector and the spatial structure feature embedding vector are input into a self-supervised contrastive learning training model to obtain a user location prediction location; wherein the self-supervised contrastive training model is established by the following steps: Position encoding is superimposed on the behavior sequence feature embedding vector and the spatial structure feature embedding vector, and combined with the social relationship embedding vector to obtain a user representation vector and a training representation vector; The user representation vector is passed through the output layer to obtain the predicted location of the user's position.
2. The method according to claim 1, characterized in that The historical behavior trajectory data includes: Location information sequence, network behavior information sequence, social relationship information sequence; the location information sequence includes location coordinates and timestamp; the network behavior information sequence includes access to mobile phone software or websites and their specific content; the social relationship information sequence includes user relationships and interaction records.
3. The method according to claim 1, characterized in that The data access method comprises: The historical behavior trajectory data is filtered and integrated into a data set.
4. The method according to claim 1, characterized in that: The data cleaning method comprises: Handle positioning errors and filter meaningless heartbeat connection signals.
5. The method according to claim 3, characterized in that: The data compression method comprises: The data set is sampled and compressed, and possible data conflicts are processed to obtain a preprocessed data set.
6. The method according to claim 1, characterized in that The data enhancement method includes: A new sequence user trajectory is created based on the preprocessed dataset, which is represented as T = {p1, p2, ..., p |T| }, where p i is the i-th trajectory point, and its structure is {lng i ,lat i ,behavior i ,time i }; The user trajectory is subjected to trajectory clipping, / trajectory masking, / stay point position shifting, / stay point simplification, / behavior transformation, / behavior masking, and the data is integrated into a training data set.
7. The method according to claim 1, characterized in that The method of establishing a behavior sequence feature embedding vector and a spatial structure feature embedding vector based on each trajectory point in the training data set, and establishing a social relationship embedding vector based on the social relationship information sequence includes: Behavior sequence feature embedding: Based on the longitude and latitude from the location information of each trajectory point in the training dataset, network behavior, the distance between the current trajectory point and the previous trajectory point, and the timestamp, a five-tuple is established as the behavior sequence feature of each trajectory point; all behavior sequence features are integrated into a behavior sequence feature embedding vector; Spatial structure feature embedding: Based on the longitude and latitude of each trajectory point in the training set, the number of the known location is obtained as a node; a spatial structure graph is established based on the node, and the spatial structure feature embedding vector is obtained through the embedding algorithm; Social relationship embedding: A directed graph is obtained based on the user ID and user relationship in the training dataset, and the social relationship embedding vector is obtained through the embedding algorithm.
8. The method according to claim 1, characterized in that The method for obtaining the user representation vector and the training representation vector comprises: The behavior sequence feature embedding vector and the spatial structure feature embedding vector are superimposed and positionally encoded to obtain two new vectors; the two new vectors are used as input and passed through a multi-head self-attention module to obtain two intermediate vectors; the intermediate vector is fused with the social relationship embedding vector and then the representation vector of the intermediate vector is generated through the self-attention layer, thereby obtaining the user representation vector and the data training representation vector.
9. An electronic device comprising a memory and a processor, wherein the memory stores a computer program, the computer program is configured to be executed by the processor, and the computer program comprises instructions for executing the method according to any one of claims 1 to 8.
10. A storage medium storing a computer program, wherein when the computer program is executed by a computer, the method according to any one of claims 1 to 8 is implemented.