A location recommendation method based on position encoder and spatiotemporal embedding

Through a method based on location encoder and space-time embedding, combining user's historical trajectory information and POI's geographical location and time information, the probability distribution of the user's next visit location is generated, and it solves the difficulty in capturing long-term behavior patterns and absolute space-time information in the prior art. The problem has significantly improved the accuracy of position prediction.

CN114741614BActive Publication Date: 2024-11-12NORTHEASTERN UNIV CHINA
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202210240378.3
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2022-03-10
Publication Date
2024-11-12
Estimated Expiration
2042-03-10

AI Technical Summary

Technical Problem

The prior art is difficult to capture the user's long-term behavior patterns and absolute spatiotemporal information when predicting human behavior trajectories, resulting in limited position prediction accuracy.

Method used

Using a position recommendation method based on position encoder and space-time embedding, relative space-time information is obtained by calculating the spatial distance and time interval between POIs, and combining the user's access time information, the self-attention network is used to generate the probability distribution of the user's next visit location. .

Benefits of technology

It significantly improves the accuracy of position prediction, makes up for the problem that traditional models are difficult to consider multi-faceted trajectory information, and improves the performance of the prediction model.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN114741614B_ABST
    Figure CN114741614B_ABST
Patent Text Reader

Abstract

The present invention provides a location recommendation method based on a position encoder and spatiotemporal embedding, which relates to the technical field of user location recommendation. The spatial geographic coordinates are encoded into vectors through the position encoder and attention, which alleviates the sparsity of the spatial geographic coordinates and strengthens the correlation between the coordinates. On the one hand, the spatiotemporal embedding self-attention model based on the position encoder replaces the traditional recurrent neural network and convolutional neural network, which can be parallel in calculation, significantly improving the training speed of the location prediction model; on the other hand, the location recommendation method based on the position encoder and spatiotemporal embedding integrates the important trajectory information such as the geographic coordinates and check-in timestamp of the POI, fully extracts the absolute spatiotemporal information and the relative spatiotemporal information, and makes up for the fact that it is difficult for traditional models to consider the impact of various trajectory information on the prediction model. Compared with traditional models, significant performance improvements have been achieved in terms of prediction accuracy.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the technical field of user location recommendation, and in particular to a location recommendation method based on a location encoder and spatiotemporal embedding. Background Art

[0002] In recent years, the rapid development of digital technology has made it easier to digitize human behavior trajectories, which has led to the popularity of location-based social networks (LBSNs) such as Foursquare and Gowalla. These datasets make it easy for researchers to predict human behavior patterns. Human mobility and flow patterns are not always predictable because they have a high degree of freedom and diversity, and capturing human trajectories is also a challenge.

[0003] As a traditional location prediction method, the main idea of ​​the Markov chain (MC) model is to represent the individual's motion trajectory as a Markov model. Then the state transition matrix is ​​calculated and the next location is predicted based on historical information. It only models short-term time series and has limited prediction capabilities. In recent years, recurrent neural networks and their variants have achieved good results in extracting long-term sequence information. STRNN uses the RNN model to combine spatiotemporal information to predict the next location. The model can automatically capture the internal representation of spatiotemporal information and model human motion patterns. The RNN model has poor ability to process long sequence data, and RNN variants (LSTM, GRU) have recently been proposed to deal with the gradient disappearance problem. The HST-LSTM model uses an encoder and a decoder and embeds spatiotemporal vectors in the LSTM to predict the next location. Attention is proposed to learn long-term correlations between sequences. DeepMove uses attention-enhanced RNN to understand the user's mobility. VANext makes full use of CNN and attention to understand the laws of historical trajectories and predict the next location.

[0004] The patent "CN111400620A User Trajectory Location Prediction Method Based on Spatiotemporal Embedding Self-Attention" calculates the spatial distance cost between all POIs and establishes spatiotemporal embedding based on the user's historical trajectory. It only considers relative spatiotemporal information and does not consider absolute spatiotemporal information. The model provided by this application obtains relative spatiotemporal information by calculating the spatial distance and time interval between POIs, obtains absolute spatiotemporal information through the location encoder and user access time information, and makes the location prediction more accurate. Summary of the invention

[0005] The technical problem to be solved by the present invention is to provide a location recommendation method based on location encoder and spatiotemporal embedding in view of the deficiencies of the above-mentioned prior art, while considering the influence of the geographical location information of POI and the time information of the user's historical visit to POI on the user's next visit behavior, and predicting the user's next visit location through the user's historical trajectory information.

[0006] The technical solution adopted by the present invention is:

[0007] A location recommendation method based on a location encoder and spatiotemporal embedding comprises the following steps:

[0008] Step 1: Perform data preprocessing and obtain the sign-in sequence of each user. The sign-in sequence is sorted one by one according to the access time.

[0009] Step 2: Filter the user trajectory information and process each user's historical check-in sequence.

[0010] Remove users whose check-in sequence is less than a, remove positions whose visit times are less than b, split the input sequence from left to right into a fixed-length sequence, and set the fixed length to L. When the input sequence length is less than L, add padding to the right; when the input sequence length is greater than L, split it;

[0011] Step 3: Map each latitude and longitude coordinate into a digital sequence, and calculate the encoding representation of the digital sequence through the sliding window and self-attention network;

[0012] Step 4: Build a spatiotemporal embedding self-attention model based on the user's historical check-in records to obtain the probability distribution of the user's next visit point;

[0013] Step 5: Input the processed user historical check-in record data into the spatiotemporal embedding self-attention model. The model gives the probability distribution of the user's next visit location and recommends locations to the user based on the top-k values ​​of the probability distribution.

[0014] The check-in includes POI identification, POI type, visit time information and POI geographic coordinate information.

[0015] The historical check-in includes POI identification, POI type, POI geographic coordinates and user visit time information.

[0016] The specific process of S3 includes the following steps:

[0017] Step 3.1: Use Mercator projection to map the entire world into a grid. Grids of different levels represent digital sequences with different granularity. Given a sequence length level m, the calculation of converting longitude and latitude coordinates into Cartesian coordinates using Mercator projection is shown in equations (1) and (2):

[0018]

[0019]

[0020] x is the Cartesian horizontal coordinate, y is the Cartesian vertical coordinate, and the geographic coordinate can be mapped to a grid; since the grid division is similar to a quadtree, all grids can be represented by a 0-3 digital sequence;

[0021] Step 3.2: Use sliding windows and sub-attention networks to further encode the digital sequence. Use a sliding window with a length of l steps and a length of s to convert the digital sequence into n sub-sequences. Enter the embedding layer to generate vectors in turn. Use the self-attention network and fully connected layer to obtain the relationship between sequences, and generate the encoding vector through the pooling layer.

[0022] The specific process of S4 includes the following steps:

[0023] Step 4.1: Embed the user's check-in record;

[0024] Step 4.2: Use the embedding representation of the check-in record to calculate the absolute spatiotemporal attention value of the user's check-in;

[0025] Step 4.3: Use the spatial distance function to calculate the spatial distance matrix between the POIs in the check-in sequence and calculate the relative spatial attention value;

[0026] Step 4.4: Calculate the visit time interval matrix of each POI in the check-in sequence and calculate the relative time attention value;

[0027] Step 4.5: Connect the absolute spatiotemporal attention value, relative spatial attention value, and relative temporal attention value of the user's sign-in sequence using a linear function to calculate the total attention value of the user's trajectory and obtain the feature matrix of the encoding of the user's first L trajectory points;

[0028] Step 4.6: Input the user trajectory representation embedded in the spatiotemporal Self-Attention into the feedforward neural network, and obtain the probability distribution of the user's next visit point through the Softmax activation function.

[0029] The specific process of S4.1 includes the following steps:

[0030] For a user check-in record of length L, use word2vec to represent the one-hot representation v of the POI identifier of the i-th POI i , the one-hot representation of POI type c i and the vector q encoded by the digital sequence iThey are respectively embedded into low-dimensional vectors, as shown in equations (3) to (5):

[0031]

[0032]

[0033]

[0034] The corresponding embedding matrices of formula (3)-formula (5) are shown in formula (6)-formula (8):

[0035]

[0036]

[0037]

[0038] Among them, |v| represents the POI identifier; |c| represents the POI type; |q| is the total number of coordinate encodings; the embedded vectors are concatenated into The access timestamp t i Encode and add to q i To distinguish the order of the sequence, as shown in formula (9) and formula (10):

[0039]

[0040]

[0041] in, Represents the vector after encoding the timestamp, d is the dimension of the encoded vector, and p i Rewrite

[0042] The embedded vector is used to represent the sequence data. The representation of the u-th user is I u ={i1, ..., i i , ..., i L}. Where i i ={p i , {lat, lon} i .t i {lat, lon} i represents the longitude and latitude coordinates of the i-th POI in the sequence, t i Indicates the timestamp of visiting the i-th POI.

[0043] The specific process of S4.2 includes the following steps:

[0044] The query, key and value are projected into the same space through the self-attention network to obtain the absolute spatiotemporal relationship of the embedded vector, as shown in Equation (11)-Equation (13):

[0045] Query = ReLU(P u W Q ) (11)

[0046] Key = ReLU(P u W K ) (12)

[0047] Value = ReLU(P u W V ) (13)

[0048] Among them, P u =[p1, ..., p L ] represents the encoding matrix of the u-th user’s check-in sequence, are the weight matrices of query, key and value respectively, d Q =d K =d V =d represents the dimension of the weight matrix; RELU represents the activation function; the absolute attention value is calculated as shown in formula (14):

[0049]

[0050] in, represents absolute spatiotemporal attention; p i It only represents the absolute time and absolute geographic coordinate information of the current POI, and does not take into account the spatiotemporal information of other check-ins. Therefore, it extracts spatiotemporal information from a local perspective.

[0051] The specific process of S4.3 includes the following steps:

[0052] Calculate the relative spatial distance between each POI and other POIs and save it in the matrix Dis;

[0053] Given two POIs as p i and p j , the distance between them dis(p i , p j ) can be calculated by formula (15):

[0054]

[0055] Among them, α i , α j Respectively represent pi and p j ; Δβ represents the difference in longitude between two POIs.

[0056] R represents the radius of the Earth;

[0057] By inputting I u To calculate Dis u , as shown in formula (16):

[0058] Dis u =Dis[I u ] (16)

[0059] Dis u The element in the i-th row and j-th column, its value represents p i and p j Distance of space; u represents the relative spatial attention matrix of user u.

[0060] The specific process of S4.4 includes the following steps:

[0061] Given a user access time series T u =[t1, ..., t L ] T , calculate the visit time interval between each POI and other POIs and save it in the matrix As shown in equations (17)-(19):

[0062]

[0063]

[0064]

[0065] in, is the user's temporal attention value matrix; MinMaxNorm(·) is a function that maps the input matrix to the [0.1] interval. T (T u ) is T u Map to function.

[0066] The specific process of S4.5 includes the following steps:

[0067] Use parameters β and γ to aggregate the user's relative temporal attention matrix and relative spatial attention matrix and connect them with the absolute spatiotemporal attention matrix, as shown in formula (20):

[0068]

[0069] S u Aggregate the absolute spatiotemporal attention information and the relative spatiotemporal attention information. Then u Use the Softmax function and perform dot product with Value, as shown in formula (21):

[0070] A u =softmax(S u )·Value (21)

[0071] Among them, A u is the feature matrix of the first L check-in codes of user u.

[0072] The specific process of S4.6 includes the following steps:

[0073] The feature matrix A of the sign-in encoding u Enter the fully connected network and softmax, and finally output the probability distribution of the next position, as shown in formula (22):

[0074]

[0075] in, Represents the probability distribution of the prediction for L+1 check-ins; the top-k locations are selected through the probability distribution to recommend locations to users.

[0076] Beneficial technical effects

[0077] The location recommendation method based on position encoder and spatiotemporal embedding provided by the present invention, through the position encoder and attention, encodes the spatial geographic coordinates into vectors, alleviates the sparsity of the spatial geographic coordinates, and grabs the correlation between the coordinates. The spatiotemporal embedding self-attention model deeply mines the user's check-in records, extracts the user's check-in feature vector, and then predicts the POI that the user will visit next time through a fully connected neural network. On the one hand, the use of the spatiotemporal embedding self-attention model based on the position encoder replaces the traditional recurrent neural network and convolutional neural network, which can be parallel in calculation, significantly improves the training speed of the location prediction model, and provides the possibility for its practical application. On the other hand, the location recommendation method based on the position encoder and spatiotemporal embedding innovatively integrates the important trajectory information such as the geographic coordinates and check-in timestamp of the POI, fully extracts the absolute spatiotemporal information and the relative spatiotemporal information, and makes up for the fact that it is difficult for traditional models to consider the impact of multi-faceted trajectory information on the prediction model. Finally, a significant performance improvement has been achieved in terms of prediction accuracy compared to previous models. BRIEF DESCRIPTION OF THE DRAWINGS

[0078] Figure 1A flow chart of a location recommendation method based on a location encoder and spatiotemporal embedding provided in an embodiment of the present invention.

[0079] Figure 2 A position encoder diagram of a position recommendation method based on a position encoder and spatiotemporal embedding provided by an embodiment of the present invention.

[0080] Figure 3 An architecture diagram of a location recommendation method based on a location encoder and spatiotemporal embedding for a location prediction method provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0081] The specific implementation of the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments;

[0082] This embodiment takes Foursquare check-in data as an example and proposes a location recommendation method based on location encoder and spatiotemporal embedding; recommends user locations, such as Figure 1 and Figure 2 As shown, the following steps are included:

[0083] Step 1: Preprocess the original check-in data and obtain the check-in sequence of each user. The check-in sequence is sorted by the visit time. Each check-in includes the POI ID, POI type, visit time and geographic coordinate information of the POI;

[0084] In this embodiment, the Foursquare check-in data set collects the daily check-in behavior of users in a city, and each record includes the user name, check-in time, check-in location, location name, location type, and location longitude and latitude coordinates and other geographic location information;

[0085] In this embodiment, the set of signed-in users is represented as U = {u1, u2, ..., u |U|}, |U| is the size of the user set, indicating the number of signed-in users, and the POI set is represented by S = {s1, s2, ..., s |S|}, |S| is the POI set size, indicating the total number of POIs;

[0086] In this embodiment, POI is a location in a coordinate system, including a location identifier ν, a location type c (restaurant, school, etc.) and geographic longitude and latitude coordinate information l;

[0087] In this embodiment, a check-in record is represented as a tuple rt u =<u,s,t> , indicating that user u visited location s at timestamp t;

[0088] User sign-in sequence is a set of user sign-ins sorted by access time. L uIndicates the length of the user's sign-in sequence, Represents the i-th check-in record in the user's check-in sequence;

[0089] In this embodiment, the location recommendation problem is: for a given user set U, POI set 5 and any user u and its trajectory The problem is to recommend to the user u +1 May visit

[0090] Step 2: Filter the user trajectory information and process each user's historical check-in sequence. Each historical check-in includes POI identification, POI type, POI geographic coordinates and user access time information;

[0091] In this embodiment, the data in the data set is filtered, and users with less than 20 historical check-in sequences are removed. Then, positions with less than 5 visits are removed, and the input user historical check-in sequence is split from left to right into a fixed-length sequence. The fixed length is set to L. When the input sequence length is less than L, padding is added to the right; when it is greater than L, it is split; where L is a configurable threshold value, which is adjusted according to actual conditions;

[0092] Step 3: Map each latitude and longitude coordinate into a digital sequence, calculate the encoded representation of the digital sequence through the sliding window and self-attention network, and use the position encoder to encode the geographic longitude and latitude coordinates;

[0093] In this embodiment, the following steps are included:

[0094] Step 3.1: First, the Mercator projection is used to map the entire world into a grid. The digital sequences represented by grids of different levels have different granularity. Each longitude and latitude coordinate can be mapped into a digital sequence of a grid. Given a sequence length level m, the calculation of converting longitude and latitude coordinates into Cartesian coordinates using the Mercator projection is as follows:

[0095]

[0096]

[0097] x is the Cartesian horizontal coordinate, y is the Cartesian vertical coordinate, and the geographic coordinate can be mapped to a grid; since the grid division is similar to a quadtree, all grids can be represented by a 0-3 digital sequence;

[0098] At the same time, grids without positions will be ignored, alleviating the data sparsity problem;

[0099] Step 3.2: Use sliding windows and sub-attention networks to further encode the digital sequence. First, use the sliding window to convert the digital sequence into n sub-sequences. Take the 8-level digital sequence 01220211 as an example. After a sliding window with a length of 5 and a step length of 1, the generated sub-sequence is 01220->12202->22021->20211, with a length of 4. Then, the sub-sequence enters the embedding layer to generate a vector at a time: use the self-attention network and the fully connected layer to obtain the association between the sequences, and then use the pooling layer to aggregate the vector representation.

[0100] In this example, the final generated vector is q i indicates; q i represents the vector obtained after the i-th POI in the user's check-in sequence is processed by the position encoder;

[0101] Step 4: Establish a spatiotemporal embedding self-attention model based on the user's historical check-in records, such as Figure 3 As shown, the probability distribution of the user's next access point is obtained. The specific method is:

[0102] Step 4.1: Embed the user's check-in record;

[0103] For a user check-in record of length L, use word2vec to represent the one-hot representation v of the POI identifier of the i-th POI i , the one-hot representation of POI type c i and the vector q encoded by the digital sequence i They are respectively embedded into low-dimensional vectors, as shown in equations (3) to (5):

[0104]

[0105]

[0106]

[0107] The corresponding embedding matrices of formula (3)-formula (5) are shown in formula (6)-formula (8):

[0108]

[0109]

[0110]

[0111] Among them, |v| represents the POI identifier; |c| represents the POI type; |q| is the total number of coordinate encodings; the embedded vectors are concatenated into The access timestamp t i Encode and add to qi To distinguish the order of the sequence, as shown in formula (9) and formula (10):

[0112]

[0113]

[0114] in, Represents the vector after encoding the timestamp, d is the dimension of the encoded vector, and p i Rewrite

[0115] The embedded vector is used to represent the sequence data. The representation of the u-th user is I u ={i1, ..., i i , ..., i L}. Where i i ={p i , {lat, lon} i .t i}; {lat, lon} i represents the longitude and latitude coordinates of the i-th POI in the sequence, t i Indicates the timestamp of visiting the i-th POI;

[0116] Step 4.2: Use the embedding representation of the check-in record to calculate the absolute spatiotemporal attention value of the user's check-in;

[0117] The query, key and value are projected into the same space through the self-attention network to obtain the absolute spatiotemporal relationship of the embedded vector, as shown in Equation (11)-Equation (13):

[0118] Query = ReLU(P u W Q ) (11)

[0119] Key = ReLU(P u W K ) (12)

[0120] Value = ReLU(P u W V ) (13)

[0121] Among them, P u =[p1, ..., p L ] represents the encoding matrix of the u-th user’s check-in sequence, are the weight matrices of query, key and value respectively, d Q =dK =d V =d represents the dimension of the weight matrix; RELU represents the activation function; the absolute attention value is calculated as shown in formula (14):

[0122]

[0123] in, represents absolute spatiotemporal attention; p i It only represents the absolute time and absolute geographic coordinate information of the current POI, and does not take into account the spatiotemporal information of other check-ins. Therefore, it extracts spatiotemporal information from a local perspective.

[0124] Step 4.3: Use the spatial distance function to calculate the spatial distance matrix between the POIs in the check-in sequence and calculate the relative spatial attention value;

[0125] Calculate the relative spatial distance between each POI and other POIs and save it in the matrix Dis;

[0126] Given two POIs as p i and p j , the distance between them dis(p i , p j ) can be calculated by formula (15):

[0127]

[0128] Among them, α i , α j Respectively represent p i and p j The latitude coordinate of the two POIs; Δβ represents the difference in longitude between the two POIs;

[0129] R represents the radius of the Earth;

[0130] By inputting I u To calculate Dis u , as shown in formula (16):

[0131] Dis u =Dis[I u ] (16)

[0132] Dis u The element in the i-th row and j-th column, its value represents p i and p j Distance of space; u represents the relative spatial attention matrix of user u;

[0133] Step 4.4: Calculate the visit time interval matrix of each POI in the check-in sequence and calculate the relative time attention value;

[0134] Given a user access time series T u =[t1, ..., t L ] T , calculate the visit time interval between each POI and other POIs and save it in the matrix As shown in formulas (17) to (19):

[0135]

[0136]

[0137]

[0138] in, is the user's temporal attention value matrix; MinMaxNorm(·) is a function that maps the input matrix to the [0.1] interval; F T (T u ) is T u Map to Function of

[0139] Step 4.5: Connect the absolute spatiotemporal attention value, relative spatial attention value, and relative temporal attention value of the user's sign-in sequence using a linear function to calculate the total attention value of the user's trajectory, and then obtain the feature matrix of the encoding of the user's first L trajectory points;

[0140] The feature matrix A of the sign-in encoding u Enter the fully connected network and softmax, and finally output the probability distribution of the next position, as shown in formula (22):

[0141]

[0142] in, Represents the probability distribution of the prediction for L+1 check-ins; selects the top-k locations through the probability distribution to recommend locations to users;

[0143] Step 4.6: Input the user trajectory representation embedded in the spatiotemporal Self-Attention into the feedforward neural network, and then use the Softmax activation function to obtain the probability distribution of the user's next visit point;

[0144] The feature matrix A of the sign-in encoding u Enter the fully connected network and softmax, and finally output the probability distribution of the next position, as shown in formula (22):

[0145]

[0146] in, Represents the probability distribution of the prediction for L+1 check-ins; selects the top-k locations through the probability distribution to recommend locations to users;

[0147] In this embodiment, the construction of the self-attention-based location recommendation model includes three parts: embedding layer, spatiotemporal self-attention layer, and prediction layer;

[0148] The embedding layer: In this example, for a user check-in record of length L, word2vec is used to represent the one-hot representation v of the POI identifier of the i-th POI i , the one-hot representation of POI type c i and the vector q processed by the position encoder i are embedded into low-dimensional vectors respectively; in order to clearly simulate the relative access time difference between locations in the trajectory, the access timestamp t i Encode and add to q i To distinguish the order of the sequence;

[0149] For a user check-in sequence of length L, the sequence data is represented by the embedded vector, and the representation of the u-th user is I u ={i1, ..., i i , ..., i L}. Where i i ={p i , {lat, lon} i .t i}; {lat, lon} i represents the longitude and latitude coordinates of the i-th POI in the sequence, t i Indicates the timestamp of visiting the i-th POI;

[0150] The spatiotemporal self-attention layer: The spatiotemporal self-attention layer is divided into absolute spatiotemporal self-attention, relative spatial self-attention layer and relative temporal self-attention layer;

[0151] Absolute spatiotemporal self-attention: The essence of self-attention can be described as a mapping from a query to a series of key-value pairs, where the key and value of self-attention are the same; the self-attention mechanism can handle longer sequences, so the self-attention mechanism is used to capture the relationship between the absolute spatiotemporal embedding vectors;

[0152] In this embodiment, the length of the input sign-in sequence is unified as L, so the input of the absolute spatiotemporal self-attention is an L×d matrix P u=[p1, ..., p L ] T , where d is p i Dimensions;

[0153] Through the weight parameter And the nonlinear activation function relu(·) calculates Query, Key and Value respectively; calculates the absolute attention value The meaning is to learn the interdependence of absolute time and space; p i It only represents the absolute time and absolute geographic coordinate information of the current POI, and does not take into account the spatiotemporal information of other check-ins. Therefore, it extracts spatiotemporal information from a local perspective.

[0154] The relative spatial attention: In this embodiment, the relative interval between the spatial coordinate information and the time information is also a kind of semantic description information; the user's movement time law and movement distance law are learned through the interval;

[0155] In this embodiment, in order to make up for the shortcomings of absolute spatiotemporal attention, we propose to learn spatial attention from a global perspective; calculate the relative spatial distance between each POI and other POIs and save it in the matrix Dis;

[0156] The relative time attention: In this embodiment, in order to capture the user's preference for time and to make up for the shortcomings of absolute spatiotemporal attention, we propose to use time intervals to learn time attention; given a user access time series T u =[t1, ..., t L ] T , calculate the visit time interval between each POI and other POIs and save it in the matrix middle;

[0157] After introducing absolute spatiotemporal attention, relative spatial attention, and relative spatial attention, the self-attention model can not only learn the information of the currently visited POI, but also learn the association information between the current POI and other POIs and the association of time intervals, so the learning ability is enhanced;

[0158] Use parameters β and γ to aggregate the user's relative time attention matrix and relative space attention matrix and connect them with the absolute space-time attention matrix.

[0159] The prediction layer: In this example, the prediction layer mainly represents the self-attention user check-in data A u The data is input into the fully connected neural network layer, and then the probability distribution of the user's next visit point is obtained through the Softmax activation function.

[0160] The feature matrix A of the sign-in encodingu Enter the fully connected network and softmax, and finally output the probability distribution of the next position;

[0161] Model parameter training: In this example, the binary cross entropy is used as the loss function to train the model; the binary cross entropy loss function contains a positive sample and a random negative sample;

[0162] Step 5: After encoding a user’s historical check-in record data, input it into the spatiotemporal embedding self-attention model. The model gives the probability distribution of the user’s next visit location, and then recommends the user’s location based on the top-k values ​​of the probability distribution.

[0163] In this embodiment, the check-in sequence data of user u is input into the model. Thus, a series of ranking scores of relationship positions are obtained, and then recommendations are made to user u from the top k positions.

Claims

1. A location recommendation method based on a location encoder and spatiotemporal embedding, characterized in that: The following steps are involved: Step 1: Perform data preprocessing and obtain the sign-in sequence of each user. The sign-in sequence is sorted one by one according to the access time. Step 2: Filter the user trajectory information and process each user's historical check-in sequence. Remove users whose check-in sequence is less than a, remove positions whose visit times are less than b, split the input sequence from left to right into a fixed-length sequence, and set the fixed length to L. When the input sequence length is less than L, add padding to the right; when the input sequence length is greater than L, split it; Step 3: Map each latitude and longitude coordinate into a digital sequence, and calculate the encoding representation of the digital sequence through the sliding window and self-attention network; Step 4: Build a spatiotemporal embedding self-attention model based on the user's historical check-in records to obtain the probability distribution of the user's next visit point; Step 4.1: Embed the user's check-in record; For a user check-in record of length L, use word2vec to represent the one-hot representation v of the POI identifier of the i-th POI i , the one-hot representation of POI type c i and the vector q encoded by the digital sequence i They are respectively embedded into low-dimensional vectors, as shown in equations (3) to (5): The corresponding embedding matrices of formula (3)-formula (5) are shown in formula (6)-formula (8): Among them, |v| represents the POI identifier; |c| represents the POI type; |q| is the total number of coordinate encodings; the embedded vectors are concatenated into The access timestamp t i Encode and add to q i To distinguish the order of the sequence, as shown in formula (9) and formula (10): in, Represents the vector after encoding the timestamp, d is the dimension of the encoded vector, and p i Rewrite The embedded vector is used to represent the sequence data. The representation of the u-th user is I u ={i1,...,i i ,...,i L }. Where i i ={p i ,{lat,lon} i .t i }; {lat,lon} i represents the longitude and latitude coordinates of the i-th POI in the sequence, t i Indicates the timestamp of visiting the i-th POI; Step 4.2: Use the embedding representation of the check-in record to calculate the absolute spatiotemporal attention value of the user's check-in; Step 4.3: Use the spatial distance function to calculate the spatial distance matrix between the POIs in the check-in sequence and calculate the relative spatial attention value; Step 4.4: Calculate the visit time interval matrix of each POI in the check-in sequence and calculate the relative time attention value; Step 4.5: Connect the absolute spatiotemporal attention value, relative spatial attention value, and relative temporal attention value of the user's sign-in sequence using a linear function to calculate the total attention value of the user's trajectory and obtain the feature matrix of the encoding of the user's first L trajectory points; Step 4.6: Input the user trajectory representation embedded in time and space Self-Attention into the feedforward neural network, and obtain the probability distribution of the user's next visit point through the Softmax activation function; Step 5: Input the processed user historical check-in record data into the spatiotemporal embedding self-attention model. The model gives the probability distribution of the user's next visit location and recommends locations to the user based on the top-k values ​​of the probability distribution.

2. The location recommendation method based on location encoder and spatiotemporal embedding according to claim 1, characterized in that: The check-in includes POI identification, POI type, access time information and POI geographic coordinate information; The historical check-in includes POI identification, POI type, POI geographic coordinates and user visit time information.

3. The location recommendation method based on location encoder and spatiotemporal embedding as claimed in claim 1, characterized in that: The specific process of S3 includes the following steps: Step 3.1: Use Mercator projection to map the entire world into a grid. Grids of different levels represent digital sequences with different granularity. Given a sequence length level m, the calculation of converting longitude and latitude coordinates into Cartesian coordinates using Mercator projection is shown in equations (1) and (2): x is the Cartesian horizontal coordinate, y is the Cartesian vertical coordinate, and the geographic coordinate can be mapped to a grid; since the grid division is similar to a quadtree, all grids can be represented by a 0-3 digital sequence; Step 3.2: Use sliding windows and sub-attention networks to further encode the digital sequence. Use a sliding window with a length of l steps and a length of s to convert the digital sequence into n sub-sequences. Enter the embedding layer to generate vectors in turn. Use the self-attention network and fully connected layer to obtain the relationship between sequences, and generate the encoding vector through the pooling layer.

4. The location recommendation method based on location encoder and spatiotemporal embedding as claimed in claim 1, characterized in that: The specific process of S4.2 includes the following steps: The query, key and value are projected into the same space through the self-attention network to obtain the absolute spatiotemporal relationship of the embedded vector, as shown in Equation (11)-Equation (13): Query=ReLU(P u IN Q ) (11) KeyReLU(P u IN K ) (12) Value=ReLU(P u W V ) (13) Among them, P u =[p1, ..., p L ] represents the encoding matrix of the u-th user’s check-in sequence, are the weight matrices of query, key and value respectively, d Q =d K =d V =d represents the dimension of the weight matrix; RELU represents the activation function; the absolute attention value is calculated as shown in formula (14): in, represents absolute spatiotemporal attention; p i It only represents the absolute time and absolute geographic coordinate information of the current POI, and does not take into account the spatiotemporal information of other check-ins. Therefore, it extracts spatiotemporal information from a local perspective.

5. The location recommendation method based on location encoder and spatiotemporal embedding as claimed in claim 1, characterized in that: The specific process of S4.3 includes the following steps: Calculate the relative spatial distance between each POI and other POIs and save it in the matrix Dis; Given two POIs as p i and p j , the distance between them dis(p i , p j ) can be calculated by formula (15): Among them, α i , α j Respectively represent p i and p j The latitude coordinate of the two POIs; Δβ represents the difference in longitude between the two POIs; R represents the radius of the Earth; By inputting I u To calculate Dis u , as shown in formula (16): Dis u =Dis[I u ] (16) Dis u The element in the i-th row and j-th column, its value represents p i and p j Distance of space; u represents the relative spatial attention matrix of user u.

6. The location recommendation method based on location encoder and spatiotemporal embedding as claimed in claim 1, characterized in that: The specific process of S4.4 includes the following steps: Given a user access time series T u =[t1, ..., t L ] T , calculate the visit time interval between each POI and other POIs and save it in the matrix As shown in equations (17)-(19): in, is the user's temporal attention value matrix; MinMaxNorm(·) is a function that maps the input matrix to the [0.1] interval; F T (T u ) is T u Map to function.

7. The location recommendation method based on location encoder and spatiotemporal embedding as claimed in claim 1, characterized in that: The specific process of S4.5 includes the following steps: Use parameters β and γ to aggregate the user's relative temporal attention matrix and relative spatial attention matrix and connect them with the absolute spatiotemporal attention matrix, as shown in formula (20): S u Aggregates absolute spatiotemporal attention information and relative spatiotemporal attention information; u Use the Softmax function and perform dot product with Value, as shown in formula (21): A u =softmax(S u )·Value (21) Among them, A u is the feature matrix of the first L check-in codes of user u.

8. The location recommendation method based on location encoder and spatiotemporal embedding as claimed in claim 1, characterized in that: The specific process of S4.6 includes the following steps: The feature matrix A of the sign-in encoding u Enter the fully connected network and softmax, and finally output the probability distribution of the next position, as shown in formula (22): in, Represents the probability distribution of the prediction for L+1 check-ins; the top-k locations are selected through the probability distribution to recommend locations to users.