Next Interest Point Recommendation Method Based on Spatiotemporal Power Law Attention

Abstract

The present invention discloses a method for next point-of-interest recommendation based on spatio-temporal power-law attention, which relates to the field of recommendation systems. First, the present invention captures the short-term preferences of the user check-in sequence; calculates the power-law distribution of time intervals as the correlation degree between two check-ins; calculates the power-law distribution of geographical distances and uses its attenuation to measure the weight between historical check-ins and the current check-in; calculates the spatio-temporal power-law attention and uses it to determine the degree of influence of the previous check-in states on the current state; represents the user u according to the representation of the points of interest visited by the user and their visit frequencies; combines the short-term preferences, long-term preferences and user representation, and predicts the next point of interest through a neural network. The beneficial effects are as follows: By using the power-law attenuation properties of the time intervals and geographical distances between check-ins to propose spatio-temporal power-law attention to model the long-term preferences of users, and considering the spatio-temporal relationship between non-consecutive check-ins in the modeling, the effect of next point-of-interest recommendation is improved.

Description

Technical Field

[0001] The present invention relates to the field of recommendation systems, and particularly to a method for recommending the next point of interest based on spatio-temporal power-law attention. Background Art

[0002] In recent years, with the rapid development of mobile Internet and global positioning system technologies and the widespread use of smart phones, location-based social networks (LBSNs) have grown increasingly large. The LBSNs platform provides a check-in function, allowing people to share their real-time locations with friends by checking in at locations and posting comments related to the locations for interaction. This has gradually become a popular social way. Usually, there are tens of thousands of locations near a user. How to obtain the locations that a user is interested in, that is, points of interest, from the massive locations has become an issue that LBSNs focuses on. Point-of-interest recommendation is one of the core functions of LBSNs, aiming to personalized recommend suitable points of interest for users from the massive locations by using the users' historical check-in records and multi-modal information.

[0003] As one of the most core issues in the field of point-of-interest recommendation, the recommendation of the next point of interest plays a crucial role in people's lives. The recommendation of the next point of interest mainly captures the transfer patterns of users by mining the users' historical check-in records and information such as time and space, and foresees the points of interest that the users will visit next. The recommendation of the next point of interest can be widely applied in various fields, such as intelligent transportation, urban planning, advertising targeted delivery, and smart tourism. By predicting the next point of interest that people will visit, the government can design more reasonable traffic planning and scheduling strategies to alleviate traffic jams and handle crowd gatherings; ride-hailing and carpooling platforms such as Didi also strongly rely on accurate prediction of the next point of interest to better estimate the travel plans of users and dispatch resources accordingly to meet the needs of users; food delivery and navigation software such as Amap also require accurate prediction technology of the next point of interest to estimate the expected arrival time, helping food delivery riders or users effectively avoid congested sections and plan their trips in advance; merchant information and coupons can be accurately sent to target users who may visit, achieving targeted advertising delivery, improving the user experience, enhancing the pertinence of merchant advertising, and saving advertising operation costs. The recommendation of the next point of interest has broad application prospects, thus attracting a research boom in the industrial and academic fields.

[0004] Since the user's check-in records can be regarded as sequences, the next POI recommendation can essentially be seen as a sequence prediction problem. Recurrent neural networks (RNNs) and their variants such as LSTM and GRU (collectively referred to as RNNs below) have been successfully used in sequence-related tasks such as language modeling, and are therefore often used in the next POI recommendation task to capture the sequential patterns in the check-in sequences. To consider spatio-temporal factors in RNNs, the time interval and geographical distance between two adjacent check-ins are used as additional inputs to RNNs. However, since the spatio-temporal effects between non-consecutive check-ins are also helpful for predicting the next POI, using only the time interval and geographical distance between two consecutive check-ins as inputs to RNNs oversimplifies the temporal and spatial patterns of human mobility and cannot fully capture the higher-order spatio-temporal patterns. Summary of the Invention

[0005] In view of the above-mentioned defects of the prior art, the technical problem to be solved by the present invention is to provide a next POI recommendation method based on spatio-temporal power-law attention, which is used to capture the spatio-temporal effects between non-consecutive check-ins in the next POI recommendation task, so as to capture the higher-order spatio-temporal patterns of human mobility and improve the effect of the next POI recommendation.

[0006] To achieve the above object, the present invention provides a next POI recommendation method based on spatio-temporal power-law attention, including the following steps:

[0007] S1. Use a recurrent neural network to capture the short-term preference h of the user's check-in sequence t ;

[0008] S2. Use the power-law distribution of the time interval as the correlation degree regarding time between the i-th and t-th check-ins to model the influence of time factors on the next POI recommendation:

[0009]

[0010] where ΔT it represents the time interval between the i-th and t-th check-in POIs, and a 1 and λ 1 are the parameters of the power-law distribution;

[0011] S3. Use the power-law decay of the distance to measure the weight between the i-th and t-th check-ins according to the geographical distance:

[0012]

[0013] where ΔD it represents the geographical distance between the i-th and t-th check-in POIs, and a 2 and λ 2 are the parameters of the power-law distribution;

[0014] S4. Calculate the spatio-temporal power-law attention, and use the spatio-temporal power-law attention to determine the influence degree of previous check-in states on the current state, so as to model the long-term preferences of users. According to the and calculate the weight coefficient α it , and combine the short-term preferences to calculate the long-term preferences

[0015]

[0016]

[0017] Among them, h i ∈R d represents the hidden state of the i-th check-in, R represents the real number field, d is the dimension size of the embedded representation, and the long-term preference is expressed as the weighting of the hidden states of check-ins from 0 to t, and the weight coefficient α it reflects the correlation degree between h i and h t ;

[0018] S5. Characterize the user u according to the representations of the points of interest visited by the user and their visit frequencies:

[0019]

[0020] Among them, B(u) is the set of points of interest visited by user u, x j is the vectorized representation of the j-th point of interest, and n j represents the number of check-ins of user u at the j-th point of interest;

[0021] S6. Combine the short-term preference h t , the long-term preference and the user representation p u , and predict the next point of interest through the neural network shown in the following formula

[0022]

[0023] Among them, W p is the neural network parameter, and γ 1 is a hyperparameter used to control the proportion of the user representation.

[0024] Recommend the predicted to the user.

[0025] Preferably, the short-term preference h that captures the user's check-in sequence t, implemented by a recurrent neural network, with GRU as the basic recurrent unit, and the basic update formula of GRU is:

[0026] r t =σ 1 (W r ·[h t-1 ,x t )

[0027] z t =σ 1 (W z ·[h t-1 ,x t )

[0028]

[0029]

[0030] Among them, x t ∈R d represents the embedded representation of the point of interest checked in at the t-th time, r t represents the reset gate, which is used to control how much information of the hidden state of the (t - 1)-th check-in is retained in the candidate hidden state of the t-th check-in, z t represents the update gate, which is used to control the selective forgetting of the hidden state of the (t - 1)-th check-in, [,] represents the concatenation of two vectors, · represents matrix multiplication, * represents element-wise multiplication, σ 1 represents the sigmoid activation function, tanh represents the hyperbolic tangent function, W r , W z and W h are the parameters that GRU needs to learn. represents the candidate hidden state of the t-th check-in, h t ∈R d represents the hidden state of the t-th check-in, that is, the short-term preference.

[0031] Preferably, after capturing the short-term preference h t and the long-term preference in the user check-in sequence, using the multi-level category information of the point of interest, an auxiliary task based on the prediction of the multi-level category of the point of interest is designed to alleviate the data sparsity problem.

[0032] Preferably, the prediction based on the multi-level category of the point of interest is expressed as:

[0033]

[0034] Among them, σ 2 represents the softplus activation function, k represents the level of the category, W k and b krespectively represent the weight matrix and bias of the k-th layer, is the output of the k-th layer, d k is the number of categories included in the k-th layer. The cross-entropy is used to calculate the prediction loss of each layer's categories:

[0035]

[0036] where N represents the maximum length of each user's check-in trajectory, and the [1, N - 1] check-ins of each user are taken as the training set. represents the category label of the k-th layer, and T represents the transpose operation.

[0037] Preferably, when training the neural network model, the loss function of the model is set includes two parts, namely the cross-entropy loss of the main task interest point prediction and the cross-entropy loss of the multi-level category prediction of the auxiliary task:

[0038]

[0039] where represents the cross-entropy loss of the main task, represents the cross-entropy loss of the auxiliary task, y t is the label of the user's t-th check-in, W p is the parameter to be learned. γ 1 and γ 2 are hyperparameters, which are respectively used to control the proportion of user representation and balance the weights of the main task and the auxiliary task loss functions of interest point prediction. The loss function is the loss of one user. During the model training process, the average of the user losses in a batch is taken as the final loss function.

[0040] Preferably, before capturing the short-term preference of the user check-in sequence pattern, the data of the user check-in sequence is preprocessed to filter out inactive users and / or inactive interest points.

[0041] Preferably, the filtering of inactive users is to delete users with less than 5 check-ins.

[0042] Preferably, the filtering of inactive interest points is to delete interest points with less than 5 check-ins.

[0043] In addition, the present invention also provides an electronic device, including:

[0044] at least one processor; and,

[0045] A memory communicatively connected to the at least one processor; wherein, the memory stores instructions executable by the at least one processor, and when the instructions are executed by the at least one processor, the at least one processor is enabled to execute the above method for next point-of-interest recommendation based on spatio-temporal power-law attention.

[0046] In addition, the present invention also provides a non-transitory computer-readable storage medium storing computer instructions for causing a computer to execute the above method for next point-of-interest recommendation based on spatio-temporal power-law attention.

[0047] Technical effects

[0048] 1. For the problem of next point-of-interest recommendation in a social network based on geographical location information, the present invention utilizes the power-law decay property of the time interval and geographical distance between each user's check-ins to propose spatio-temporal power-law attention to model the long-term preferences of users, and considers the spatio-temporal relationship between non-consecutive check-ins in the modeling of users' long-term preferences, thereby improving the effect of next point-of-interest recommendation;

[0049] 2. Further, from the perspective of designing an auxiliary task for multi-level category prediction of points of interest, the present invention utilizes multi-level category information to alleviate the data sparsity problem, further explores users' travel intentions, and improves the recommendation effect.

[0050] The following will further illustrate the concept, specific structure and technical effects of the present invention with reference to the accompanying drawings, so as to fully understand the purpose, features and effects of the present invention. Brief description of the drawings

[0051] Figure 1 is a schematic diagram of a single user's consecutive check-in activities in an embodiment of the present invention;

[0052] Figure 2 is a structural diagram of the PowerUP model in an embodiment of the present invention

[0053] Figure 3 is an example diagram of the geographical distance of non-consecutive check-ins in an embodiment of the present invention;

[0054] Figure 4 is a relationship diagram of the time interval and check-in frequency of each data set in a double logarithmic coordinate;

[0055] Figure 5 is an example diagram of multi-level point-of-interest categories;

[0056] Figure 6.a is an experimental result diagram for verifying the influence of sequence length on the powerUP model in NYC;

[0057] Figure 6.bExperimental result graph for verifying the influence of sequence length in TKY on the powerUP model;

[0058] Figure 7.a Experimental result graph for verifying the influence of vector dimension in NYC on each model;

[0059] Figure 7.b Experimental result graph for verifying the influence of vector dimension in TKY on the PowerUP model;

[0060] Figure 8 To verify the hyperparameters λ 1 and λ 2 on the experimental results;

[0061] Figure 9.a Visualization graph of the interest point representation of the LSTPM model;

[0062] Figure 9.b Visualization graph of the interest point representation of the TiSASRec model;

[0063] Figure 9.c Visualization graph of the interest point representation of the PowerUP model. Detailed implementation manners

[0064] The following introduces multiple preferred embodiments of the present invention with reference to the accompanying drawings of the specification, making its technical content clearer and easier to understand. The present invention can be embodied in many different forms of embodiments, and the protection scope of the present invention is not limited to the embodiments mentioned in the text.

[0065] In the embodiments of the present invention, the variables and their mathematical expressions are defined as follows: The user set in the check-in data is defined as U = {u 1 , u 2 , … u |U|}, and the total number of users is |U|; the location set in the check-in data is defined as L = {l 1 , l 2 , … l |L|}, and the total number of locations is |L|; the geographical coordinates of the interest point l i are represented by p i = (lon i , lat i ), that is, longitude and latitude. In addition, the time interval between the i-th and j-th check-ins is recorded as ΔT ij = |t i - t j |, and the geographical distance is recorded as ΔD ij = Haversine(p I , p j ), where the Haversine formula is as follows:

[0066]

[0067] Among them, R 1 represents the radius of the earth, which is 6371 km.

[0068] The next POI recommendation method implemented by the PowerUP model constructed in a preferred embodiment of the present invention is used to capture the spatio-temporal effect between non-consecutive check-ins in the next POI recommendation task, so as to capture the high-order spatio-temporal law of human movement and improve the effect of the next POI recommendation. In another preferred embodiment of the present invention, in order to alleviate the data sparsity problem, the present invention utilizes the multi-level category information of POIs and designs an auxiliary task based on the prediction of the multi-level categories of POIs, so as to further explore the travel intention of users.

[0069] In a preferred embodiment of the present invention, according to the time interval and geographical distance data between each check-in of the user, the short-term preference of the user check-in sequence is captured first. When modeling the long-term preference, it is necessary to obtain the historical state useful for the current state. Using the idea of the non-local network, the long-term preference is expressed in the form of the weighted hidden states of each check-in. The weight coefficient of each historical hidden state reflects its correlation with the current state. Since there is a strong correlation between check-ins with a small time interval and a short geographical distance, the weight coefficient needs to combine time and space factors. From the time perspective, the power-law distribution of the time interval is used as the correlation degree between two check-ins; from the space perspective, the power-law decay of the distance is used to measure the weight between the historical check-in and the current check-in. Combining time and space factors, spatio-temporal power-law attention is used to determine the influence degree of each historical check-in state on the current state, so as to model the long-term preference of the user. Finally, the cross-entropy loss function is used to measure the prediction performance of the model and train the model.

[0070] Figure 1 It is a schematic diagram of the check-in activity of a single user in the dataset in the embodiment of the present invention. Among them, P 1 , P 2 , P 3 , …, P i is the POI sequence; Δt 1 , Δt 2 , …, Δt i-1 is the time interval between two adjacent check-in records;

[0071] Δd 1 , Δd 2 , …, Δd i-1is the geographical distance between two adjacent points of interest. The original data is the user's check-in record, including user number, point of interest number, type of point of interest, longitude and latitude of the point of interest, etc. The original data set is grouped according to the user number and sorted in order of check-in time to generate each user's check-in sequence data. The present invention only considers using the user's most recent N check-in data to model the spatiotemporal information representation of the learning data and predict the next point of interest. For each user, his [2, N]th check-in is used as a test set, and his [2, N-1]th check-in is used as an input sequence to predict the Nth check-in point of interest. Figure 2 The figure is a schematic diagram of the PowerUP model structure of a preferred embodiment of the present invention.

[0072] Figure 3 This is an example of a non-continuous check-in geographical distance in an embodiment of the present invention, such as Figure 3 As shown, the interest point p of the t-1th check-in t-1 Point of interest p with t+1 check-in t+1 The distance is greater than the tth check-in location p t Distance to point of interest p t+1 According to the characteristic that points of interest with the same function in the city are often clustered, it is known that the closer the distance between the past check-in points of interest and the current points of interest is, the more helpful it is for predicting the next point of interest. This shows that p t-1 For the prediction p t+1 This is also true for time patterns. When the time interval between two non-consecutive check-ins is close to one day (or other periods, such as weeks or months), it is also helpful for predicting the next point of interest. Therefore, the time and space patterns between non-consecutive check-ins play an important role in predicting the next point of interest.

[0073] In a preferred embodiment of the present invention, a recurrent neural network is first used to capture the short-term preference j of the user's check-in sequence. t When modeling long-term preferences, the present invention uses non-local operations to learn the user's long-term preferences. Using the idea of ​​non-local networks, each position of the input signal is represented as a weighted form of all positions to model non-local long-term dependencies:

[0074]

[0075] Among them, long-term preference It is represented as the weighted sum of all the previous sign-in hidden states. Each historical hidden state j i The weight coefficient α itreflects its correlation with the current state. Generally, there is a strong correlation between check-ins with small time intervals and short geographical distances, that is, the time interval and geographical distance between check-ins reflect the correlation between them. Therefore, α it needs to consider spatio-temporal factors.

[0076] Figure 4 shows the relationship between the time interval and access frequency between consecutive visits in the two datasets of Foursuqre-NYC and Foursuqre-TKY in a double-logarithmic coordinate system. It can be seen that in the double-logarithmic coordinate system, both the time interval and access frequency of the two datasets show a linear relationship with a negative slope, from which it can be inferred that the time interval and access frequency conform to the power-law distribution.

[0077] In the embodiment, the power-law distribution of the time interval is used as the correlation degree between two check-ins to model the influence of time factors on the recommendation of the next point of interest:

[0078]

[0079] where, ΔT it represents the time interval between the i-th and t-th check-in points of interest, and a 1 and λ 1 are the parameters of the power-law distribution. is used as the correlation degree with respect to time between the i-th and t-th check-ins.

[0080] From a spatial perspective, since points of interest with the same function often show an aggregated state, such as functional areas like residential areas and commercial areas in a city. The closer a user is to such an area, the higher the degree to which their behavior can be predicted, that is, the greater the impact of this historical state on the current state. The first law of geography also states that "everything is related, and the closer the distance, the greater the correlation", and multiple studies have shown that the moving distance of humans and the check-in frequency have a scale-free property, that is, the power-law distribution can better describe this distance decay phenomenon.

[0081] When modeling long-term preferences, the PowerUP model uses the power-law decay of distance to measure the weight between historical check-ins and the current check-in:

[0082]

[0083] where, ΔD it represents the geographical distance between the i-th and t-th check-in points of interest, and a 2 and λ 2 are the parameters of the power-law distribution, and the historical state h i can be measured according to the geographical distance for the current state ht The importance

[0084] In the embodiments of the present invention, PowerUP proposes spatio-temporal power-law attention to determine the degree of influence of previous check-in states on the current state, so as to model the long-term preferences of users. PowerUP uses the softmax function to combine the time and space power-law distributions to obtain the weight coefficient α for modeling the long-term preferences of users it , and based on the spatio-temporal power-law attention mechanism for the long-term preferences of users Perform modeling. According to the power-law distribution of the time interval And the power-law decay of the distance Calculate to obtain α it , and combine the short-term preferences to calculate the long-term preferences

[0085]

[0086]

[0087] Among them, h i ∈R d Represents the hidden state of the i-th check-in, R represents the real number field, d is the dimension size of the embedded representation, and the long-term preference Is expressed as the weighting of the hidden states of check-ins from 0 to t, and the weight coefficient α it Reflects the correlation between h i And h t .

[0088] In line with the idea of "you are what you eat", the places a person often goes to can reflect their personality and living status, etc. PowerUP does not provide additional parameters for the representation of users, but uses the representations of the points of interest visited by users and their visit frequencies to characterize users, and obtains the representation p of users u :

[0089]

[0090] Among them, B(u) is the set of points of interest visited by user u, x j Is the vectorized representation of the j-th point of interest l j , and n j Represents the number of check-ins of user u at the j-th point of interest l j .

[0091] Combining the short-term preference h t , the long-term preference And the user representation p u , predict the next point of interest through the neural network shown in the following formula

[0092]

[0093] Among them, W p is the neural network parameter, and γ 1 is a hyperparameter used to control the proportion of user representation.

[0094] Obtain the next point of interest After that, it can be recommended to the user.

[0095] Regarding the problem of recommending the next point of interest in a social network based on geographical location information, the embodiments of the present invention utilize the power-law decay property of the time interval and geographical distance between each user check-in to propose spatio-temporal power-law attention to model the long-term preferences of users, consider the spatio-temporal relationship between non-consecutive check-ins in the long-term preference modeling of users, and improve the effect of recommending the next point of interest.

[0096] In another preferred embodiment of the present invention, since GRU has achieved performance similar to that of LSTM in multiple tasks and has fewer parameters and is easier to converge, the present invention is implemented through a neural network, using GRU as the basic recurrent unit to capture the short-term preference h t .

[0097] The basic update formula of GRU is:

[0098] r t =σ 1 (W r ·[h t-1 , x t )

[0099] z t =σ 1 (W z ·[h t-1 , x t )

[0100]

[0101]

[0102] Among them, x t ∈R d represents the embedded representation of the point of interest of the t-th check-in, d is the size of the embedded representation dimension, r t represents the reset gate, which is used to control how much information of the hidden state of the (t - 1)-th check-in is retained in the candidate hidden state of the t-th check-in, z t represents the update gate, which is used to control the selective forgetting of the hidden state of the (t - 1)-th check-in, [,] represents the concatenation of two vectors, · represents matrix multiplication, * represents element-wise multiplication, and σ 1denotes the sigmoid activation function, tanh denotes the hyperbolic tangent function, and W r 、W z and W h are the parameters that the GRU needs to learn. denotes the candidate hidden state at the t-th check-in. By using the reset gate r t to reset the previous data, we get r t ⊙h t-1 , and then concatenate it with x t and obtain it through a tanh activation function. h t ∈R d denotes the hidden state at the t-th check-in, that is, the short-term preference. By forgetting some information in h t-1 and adding some information of the current candidate hidden state , we get it.

[0103] GRU combines the cell state and the hidden state in LSTM and replaces the forget gate and the input gate of LSTM with an update gate z t , which is used to control the selective forgetting of the hidden state at the previous moment. The reset gate r t is used to control how much information of the hidden state at the previous moment is retained in the candidate hidden state .

[0104] In another preferred embodiment of the present invention, after capturing the short-term preference and the long-term preference in the sequence, the data sparsity problem in the next point of interest recommendation is solved. Figure 5 is a multi-level point of interest category example diagram. The Foursquare dataset provides the category information of the points of interest, and this category information presents a hierarchical structure with the granularity from fine to coarse. Each point of interest in the Foursquare dataset has at most four layers of category information. As Figure 5 shown, the category to which the point of interest "4bbe6fd54e7bd13a76029b7f" belongs, from fine to coarse, successively belongs to the categories of "College Arts Building", "College and University", "Education", and "Community and Government". The higher the level, the coarser the granularity, and the more abstract the semantic information it contains; therefore, the category information presenting a hierarchical structure can further reflect the user's true movement intention, and reasonably using the multi-level category information can provide rich semantic information for the next point of interest recommendation, thereby alleviating the data sparsity problem.

[0105] In another preferred embodiment of the present invention, a method for alleviating data sparsity is designed from the perspective of a design assistance task by using the multi-level category information of points of interest. PowerUP designs the assistance task as the multi-level category prediction of points of interest, and the formula for designing the multi-level category prediction layer is as follows:

[0106]

[0107] where σ 2 represents the softplus activation function, k represents the level of the category, W k and b k represent the weight matrix and bias of the k-th layer respectively. is the output of the k-th layer, d k is the number of categories included in the k-th layer. The cross entropy is used to calculate the prediction loss of each level of categories:

[0108]

[0109] where N represents the maximum length of the check-in trajectory of each user, and the [1, N - 1] check-ins of each user are taken as the training set. T represents the transpose operation, represents the category label of the k-th layer. By minimizing the loss of the multi-level point of interest category prediction, the representation of the points of interest learned by the model contains richer semantic information, and at the same time, the representations of the points of interest belonging to the same category are also relatively similar.

[0110] By minimizing the loss of the multi-level point of interest category prediction, the representation of the points of interest learned by the model contains richer semantic information, and at the same time, the representations of the points of interest belonging to the same category are also relatively similar. Therefore, the points of interest with fewer check-ins can learn better representations through the assistance task of the multi-level point of interest category prediction, thereby improving the effect of the next point of interest recommendation.

[0111] In another preferred embodiment of the present invention, when training the neural network model, the loss function of the PowerUP model is set The loss function contains two parts, namely the cross entropy loss of the main task point of interest prediction and the cross entropy loss of the multi-level category prediction of the assistance task:

[0112]

[0113] where, represents the cross entropy loss of the main task, represents the cross entropy loss of the assistance task, y t is the label of the user's t-th check-in, W p is the parameter to be learned. γ 1 and γ 2are hyperparameters, which are respectively used to control the proportion of user representation and balance the weights of the loss functions of the main task and the auxiliary task for predicting points of interest. By designing hyperparameters to control the weights of the loss functions of the main task and the auxiliary task, in the above formula is the loss of a user. During the model training process, the average of the user losses in a batch is taken as the final loss function.

[0114] In another preferred embodiment of the present invention, before capturing the short-term preferences of the user check-in sequence, the data of the user check-in sequence is preprocessed to filter out inactive users or inactive points of interest. In some other embodiments, inactive users and inactive points of interest can also be filtered out simultaneously. For example, users with less than 5 check-ins and / or points of interest with less than 5 check-ins are deleted.

[0115] Next, the performance of the next point of interest recommendation system based on spatio-temporal power-law attention in the embodiments of the present invention is evaluated. Two evaluation metrics, the recall rate Recall@K of top K and the mean reciprocal rank (MRR), are used to evaluate the performance of the method.

[0116] The recall rate refers to the ratio of the positive samples predicted by the model to all positive samples. In the context of next point of interest recommendation, it refers to the ratio of the labels within the top K after sorting the predicted values of the model for each point of interest in descending order. The calculation method of Recall@K is as follows:

[0117]

[0118] where K ∈ {1, 5, 10, 20}, and S label respectively represent the top K points of interest recommended by the model to the user and the points of interest actually visited by the user, that is, the labels. Obviously, in the next point of interest recommendation task, there is only one point of interest that the user will visit at the next time step, that is, |S label | = 1.

[0119] (1) MRR is the average of the reciprocals of the ranks of positive samples among all samples, which reflects the overall ranking ability of the model. The calculation method is as follows:

[0120]

[0121] where rank u represents the rank of the label of user u in the model recommendation list. The higher the rank of the label in the recommendation list, the higher the MRR value and the better the model performance.

[0122] In the embodiments of the present invention, two widely used real-world datasets, Foursquare-NYC and Foursquare-TKY, are selected for the dataset. The dataset is preprocessed. To reduce the influence of inactive users and unpopular locations, users with less than 5 check-ins and points of interest with less than 5 check-ins are deleted. In the experiment, only the most recent N visits of each user are used. The dataset is divided into a training set and a test set. Among them, the [1, N - 1] check-ins of each user are used as the training set, and the [2, N] check-ins are used as the test set. Table 1 shows the statistical information of users, points of interest, and check-ins in the dataset after preprocessing.

[0123] Table 1 Dataset Statistical Information

[0124]

[0125] The embodiments of the present invention mainly use spatio-temporal attention for the recommendation of the next point of interest. There are mainly three experiments in this experimental part: (1) A comparative experiment on the performance of the next point of interest recommendation of the model of the present invention; (2) Design an ablation experiment to verify the functions of each module of the model of the present invention; (3) Design an experiment to verify the robustness and interpretability of the model.

[0126] In the comparative experiment on the performance of the next point of interest recommendation, the PowerUP model implemented in the embodiments of the present invention is compared with the following models respectively:

[0127] (1) The TMCA model adopts an encoder-decoder structure based on LSTM, and proposes two attention mechanisms to adaptively select relevant historical check-ins and context factors.

[0128] (2) The DeepMove model proposes to use an attention mechanism to obtain relevant information from historical check-ins to model long-term user preferences, and uses RNN to model short-term preferences.

[0129] (3) The LSTPM model uses the temporal and spatial correlations between the current trajectory and historical trajectories to model long-term preferences, and uses geo-dilated RNN to capture the geographical connections between discontinuous check-ins when modeling short-term preferences.

[0130] (4) The STAN model is a next point of interest recommendation model based on Transformer. It considers the time interval and geographical distance between discontinuous check-ins, and uses linear interpolation to learn the representations of different time intervals and geographical distances.

[0131] (5) The TiSASRec model is a sequence recommendation model based on Transformer. It performs personalized processing on the time intervals of different users to obtain relative time intervals, and considers the influence of different time intervals when calculating self-attention.

[0132] For the two datasets of NYC and TKY, the dimensions of the representation of points of interest and users are both set to 100, the maximum length of the input sequence is 100, the Adam optimizer is used for optimization, the learning rate is 0.001, γ 1 is 0.2, γ 2 is set to 0.1, α 1 and α 2 are set to 1, λ 1 and λ 2 are set to 0.001. The experimental results are shown in Tables 2 and 3. The data in bold in the tables are the highest experimental results.

[0133] Table 2 Comparative Experiments on Recommendation Performance on NYC Dataset

[0134]

[0135] Table 3 Comparative Experiments on Recommendation Performance on TKY Dataset

[0136]

[0137]

[0138] On the two datasets of Foursquare-NYC and Foursquare-TKY, the model PowerUP adopted in the embodiments of the present invention is significantly higher than other models except for the Transformer-based TiSASRec in all five evaluation metrics of Recall@K and MRR. In the Foursquare-NYC dataset, the Recall@1, Recall@5 and MRR evaluation metrics of the PowerUP model are improved by 4.35%, 1.6% and 4.13% respectively compared with the optimal results in the comparative models; in the Foursquare-TKY dataset, the Recall@1, Recall@5, Recall@10 and MRR evaluation metrics of the PowerUP model are improved by 3.31%, 3.14%, 0.89% and 3.4% respectively compared with the optimal results in the comparative models. Since MRR refers to the overall ranking result of the model and the smaller the K in Recall@K (such as Recall@1 and Recall@5), the more accurately the result can indicate the accuracy of the model prediction. And the MRR, Recall@1 and Recall@5 of the PowerUP model in the two datasets are significantly higher than all comparative models, which shows that PowerUP is helpful to improve the effect of the next point of interest recommendation by using spatio-temporal power-law attention to model the long-term and short-term preferences of users and the auxiliary task of predicting based on the multi-level categories of points of interest.

[0139] To study the effectiveness of each module in the PowerUP model, the present invention sets up variants of the model for ablation experiments, including the following model variants:

[0140] (1) PowerUP-prod: A variant of the PowerUP model that explores the relationship between time intervals and geographical distances. That is, the power-law decay between the time interval and geographical distance between non-consecutive check-ins is changed from addition to multiplication, i.e.,

[0141] (2) w / o time-interval: A variant of the PowerUP model, that is, the influence brought by the power-law attention of the time interval between non-consecutive check-ins is removed.

[0142] (3) w / o dis-interval: A variant of the PowerUP model, that is, the influence brought by the power-law attention of the geographical distance between non-consecutive check-ins is removed.

[0143] (4) w / o pla: A variant of the PowerUP model, that is, the long-term preference obtained by aggregating the hidden states of multiple time steps of GRU by removing the spatio-temporal power-law attention

[0144] (5) w / o usr: A variant of the PowerUP model, that is, the frequency-based user personalized representation pu is removed.

[0145] (6) w / o A variant of the PowerUP model, that is, the prediction of the highest-level category is removed.

[0146] (7) w / o A variant of the PowerUP model, that is, the prediction of the highest-level and second-highest-level categories is removed.

[0147] (8) w / o A variant of the PowerUP model, that is, only the category of the point of interest is predicted.

[0148] (9) w / o A variant of the PowerUP model, that is, the auxiliary task based on multi-level point-of-interest category prediction is removed.

[0149] The results of the ablation experiments are shown in Tables 4 and 5.

[0150] Table 4 Results of ablation experiments on the NYC dataset

[0151]

[0152] Table 5 Results of ablation experiments on the TKY dataset

[0153]

[0154]

[0155] As can be seen from Table 4 and Table 5, in the two datasets of NYC and TKY, each module of the PowerUP model contributes to the improvement of the performance of the next POI recommendation. Among them, the long-term preference based on spatio-temporal power-law attention has the greatest impact on the next POI recommendation task. After removing the long-term preference based on spatio-temporal power-law attention, i.e., w / o pla, Recall@1 and MRR in the NYC dataset decreased by 6.06% and 9% respectively, while in the TKY dataset they decreased by 3.52% and 4.93% respectively. Even when not using the auxiliary task based on multi-level POI category prediction, i.e., w / o The evaluation indicators of the PowerUP model in the two datasets are still higher than those of other comparison algorithms except TiSASRec, and are higher than TiSASRec in terms of Recall@1 and MRR. This reflects the influence of the time interval, geographical distance between non-adjacent check-ins, and long-term preference on the next POI recommendation. In addition, the auxiliary task based on multi-level POI category prediction contributes second only to the long-term preference based on spatio-temporal power-law attention to the improvement of the experimental results, indicating that by predicting the categories at each level of the POI through the auxiliary task, the model can further perceive the user's real travel intention and alleviate the data sparsity problem, thereby improving the recommendation accuracy of the model. In addition, this paper observes that adding the spatio-temporal power-law distributions has a better experimental effect in the NYC dataset, while multiplying the spatio-temporal power-law distributions (PowerUP-prod) has a higher experimental effect than adding (PowerUP) in most indicators for the TKY dataset. This may be because the check-in records included in different datasets are from different cities, and the travel patterns of people are different due to the different urban layouts of different cities.

[0156] Some embodiments of the present invention compare the performance of PowerUP and each model under different sequence lengths and vector dimensions to verify the robustness of the system.

[0157] (1) Influence of sequence length on the experiment

[0158] Figure 6.a and Figure 6.bShows the experimental results of the PowerUP model on the Foursqure-NYC and FoursquareTKY datasets when the sequence lengths are 20, 40, 60, 80, and 100 respectively. It can be seen that the model performance does not change drastically with the decrease in sequence length, and the performance also steadily improves with the increase in sequence length in both the NYC and TKY datasets, indicating that PowerUP is relatively robust to sequence length.

[0159] (2) Influence of vector dimension on the experiment

[0160] Figure 7.a Intuitively reflects the changing trend of the MRR index of each model in NYC with the vector dimension. It can be seen that the MRR of the PowerUP model is higher than that of other models when the vector dimensions are 40, 60, and 80. Figure 7.b Shows the trend of the performance of PowerUP in the TKY dataset changing with the vector dimension. The above two figures can prove that PowerUP can maintain high performance under different vector dimensions, so PowerUP is relatively robust to dimensions.

[0161] (3) Influence of hyperparameters on the experiment

[0162] Figure 8 Shows the experimental results under different λ 1 and λ 2 Although the performance of PowerUP fluctuates slightly with the change of parameters, the fluctuation range is not large. PowerUP can maintain high performance under different hyperparameters, so PowerUP is relatively robust to hyperparameters.

[0163] Finally, in order to qualitatively evaluate the interpretability of the PowerUP model, t-SNE is used to visualize the embedded representations of the points of interest learned by the LSTPM model, TiSASRec model, and PowerUP model in the NYC dataset in three-dimensional space, as Figure 8 shown. The nodes in the figure are the results of the dimensionality reduction of the points of interest, and different colors represent different categories (the highest-level categories are used here). From Figure 9.a , 9.b , and 9.c, it can be seen that for the representations of the points of interest learned by the PowerUP model, the points of interest of the same category are closer and show a certain degree of aggregation, and there are relatively obvious boundaries between different categories. Therefore, it shows that the representations of the points of interest learned by PowerUP contain strong semantic information and have good interpretability.

[0164] In summary, the next interest point recommendation method based on spatio-temporal power-law attention proposed by the present invention is superior to other comparative experiments in terms of recommendation performance, thus proving the effectiveness of the embodiments of the present invention and enabling better application to the recommendation task of the next interest point. In addition, the ablation experiment verifies the effectiveness of the spatio-temporal power-law attention proposed by the embodiments of the present invention and the method for predicting the categories of each level of interest points through auxiliary tasks.

[0165] The preferred specific embodiments of the present invention have been described in detail above. It should be understood that those of ordinary skill in the art can make many modifications and variations based on the concept of the present invention without creative efforts. Therefore, all technical solutions that can be obtained by those skilled in the art in the technical field based on the concept of the present invention through logical analysis, reasoning or limited experiments on the basis of the prior art shall fall within the protection scope determined by the claims.

Claims

1. A method for next point - of - interest recommendation based on spatio - temporal power - law attention, characterized in that, it includes the following steps: S1. Use a recurrent neural network to capture the short-term preference h of the user's check-in sequence t ; S2. Power-law distribution of usage time intervals As the degree of correlation with time between the i-th and t-th check-ins, to model the influence of time factors on the recommendation of the next point of interest: Among them, ΔT it represents the time interval between the i-th and t-th check-in points of interest, a 1 and λ 1 are parameters of the power-law distribution; S3. Utilize the power-law decay of distance Measure the weight between the i-th check-in and the t-th check-in according to the geographical distance: Among them, ΔD it represents the geographical distance between the i-th and t-th check-in points of interest, a 2 and λ 2 are parameters of the power-law distribution; S4. Calculate the spatio-temporal power-law attention, use the spatio-temporal power-law attention to determine the degree of influence of previous check-in states on the current state, so as to model the long-term preferences of users. According to the and calculate the weight coefficient α it , and calculate the long-term preference in combination with the short-term preference where h i ∈R d represents the hidden state of the i-th check-in, R represents the real number field, d is the dimension size of the embedded representation, and the long-term preference is expressed as the weighted sum of the hidden states of the 0-th to t-th check-ins, and the weight coefficient α it reflects the correlation between h i and h t ; S5. Characterize the user u according to the representation of the points of interest visited by the user and their visit frequencies; Among them, B(u) is the set of points of interest visited by user u, and x j is the vectorized representation of the j-th point of interest, and n j represents the number of check-ins of user u at the j-th point of interest; S6. Combine the short-term preference h t , the long-term preference and the user expression p u , and predict the next point of interest through the neural network shown in the following formula Among them, W p is the neural network parameter, and γ 1 is a hyperparameter used to control the proportion of user representation.

2. The method for next point - of - interest recommendation based on spatio - temporal power - law attention according to claim 1, characterized in that, The short-term preference h for capturing the user check-in sequence t , which is implemented by a recurrent neural network, with GRU as the basic recurrent unit. The basic update formula of GRU is as follows: r t = σ 1 (W r ·[h t-1 ,x t ) z t = σ 1 (W z · [h t-1 , x t ) where x t ∈R d represents the embedded representation of the point of interest for the t-th check-in, r t represents the reset gate, which is used to control how much information of the hidden state of the (t - 1)-th check-in is retained in the candidate hidden state of the t-th check-in, z t represents the update gate, which is used to control the selective forgetting of the hidden state of the (t - 1)-th check-in. [,] represents the concatenation of two vectors, · represents matrix multiplication, * represents element-wise multiplication, σ 1 represents the sigmoid activation function, tanh represents the hyperbolic tangent function, W r 、W z and W h are the parameters to be learned by the GRU, represents the candidate hidden state of the t-th check-in, h t ∈R d represents the hidden state of the t-th check-in, that is, the short-term preference.

3. The method for next point - of - interest recommendation based on spatio - temporal power - law attention according to claim 1, characterized in that, Capture the short-term preference h in the user check-in sequence t and long-term preference After that, using the multi-level category information of points of interest, design an auxiliary task based on the prediction of multi-level categories of points of interest to alleviate the data sparsity problem.

4. The method for next point - of - interest recommendation based on spatio - temporal power - law attention according to claim 3, characterized in that, the expression of the multi - level category prediction based on points of interest is: Among them, σ 2 represents the sotfplus activation function, k represents the level of the category, W k and b k represent the weight matrix and bias of the k-th layer respectively, is the output of the k-th layer, d k is the number of categories included in the k-th layer, and the prediction loss of each layer's category is calculated using cross-entropy: Among them, N represents the maximum length of each user's check-in trajectory. The [1, N - 1] check-ins of each user are used as the training set. represents the class label of the k-th layer, and T represents the transpose operation.

5. The method for next point - of - interest recommendation based on spatio - temporal power - law attention according to claim 4, characterized in that, When training the neural network model, set the loss function of the model It includes two parts, namely the cross-entropy loss of the main task interest point prediction and the cross-entropy loss of the multi-level category prediction of the auxiliary task: Among them, represents the cross-entropy loss of the main task, represents the cross-entropy loss of the auxiliary task, y t is the label of the user's t-th check-in, γ 2 is a hyperparameter used to balance the weights of the main task and the auxiliary task loss functions for point-of-interest prediction. The loss function is the loss of a user. During model training, the average of the user losses in a batch is used as the final loss function.

6. The method for next point - of - interest recommendation based on spatio - temporal power - law attention according to claim 1, characterized in that, before capturing the short - term preferences of the user check - in sequence, pre - process the data of the user check - in sequence to filter out inactive users and / or inactive points of interest.

7. The method for next point - of - interest recommendation based on spatio - temporal power - law attention according to claim 6, characterized in that, the filtering of inactive users is to delete users with less than 5 check - in times.

8. The method for next point - of - interest recommendation based on spatio - temporal power - law attention according to claim 6 or 7, characterized in that, the filtering of inactive points of interest is to delete points of interest with less than 5 check - ins.

9. An electronic device, characterized in that, the electronic device includes: at least one processor; and, a memory communicatively connected to the at least one processor; wherein, the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to execute the method according to any one of the preceding claims 1 - 8.

10. A non - transitory computer - readable storage medium, characterized in that, the non - transitory computer - readable storage medium stores computer instructions for causing the computer to execute the method according to any one of the preceding claims 1 - 8.

Images