Next interesting point of interest recommendation system based on spatiotemporal power law attention

Abstract

The application discloses a next interest point recommendation system based on space-time power law attention, and relates to the field of recommendation system. t The application comprises a short-term preference module: a recurrent neural network is used to capture the short-term preference h t of a user check-in sequence; a long-term preference module: a time interval power law distribution, a geographical distance power law distribution and space-time power law attention are calculated; the short-term preference h t is combined with the long-term preference and user representation p u to predict a next interest point through a neural network and make a recommendation. The system uses the power law attenuation properties of the time interval and geographical distance between each check-in to model the long-term preference of a user through space-time power law attention, and considers the space-time relationship between non-continuous check-ins in modeling, thereby improving the effect of next interest point recommendation.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of recommendation systems, and in particular to a next point of interest recommendation system based on spatiotemporal power-law attention. BACKGROUND

[0002] In recent years, with the rapid development of mobile Internet and global positioning system technology and the widespread use of smart phones, location-based social networks (LBSNs) have grown rapidly. LBSNs platforms provide check-in functions, and people can share their real-time locations with friends by checking in at a location and interacting by posting comments related to the location. This has gradually become a popular way of socializing. There are usually tens of thousands of locations in the vicinity of a user, and how to obtain a location of interest from the vast number of locations has become a problem for LBSNs. Point of interest recommendation is one of the core functions of LBSNs, and aims to use a user's historical check-in records and multi-modal information to recommend suitable points of interest to the user from the vast number of locations.

[0003] As one of the most core problems in the field of point of interest recommendation, next point of interest recommendation plays a vital role in people's lives. Next point of interest recommendation mainly captures the transfer rules of users by mining their historical check-in records and information such as time and space, and predicts the point of interest that the user will visit next. Next point of interest recommendation can be widely applied in various fields, such as intelligent transportation, urban planning, targeted advertising, smart tourism, etc. By predicting the next point of interest that people will visit, the government can design more reasonable traffic planning and scheduling strategies to alleviate traffic congestion and handle crowd gathering; ride-hailing and carpooling platforms such as Didi also strongly rely on accurate next point of interest prediction to better estimate users' travel plans and adjust resources accordingly to meet user demand; food delivery and navigation software such as Meituan and Gaode also need accurate next point of interest prediction technology to estimate the estimated time of arrival, helping food delivery riders or users to effectively avoid congested road sections and plan their travel in advance; business information and coupons can be accurately distributed to target users who are likely to visit, enabling targeted advertising, improving user experience, and saving advertising operating costs. Next point of interest recommendation has broad application prospects, and has therefore attracted a research boom in the industry and academia.

[0004] Since the check-in records of a user can be regarded as a sequence, the next POI recommendation can be essentially regarded as a sequence prediction problem, and recurrent neural networks (RNNs) and their variants, such as LSTM and GRU (hereinafter collectively referred to as RNNs), have been successfully used in sequence-related tasks such as language modeling, and thus are often used in the next POI recommendation task to capture the sequence patterns in the check-in sequence. In order to consider the time-space factor in the RNNs, the time interval between the adjacent two check-ins and the geographical distance are taken as additional inputs of the RNNs. However, since the time-space effect between non-continuous check-ins is also helpful for the prediction of the next POI, only taking the time interval and the geographical distance between the continuous two check-ins as the inputs of the RNNs oversimplifies the time and space rules of human movement and cannot completely capture the high-order time and space rules. SUMMARY

[0005] In view of the above defects of the prior art, the technical problem to be solved by the present application is to provide a next POI recommendation system based on a time-space power law attention, which is used to capture the time-space effect between non-continuous check-ins in the next POI recommendation task, to capture the high-order time and space rules of human movement, and to improve the effect of the next POI recommendation.

[0006] To achieve the above-mentioned purpose, the present application provides a next POI recommendation system based on a time-space power law attention, comprising:

[0007] a short-term preference module: using a recurrent neural network to capture the short-term preference h t of a user check-in sequence;

[0008] a long-term preference module: calculating a time interval power law distribution, a geographical distance power law distribution and a time-space power law attention, and combining the short-term preference h t to calculate the long-term preference of the user;

[0009] a next POI recommendation module: combining the short-term preference h t , the long-term preference and a user representation p u , predicting the next POI through a neural network and recommending.

[0010] As a preferred, the long-term preference module comprises:

[0011] using the power law distribution of the time interval as the degree of correlation about time between the ith check-in and the tth check-in to model the influence of the time factor on the next POI recommendation:

[0012]

[0013] wherein, ΔT itrepresents the time interval between the i-th and t-th check-in interest points, a1 and λ1 are the parameters of the power-law distribution;

[0014] Utilizing the power-law decay of distance According to the geographical distance between the i-th check-in and the t-th check-in, the weight between them is measured:

[0015]

[0016] where ΔD it represents the geographical distance between the i-th and t-th check-in interest points, a2 and λ2 are the parameters of the power-law distribution;

[0017] The spatiotemporal power-law attention is calculated, the spatiotemporal power-law attention is utilized to determine the degree of influence of the previous check-in state on the current state, thereby modeling the long-term preference of the user, and according to the and The weight coefficient α it is calculated, and the long-term preference is calculated in combination with the short-term preference

[0018]

[0019]

[0020] where h i ∈ R d represents the hidden state of the i-th check-in, R represents the real number field, d is the dimension size of the embedding representation, and the long-term preference represents the weighted hidden state of the 0-th to t-th check-in, the weight coefficient α it reflects the correlation degree of h i and h t .

[0021] As preferred, the user representation p u is characterized according to the representation of the interest points visited by the user and the visit frequency thereof:

[0022]

[0023] where B(u) is the set of interest points visited by the user u, x j is the vectorized representation of the j-th interest point, n j represents the number of check-ins of the user u at the j-th interest point;

[0024] As preferred, the next interest point recommendation module predicts the next interest point through a neural network shown by the following formula:

[0025]

[0026] wherein, W p is a neural network parameter, and γ1 is a hyperparameter for controlling the proportion of user representation.

[0027] As a preference, the next interest point recommendation system further comprises a data sparsity alleviating module: capturing short-term preferences h t and long-term preferences After that, an auxiliary task based on interest point multi-level category prediction is designed using interest point multi-level category information to alleviate the data sparsity problem.

[0028] As a preference, when the neural network model is trained, the loss function of the model is set as which contains two parts, namely, the cross-entropy loss of the main task interest point prediction and the cross-entropy loss of the multi-level category prediction of the auxiliary task.

[0029] As a preference, the next interest point recommendation system further comprises a preprocessing module: preprocessing the data of the user check-in sequence.

[0030] As a preference, the preprocessing of the data of the user check-in sequence comprises filtering inactive users and / or inactive interest points.

[0031] Technical effects

[0032] 1. The present application aims at the next interest point recommendation problem in the social network based on geographic location information, uses the time interval between each check-in of the user and the power-law decay property of the geographic distance to propose a space-time power-law attention to model the long-term preference of the user, considers the space-time relationship between non-continuous check-ins in the modeling of the long-term preference of the user, and improves the effect of the next interest point recommendation.

[0033] 2. The present application further starts from the perspective of designing an auxiliary task of interest point multi-level category prediction, uses multi-level category information to alleviate the data sparsity problem, further excavates the travel intention of the user, and improves the recommendation effect.

[0034] The concept, specific structure and technical effects of the present application will be further described below in combination with the drawings to fully understand the purpose, features and effects of the present application. DETAILED DESCRIPTION

[0035] Figure 1 is a structure diagram of the next interest point recommendation system in the embodiment of the present application;

[0036] Figure 2 is a schematic diagram of continuous check-in activities of a single user in the embodiment of the present application;

[0037] Figure 3 is a structure diagram of the PowerUP model in the embodiment of the present application

[0038] Figure 4 This is an example diagram of the geographical distance of non-continuous check-ins in an embodiment of the present invention;

[0039] Figure 5 This is a graph showing the relationship between the time interval of each dataset and the check-in frequency on a double logarithmic coordinate system.

[0040] Figure 6 Example diagram of multi-level point of interest categories;

[0041] Figure 7.a The experimental results are shown in the figure to verify the effect of sequence length on the powerUP model in NYC.

[0042] Figure 7.b The experimental results are shown in the figure to verify the effect of sequence length on the powerUP model in TKY.

[0043] Figure 8.a The experimental results are shown in the figure to verify the impact of vector dimension on each model in NYC.

[0044] Figure 8.b The experimental results are shown in the figure to verify the impact of vector dimension in TKY on the PowerUP model;

[0045] Figure 9 To verify the influence of hyperparameters λ1 and λ2 on the experimental results;

[0046] Figure 10.a A visualization of the interest point representation in the LSTPM model;

[0047] Figure 10.b A visualization of the interest point representation in the TiSASRec model;

[0048] Figure 10.c A visualization of interest points in the PowerUP model. Detailed Implementation

[0049] The following description, with reference to the accompanying drawings, illustrates several preferred embodiments of the present invention to make its technical content clearer and easier to understand. The present invention can be embodied in many different forms, and the scope of protection of the present invention is not limited to the embodiments mentioned herein.

[0050] In this embodiment of the invention, the variables and their mathematical expressions are defined as follows: U = {u1, u2, ... u} |U| Let L = {l} define the set of users in the check-in data, with a total number of users of |U|; l ,l2,…l |L| Let} define the set of locations in the check-in data, with a total number of locations of |L|; use p i =(lon) i ,lati ) represents the geographic coordinates, i.e., latitude and longitude, of the interest point l i . In addition, the time interval between the i-th and j-th check-ins is denoted as ΔT ij = |t i -t j |, and the geographic distance is denoted as ΔD ij = Haversine (p I , p j ), where the Haversine formula is as follows:

[0051]

[0052] where R1 represents the Earth's radius, which is 6371 km.

[0053] The next interest point recommendation system constructed in one preferred embodiment of the present application is used to capture the spatio-temporal effect between non-continuous check-ins in the next interest point recommendation task, to capture the high-order spatio-temporal law of human movement, and to improve the effect of next interest point recommendation. In another preferred embodiment of the present application, in order to alleviate the data sparsity problem, the present application uses the multi-level category information of interest points, designs an auxiliary task based on interest point multi-level category prediction, and further excavates the travel intention of the user.

[0054] In one preferred embodiment of the present application, the PowerUP model constructed according to the time interval and geographic distance data between each check-in of the user first captures the short-term preference of the user check-in sequence. When modeling the long-term preference, the history state useful to the current state needs to be obtained. The long-term preference is represented in the form of weighted history check-in hidden state by using the idea of non-local network. The weight coefficient of each history hidden state reflects the correlation degree with the current state. Since there is a strong correlation between the check-ins with small time interval and short geographic distance, the weight coefficient needs to be combined with the time and space factors. From the time perspective, the power-law distribution of the time interval is used as the correlation degree between two check-ins. From the space perspective, the power-law decay of the distance is used to measure the weight between the history check-in and the current check-in. By comprehensively considering the time and space factors, the spatio-temporal power-law attention is used to determine the influence degree of the history check-in state on the current state, so as to model the long-term preference of the user. Finally, the cross-entropy loss function is used to measure the prediction performance of the model and the model is trained.

[0055] Figure 2 is a schematic diagram of single user check-in activities in the data set in the embodiment of the present application. Wherein P1, P2, P3, …, P i is an interest point sequence; Δt1, Δt2, …, Δt i-1 is the time interval between two adjacent check-in records;

[0056] Δd1,Δd2,…,Δd i-1 The distance between two adjacent points of interest is denoted as . The original data consists of user check-in records, including user ID, point of interest ID, point of interest type, and point of interest latitude and longitude. The original dataset is grouped according to user ID and sorted chronologically by check-in time to generate check-in sequence data for each user. This invention only considers using the user's most recent N check-in records to model and learn the spatiotemporal information representation of the data and predict the next point of interest. For each user, their [2, N]th check-in is used as the test set, and their [2, N-1]th check-in is used as the input sequence to predict the point of interest for the Nth check-in. Figure 3 This is a schematic diagram of the PowerUP model structure according to a preferred embodiment of the present invention.

[0057] Figure 4 This is an example of non-continuous check-in geographical distance in an embodiment of the present invention, such as... Figure 4 As shown, the point of interest p for the (t-1)th check-in t-1 Interest point p with t+1 check-in t+1 The distance is greater than the location p of the t-th check-in. t Distance from point of interest p t+1 Based on the characteristic that points of interest with similar functions in a city often cluster together, it is known that the closer the past check-in point of interest is to the current point of interest, the more helpful it is in predicting the next point of interest. This indicates that p t-1 For predicting p t+1 This is even more advantageous. The same applies to temporal patterns; when the time interval between two non-consecutive check-ins is close to one day (or other cycles, such as weeks or months), it is helpful in predicting the next point of interest. Therefore, the spatiotemporal patterns between non-consecutive check-ins play an important role in predicting the next point of interest.

[0058] like Figure 1 As shown, in a preferred embodiment of the present invention, the next point of interest recommendation system includes: a short-term preference module, a long-term preference module, and a next point of interest recommendation module.

[0059] First, the short-term preference module uses a recurrent neural network to capture the short-term preference h of the user's check-in sequence. t Then, when modeling long-term preferences, this embodiment of the invention utilizes non-local operations to learn the user's long-term preferences. By utilizing the concept of nonlocal networks, each position of the input signal is represented as a weighted sum of all positions, thus modeling long-term nonlocal dependencies:

[0060]

[0061] Among them, long-term preferences Weighted by the historical check-in hidden state h i The weight coefficient a it Reflects the degree of correlation with the current state, and the check-in between the time interval is small, the geographical distance is short, and the correlation between the check-in is strong, that is, the time interval and the geographical distance between the check-in reflect the correlation between them. Therefore, a it Need to combine the space-time factor.

[0062] Figure 5 The relationship between the time interval between consecutive visits in the Foursuqre-NYC and Foursuqre-TKY data sets and the visit frequency in the double logarithmic coordinate system is shown. It can be seen that in the double logarithmic coordinate system, the time interval and the visit frequency of the two data sets show a linear relationship with a negative slope. It can be inferred that the time interval and the visit frequency conform to the power law distribution.

[0063] In the embodiment, the power law distribution of the time interval As the correlation between the two check-ins, to model the influence of the time factor on the next interest point recommendation:

[0064]

[0065] Where, ΔT it Indicates the time interval between the i-th and t-th check-in interest points, a1 and λ1 are the parameters of the power law distribution. As the correlation between the i-th and t-th check-in about time.

[0066] From the spatial point of view, since interest points with the same function often show a clustered state, such as residential areas, commercial areas and functional areas in cities. The closer the user is to such an area, the higher the degree of predictability of his behavior, that is, the greater the influence of this historical state on the current state. The first law of geography also points out that "everything is connected, the closer the distance, the greater the correlation", and many studies have shown that the distance of human movement and the check-in frequency have a scale-free property, that is, the power law distribution can better describe this distance decay phenomenon.

[0067] In long-term preference modeling, the PowerUP model uses the power law decay of distance To measure the weight between historical check-in and current check-in:

[0068]

[0069] Where, ΔD it Indicates the geographical distance between the i-th and t-th check-in interest points, a2 and λ2 are the parameters of the power law distribution, The historical state h i According to the geographical distance, the current state ht The importance of.

[0070] In this embodiment of the invention, PowerUP proposes a spatiotemporal power-law attention mechanism to determine the degree of influence of each previous check-in state on the current state, thereby modeling the user's long-term preferences. PowerUP uses a softmax function combined with temporal and spatial power-law distributions to obtain the weighting coefficients α for modeling the user's long-term preferences. it And based on the spatiotemporal power-law attention mechanism, it analyzes users' long-term preferences. Modeling is performed based on the power-law distribution of the time interval. Power-law decay of the distance α was calculated it And combine short-term preferences to calculate long-term preferences

[0071]

[0072]

[0073] Among them, h i ∈R d Let R represent the hidden state of the i-th check-in, R represent the real number field, d be the dimension of the embedded representation, and long-term preference be the value of the hidden state. This represents the weighted average of the hidden states of check-ins from 0 to t, with a weighting coefficient α. it Reflecting h i with h t The degree of relevance.

[0074] The next point of interest recommendation module combines the aforementioned short-term preferences h t The aforementioned long-term preferences and the user representation p u The next point of interest is predicted using the neural network shown in the following formula. And recommend:

[0075]

[0076] Among them, W p Here, γ is the neural network parameter, and γ1 is a hyperparameter used to control the proportion of user representation.

[0077] In a preferred embodiment of the present invention, consistent with the idea that "a person's character is reflected in their food," places a person frequently visits can reflect their personality and lifestyle. PowerUP does not provide additional parameters for the user's representation; instead, it uses the representations of the user's visited points of interest and their frequency of visits to characterize the user, thus obtaining a representation p of the user. u :

[0078]

[0079] where B(u) is the set of visited points of interest of user u, x j is the vectorized representation of the jth check-in point l j , n j is the number of check-ins of user u at the jth point of interest l j .

[0080] For the next point of interest recommendation problem in the social network based on geographic location information, the embodiment of the application utilizes the time interval between each check-in of a user and the power-law decay property of the geographic distance to propose a spatio-temporal power-law attention to model the long-term preference of the user, and the spatio-temporal relationship between non-continuous check-ins is considered in the modeling of the long-term preference of the user, thereby improving the effect of the next point of interest recommendation.

[0081] In another preferred embodiment of the application, since the GRU achieves similar performance as the LSTM in multiple tasks and has fewer parameters and is easier to converge, the application is implemented by using a neural network, uses the GRU as a basic recurrent unit, and captures the short-term preference h t in the check-in sequence.

[0082] The basic update formula of the GRU is:

[0083] r t =σ1(W r ·[h t-1 ,x t ])

[0084] z t =σ1(W z ·[h t-1 ,x t ])

[0085]

[0086]

[0087] where x t ∈R d is the embedded representation of the tth check-in point, d is the dimension size of the embedded representation, r t is the reset gate, which is used to control how much information of the (t-1)th check-in hidden state is retained to the tth check-in candidate hidden state, z t is the update gate, which is used to control the selective forgetting of the (t-1)th check-in hidden state, [ ] represents the concatenation of two vectors, · represents the matrix multiplication, * represents the multiplication of bit elements, σ1 represents the sigmoid activation function, tanh represents the hyperbolic tangent function, W r , W z and W h are parameters that need to be learned by the GRU. The candidate hidden state of the t-th check-in, obtained by using a reset gate r t The data before resetting is obtained r t ⊙h t-1 , and then concatenated with x t , and then passed through a tanh activation function to obtain h t ∈R d The hidden state of the t-th check-in, i.e. short-term preference, is obtained by forgetting some information in h t-1 and adding some information of the current candidate hidden state .

[0088] The GRU combines the cell state and hidden state in the LSTM, and replaces the forget gate and input gate of the LSTM with an update gate z t , which is used to control the selective forgetting of the hidden state of the previous time. The reset gate r t is used to control how much information of the hidden state of the previous time is retained in the candidate hidden state .

[0089] In another preferred embodiment of the present application, after capturing the short-term preference and long-term preference in the sequence, the data sparsity problem in the next interest point recommendation is solved. Figure 6 is a multi-level interest point category example diagram. The Foursquare dataset provides category information of interest points, and the category information presents a hierarchical structure from fine to coarse granularity. The category information of each interest point in the Foursquare dataset has at most four layers. As shown in Figure 6 , the category to which the interest point “4bbe6fd54e7bd13a76029b7f” belongs belongs to the “College Arts Building”, “College and University”, “Education” and “Community and Government” categories in order from fine to coarse granularity. The higher the level of the category, the coarser the granularity, and the more abstract the semantic information contained; therefore, the category information presenting a hierarchical structure can further reflect the real moving intention of the user, and reasonably using the multi-level category information can provide rich semantic information for the next interest point recommendation to alleviate the data sparsity problem.

[0090] In another preferred embodiment of the present application, the multi-level category information of the interest point is used to design a method for alleviating data sparsity from the perspective of designing auxiliary tasks. PowerUP designs the auxiliary task as multi-level category prediction of the interest point, and the formula for designing the multi-level category prediction layer is:

[0091]

[0092] wherein, σ 2 represents the sotfplus activation function, k represents the hierarchy of the category, W k and b k respectively represent the weight matrix and bias of the kth layer. is the output of the kth layer, d k is the number of categories contained in the kth layer. The prediction loss of each hierarchical category is calculated using cross-entropy:

[0093]

[0094] wherein, N represents the maximum length of the check-in trajectory of each user, the [1, N-1]th check-in of each user is taken as the training set, T represents the transpose operation, represents the category label of the kth layer, by minimizing the loss of the multi-level interest point category prediction, the representation of the interest point learned by the model contains richer semantic information, and the representations of the interest points belonging to the same category are also relatively similar.

[0095] By minimizing the loss of the multi-level interest point category prediction, the representation of the interest point learned by the model contains richer semantic information, and the representations of the interest points belonging to the same category are also relatively similar. Therefore, the interest points with fewer check-in times can learn better representations through the auxiliary task of multi-level interest point category prediction, thereby improving the effect of next interest point recommendation.

[0096] In another preferred embodiment of the present application, when the neural network model is trained, the loss function of the PowerUP model is set as The loss function contains two parts, which are the cross-entropy loss of the main task interest point prediction and the cross-entropy loss of the auxiliary task multi-level category prediction, respectively:

[0097]

[0098] wherein, represents the cross-entropy loss of the main task, represents the cross-entropy loss of the auxiliary task, y t is the label of the tth check-in of the user, W p is the parameter to be learned. γ1 and γ2 are hyperparameters, which are respectively used to control the proportion of the user representation and the proportion of the main task and auxiliary task loss functions of the interest point prediction, by designing the hyperparameters to control the proportion of the main task and auxiliary task loss functions, the proportion of the main task and auxiliary task loss functions in the above formula is is the loss of a user, in the model training process, the loss of a batch of users is averaged as the final loss function.

[0099] In another preferred embodiment of the present application, the next interest point recommendation system based on spatiotemporal power-law attention further comprises a preprocessing module for preprocessing data of the user check-in sequence. The data processed by the preprocessing module enters the short-term preference module and the long-term preference module for corresponding short-term preference and long-term preference modeling.

[0100] Preferably, the preprocessing module comprises filtering of inactive users or inactive interest points. In some other embodiments, both inactive users and inactive interest points can be filtered.

[0101] The performance of the next interest point recommendation system based on spatiotemporal power-law attention of the embodiments of the present application is evaluated below. Two evaluation indexes, top K recall rate Recall@K and mean reciprocal rank (MRR), are used to evaluate the performance of the method.

[0102] Recall rate refers to the ratio of positive samples predicted by the model to all positive samples. In the next interest point recommendation, it refers to the ratio of the label in the top K after the predicted values of the model for each interest point are arranged in descending order. The calculation method of Recall@K is:

[0103]

[0104] wherein K∈{1,5,10,20}, and S label respectively represent the top K interest points recommended by the model to the user and the interest points actually visited by the user, i.e., the label. Obviously, in the next interest point recommendation task, the interest point visited by the user at the next time step is only one, i.e., |S label |=1.

[0105] (1) MRR refers to the average value of the reciprocal of the ranking of positive samples in all samples, reflecting the overall ranking ability of the model. The calculation method is:

[0106]

[0107] wherein rank u represents the ranking of the label of the user u in the model recommendation list. When the label is ranked higher in the recommendation list, the MRR value is higher, and the model effect is better.

[0108] In the embodiments of the application, two widely used real-world datasets Foursquare-NYC and Foursquare-TKY are selected, and the datasets are preprocessed. In order to reduce the influence of inactive users and unpopular places, users with less than 5 check-ins and points of interest with less than 5 check-ins are deleted. In the experiment, only the latest N visits of each user are used, and the dataset is divided into a training set and a test set, wherein the [1, N-1] check-ins of each user are used as the training set, and the [2, N] check-in is used as the test set. Table 1 shows the statistical information of users, points of interest and check-ins in the preprocessed dataset.

[0109] Table 1: Dataset statistics

[0110]

[0111] The embodiments of the application mainly use spatio-temporal attention to recommend the next point of interest. The experiment part mainly has three experiments: (1) comparison experiment of next point of interest recommendation performance of the model of the application; (2) design of ablation experiment to verify the role of each module of the model of the application; (3) design of experiment to verify the robustness and explainability of the model.

[0112] In the next point of interest recommendation performance comparison experiment, the PowerUP model implemented in the embodiments of the application is compared with the following models:

[0113] (1) The TMCA model adopts an encoder-decoder structure based on LSTM, and proposes two attention mechanisms to adaptively select relevant historical check-ins and context factors.

[0114] (2) The DeepMove model proposes to use attention mechanisms to obtain relevant information from historical check-ins to model long-term user preferences, and uses RNN to model short-term preferences.

[0115] (3) The LSTPM model uses the time and spatial correlation of the current trajectory and the historical trajectory to model long-term preferences, and uses geo-dilated RNN to capture the geographical relationship between non-continuous check-ins when modeling short-term preferences.

[0116] (4) The STAN model is a next point of interest recommendation model based on Transformer, which considers the time interval and geographical distance between non-continuous check-ins, and uses linear interpolation to learn the representation of different time intervals and geographical distances.

[0117] (5) The TiSASRec model is a sequence recommendation model based on Transformer, which personalizes the time interval of different users to obtain the relative time interval, and considers the influence of different time intervals when calculating self-attention.

[0118] For both NYC and TKY datasets, the dimension of the representation of the points of interest and the users is set to 100, the maximum length of the input sequence is 100, the Adam optimizer is used for optimization, the learning rate is 0.001, γ1 is set to 0.2, γ2 is set to 0.1, α1 and α2 are set to 1, and λ1 and λ2 are set to 0.001. The experimental results are shown in Tables 2 and 3, and the black data in the table is the highest experimental result.

[0119] Table 2 Comparison of recommendation performance on NYC dataset

[0120]

[0121]

[0122] Table 3 Comparison of recommendation performance on TKY dataset

[0123]

[0124] On the Foursquare-NYC and Foursquare-TKY datasets, the model PowerUP used by the embodiments of the present application is significantly higher than other models except the TiSASRec based on Transformer in the five evaluation indicators of Recall@K and MRR. In the Foursquare-NYC dataset, the Recall@1, Recall@5 and MRR evaluation indicators of the PowerUP model are improved by 4.35%, 1.6% and 4.13% respectively compared with the optimal results of the comparison models; in the Foursquare-TKY dataset, the Recall@1, Recall@5, Recall@10 and MRR evaluation indicators of the PowerUP model are improved by 3.31%, 3.14%, 0.89% and 3.4% respectively compared with the optimal results of the comparison models. Since MRR refers to the overall ranking result of the model and the result of the smaller K in Recall@K (such as Recall@1 and Recall@5) is more indicative of the accuracy of the model prediction, and the MRR, Recall@1 and Recall@5 of the PowerUP model in the two datasets are significantly higher than all comparison models, which shows that the PowerUP helps to improve the effect of next point of interest recommendation by modeling the long-term and short-term preferences of users through the use of the spatiotemporal power law attention and the auxiliary task of point of interest multi-level category prediction.

[0125] In order to study the effectiveness of each module in the PowerUP model, the present application sets up model variants for ablation experiments, including the following model variants:

[0126] (1) PowerUP-prod: a variant of PowerUP model that explores the relationship between the time interval and the geographical distance. That is, the power-law decay of the time interval between non-consecutive check-ins and the geographical distance is changed from addition to multiplication, i.e.

[0127] (2) w / o time-interval: a variant of PowerUP model that removes the influence of the power-law attention of the time interval between non-consecutive check-ins.

[0128] (3) w / o dis-interval: a variant of PowerUP model that removes the influence of the power-law attention of the geographical distance between non-consecutive check-ins.

[0129] (4) w / o pla: a variant of PowerUP model that removes the long-term preference of the GRU hidden states aggregated by the spatio-temporal power-law attention

[0130] (5) w / o usr: a variant of PowerUP model that removes the frequency-based user personalized representation pu.

[0131] (6) w / o a variant of PowerUP model that removes the prediction of the highest-level category.

[0132] (7) w / o a variant of PowerUP model that removes the prediction of the highest-level and the second-highest-level categories.

[0133] (8) w / o a variant of PowerUP model that only predicts the category of the interest point.

[0134] (9) w / o a variant of PowerUP model that removes the auxiliary task of the multi-level interest point category prediction.

[0135] The ablation experiment results are shown in Tables 4 and 5.

[0136] Table 4 Ablation experiment results on the NYC dataset

[0137]

[0138]

[0139] Table 5 Ablation experiment results on the TKY dataset

[0140]

[0141] From Table 4, Table 5, it can be seen that in the two data sets of NYC and TKY, the performance of each module of the PowerUP model for next point of interest recommendation is improved. Among them, the long-term preference based on the spatiotemporal power law attention has the greatest impact on the next point of interest recommendation task. After removing the long-term preference based on the spatiotemporal power law attention, that is, w / o pla, the Recall@1 and MRR in the NYC dataset decrease by 6.06% and 9%, respectively, and in the TKY dataset, they decrease by 3.52% and 4.93%, respectively. Even when the auxiliary task based on the prediction of the multi-level point of interest categories is not used, that is, w / o The evaluation indexes of the PowerUP model in the two datasets are still higher than those of other comparative algorithms except TiSASRec, and are higher than TiSASRec in the Recall@1 and MRR indexes. This reflects the influence of the time interval and geographical distance between non-adjacent check-ins and the long-term preference on the next point of interest recommendation. In addition, the auxiliary task based on the prediction of the multi-level point of interest categories is only second to the long-term preference based on the spatiotemporal power law attention in improving the experimental results, which shows that by predicting the categories of each level of the point of interest through the auxiliary task, the model can further perceive the real travel intention of the user and alleviate the data sparsity problem, thereby improving the recommendation accuracy of the model. In addition, it is observed in this paper that the experimental effect of adding the time and space power law distribution in the NYC dataset is better, while for the TKY dataset, the experimental effect of multiplying the time and space power law distribution (PowerUP-prod) is higher than that of adding (PowerUP) in most indexes. This may be due to the fact that different datasets contain check-in records of different cities, and because the city layouts of different cities are different, the travel habits of people are different.

[0142] Some embodiments of the application compare the performance of PowerUP and each model under different sequence lengths and vector dimensions, and verify the robustness of the system.

[0143] (1) The influence of sequence length on the experiment

[0144] Figure 7.a and Figure 7.b It is shown that on the two datasets of Foursqure-NYC and FoursquareTKY, the experimental results of the PowerUP model when the sequence length is 20, 40, 60, 80 and 100, respectively. It can be seen that the model performance does not change dramatically with the decrease of the sequence length, and the performance is steadily improved in the two datasets of NYC and TKY with the increase of the sequence length, indicating that PowerUP is relatively robust to sequence length.

[0145] (2) The influence of vector dimension on the experiment

[0146] Figure 8.a The trend of the MRR index of each model in NYC with the change of the vector dimension is intuitively reflected. It can be seen that the MRR of the PowerUP model is higher than that of other models when the vector dimension is 40, 60 and 80. Figure 8.b The trend of the performance of PowerUP in the TKY data set with the change of the vector dimension is shown. The above two figures can prove that PowerUP can maintain high performance under different vector dimensions, so PowerUP is relatively robust to the dimension.

[0147] (3) Influence of hyperparameters on the experiment

[0148] Figure 9 The experimental results under different λ1 and λ2 are shown. Although the performance of PowerUP fluctuates slightly with the change of the parameters, the fluctuation range is not large. PowerUP can maintain high performance under different hyperparameters, so PowerUP is relatively robust to hyperparameters.

[0149] Finally, in order to qualitatively evaluate the explainability of the PowerUP model, the embedded representation of the learned interest points of the LSTPM model, the TiSASRec model and the PowerUP model in the NYC data set is visualized in a three-dimensional space by using t-SNE, as shown in Figure 9 The nodes in the figure are the results of dimension reduction of the interest points, and different colors represent different categories (here, the highest level categories are used). As can be seen from Figure 10.a , 10.b , 10.c, for the interest point representation learned by the PowerUP model, interest points of the same category are close and in a certain degree of aggregation state, and there is a relatively obvious boundary between different categories. Therefore, it is proved that the interest point representation learned by PowerUP contains strong semantic information and has good explainability.

[0150] In summary, the next interest point recommendation system based on the spatiotemporal power-law attention proposed in the present application is superior to other comparative experiments in terms of recommendation performance, thereby proving the effectiveness of the embodiments of the present application, and can be better applied to the next interest point recommendation task. In addition, the ablation experiment verifies the effectiveness of the spatiotemporal power-law attention proposed in the embodiments of the present application and the method of predicting the categories of interest points at each level through auxiliary tasks.

[0151] The preferred embodiments of the present application are described in detail above. It should be understood that those skilled in the art can make many modifications and changes to the present application without creative labor based on the concept of the present application. Therefore, any technical solution obtained by logical analysis, reasoning or limited experiment based on the prior art according to the concept of the present application shall be within the protection scope defined by the claims.

Claims

1. A next interest point recommendation system based on spatiotemporal power-law attention, characterized in that, include: Short-term preference module: Utilizes a recurrent neural network to capture the short-term preferences of user check-in sequences. ; Long-term preference module: calculates the power-law distribution of time intervals, the power-law distribution of geographical distance, and the spatiotemporal power-law attention, combined with the aforementioned short-term preferences. Calculate users' long-term preferences ; The long-term preference module includes: Power-law distribution using time intervals As the first Second and third To model the time correlation between check-ins, and to evaluate the impact of time factors on the recommendation of the next point of interest: ; in, This represents the time interval between the i-th and t-th check-ins at points of interest. and These are the parameters of the power-law distribution; Power-law decay of distance The weight between the i-th and t-th check-ins is determined based on geographical distance: ; in, This represents the geographical distance between the points of interest for the i-th and t-th check-ins. and These are the parameters of the power-law distribution; Calculate spatiotemporal power-law attention, and use it to determine the magnitude of the influence of each previous check-in state on the current state, thereby modeling the user's long-term preferences, based on the aforementioned... and Calculate the weighting coefficients And combine short-term preferences to calculate long-term preferences : ; ; in, Let R represent the hidden state of the i-th check-in, R represent the real number field, d be the dimension of the embedded representation, and long-term preference be the value of the hidden state. This represents the weighted average of the hidden states of check-ins from 0 to t, with the weight coefficients... reflect and The degree of relevance; Next point of interest recommendation module: combining the aforementioned short-term preferences The aforementioned long-term preferences and user comments Predicting the next point of interest using a neural network And recommended; It also includes a module to mitigate data sparsity: capturing short-term preferences in the user check-in sequence. and long-term preferences Then, by utilizing the multi-level category information of interest points, an auxiliary task based on multi-level category prediction of interest points is designed to alleviate the data sparsity problem. During the training of the neural network model, the loss function of the model is set. It consists of two parts: the cross-entropy loss for predicting the main task's points of interest and the cross-entropy loss for predicting the multi-level categories of the auxiliary task.

2. The next interest point recommendation system based on spatiotemporal power-law attention as described in claim 1, characterized in that, The user representation Based on the representation of user-visited points of interest and their frequency of access, it can be characterized as follows: ; Where B(u) is the set of interest points visited by user u. This is the vectorized representation of the j-th point of interest. This represents the number of times user u checks in at the j-th point of interest.

3. The next interest point recommendation system based on spatiotemporal power-law attention as described in any one of claims 1 to 2, characterized in that, The next point of interest prediction module predicts the next point of interest using the neural network shown in the following formula. : ; in, For neural network parameters, This is a hyperparameter used to control the proportion of user representation.

4. The next interest point recommendation system based on spatiotemporal power-law attention as described in claim 1, characterized in that, It also includes a preprocessing module: preprocessing the data of the user check-in sequence.

5. The next interest point recommendation system based on spatiotemporal power-law attention as described in claim 4, characterized in that, The preprocessing of user check-in sequence data includes filtering inactive users and / or inactive points of interest.

Images