City POI recommendation and timestamp prediction method based on user historical check-in sequence

Abstract

The present invention discloses a city POI recommendation and timestamp prediction method based on a user historical check-in sequence in the field of city recommendation technology. The specific steps are: collecting the user's historical check-in data, and sorting the check-in record of each user in chronological order to form a user check-in sequence, then transmitting the data to a server in real time and storing it, preprocessing the data, and then extracting a valid POI sequence and a timestamp sequence from the preprocessed user check-in sequence, inputting the POI sequence and the timestamp sequence into a constructed TAPT model, and using the efficient encoder module and embedding connection layer in the TAPT model to fuse the POI embedding with the timestamp embedding to form a joint embedding, and finally, inputting the joint embedding into a POI recommendation model and a timestamp predictor to achieve joint prediction of the POI and its corresponding timestamp, thereby overcoming the problems of the prior art in terms of user behavior modeling and time information processing.

Description

Technical Field

[0001] The present invention relates to the technical field of city recommendation, and in particular to a method for city POI recommendation and timestamp prediction based on user historical check-in sequences. Background Art

[0002] The rapid development of location-based social networks has attracted widespread attention to personalized location recommendation services. Sequential point of interest (POI) recommendation aims to predict the next POI to be visited based on the user's historical check-in sequence. As the urban environment becomes increasingly complex, higher requirements are placed on the accuracy and timeliness of recommendations. The application of sequential POI recommendation in cities has become crucial. At present, the main recommendation technologies based on deep learning are: ① Multilayer Perceptron; ② Recurrent Neural Network; ③ Convolutional Neural Network; ④ Transformer-based technology.

[0003] The Transformer model has become a mainstream technology. It effectively captures the temporal relationship in the user check-in sequence through the self-attention mechanism and accurately models long-distance dependencies, thereby improving the recommendation effect. Its core encoder-decoder structure supports parallel processing of input sequences, greatly improving computational efficiency. Transformer-based sequential POI recommendation can not only capture the user's historical preferences, but also combine complex spatiotemporal features, becoming an important technical means for urban POI recommendation.

[0004] There are two major problems with existing sequential POI recommendation methods. First, these methods usually focus on sequence correlation and ignore the specific timestamp information contained in the check-in sequence, resulting in incomplete modeling of user behavior patterns, which in turn affects the accuracy of recommendation results. Second, although the recommendation method that uses timestamps as auxiliary information introduces the time factor, it cannot accurately capture the user's daily routines and time preferences, resulting in insufficient personalization and unsatisfactory recommendation results. In short, these methods have limitations in modeling long-distance dependencies and are difficult to effectively handle complex spatiotemporal features, further weakening the accuracy and effectiveness of recommendations. Summary of the invention

[0005] The purpose of this section is to summarize some aspects of the embodiments of the present invention and briefly introduce some preferred embodiments. Some simplifications or omissions may be made in this section and the specification abstract and the invention title of this application to avoid blurring the purpose of this section, the specification abstract and the invention title, and such simplifications or omissions cannot be used to limit the scope of the present invention.

[0006] Therefore, the purpose of the present invention is to provide a city POI recommendation and timestamp prediction method based on user historical check-in sequences to solve the problems raised in the above background technology.

[0007] To solve the above technical problems, according to one aspect of the present invention, the present invention provides the following technical solutions:

[0008] The city POI recommendation and timestamp prediction method based on the user's historical check-in sequence includes the following steps:

[0009] S1. Collect the user's historical check-in data and sort each user's check-in record in chronological order to form a user check-in sequence, and then transmit the data to the server in real time and store it;

[0010] S2, preprocess the data, and then extract the valid POI sequence and timestamp sequence from the preprocessed user check-in sequence, which are expressed as and ;

[0011] S3. Input the POI sequence and timestamp sequence into the constructed TAPT model. Utilize the efficient encoder module and embedding connection layer in the TAPT model to fuse the POI embedding with the timestamp embedding to form a joint embedding. Finally, the joint embedding is input into the POI recommendation model and timestamp predictor to achieve joint prediction of POI and its corresponding timestamp.

[0012] As a preferred solution of the city POI recommendation and timestamp prediction method based on the user's historical check-in sequence described in the present invention, in step S1, a mobile application and a GPS sensor are used to regularly collect the user's historical check-in data, including each POI and its corresponding timestamp, and the data is transmitted to a server in real time and stored.

[0013] As a preferred solution of the city POI recommendation and timestamp prediction method based on the user historical check-in sequence of the present invention, in step S2, the specific steps of preprocessing the data are as follows:

[0014] First, data cleaning is performed to remove user and POI interaction data with missing values ​​or outliers, and each user's historical check-in data is sorted in chronological order. The formula is as follows:

[0015] ;

[0016] in, It is The historical check-in sequence of users after sorting, It is The original input sequence of users, represents the sorting function, Indicates sorting by timestamp;

[0017] Then, filtering technology is used to remove inactive users and unpopular POI interactions to ensure the quality and representativeness of the data. The operation is as follows:

[0018] ;

[0019] ;

[0020] in, and Represent the filtered active user set and popular POI set respectively. and Respectively represent users Check-ins and points of interest The number of interactions, and They represent the check-in number threshold and interaction number threshold respectively.

[0021] As a preferred solution of the city POI recommendation and timestamp prediction method based on user historical check-in sequences described in the present invention, the TAPT model includes an encoder module, a POI recommendation module and a timestamp prediction module.

[0022] As a preferred solution of the city POI recommendation and timestamp prediction method based on user historical check-in sequence described in the present invention, the encoder module is composed of a POI encoder and a timestamp encoder;

[0023] When processing a POI sequence, convert it to ,in represents the sequence length, Represents the embedding dimension of POI. The POI sequence is converted into the corresponding embedding. The formula is as follows:

[0024] ;

[0025] in, represents POI embedding, is the POI encoder, is the input POI sequence;

[0026] In order to capture the daily behavior patterns of users, the timestamp sequence is decoupled into three components: hours, minutes, and seconds. The decoupling formula is as follows:

[0027] ;

[0028] in, is a decoupled timestamp sequence, is the timestamp decoupler, is the input timestamp sequence,

[0029] Afterwards, Encoded as a learnable embedding, the formula is as follows:

[0030] ;

[0031] in, is timestamp embedding, is the timestamp embedding dimension, is the timestamp encoder,

[0032] After obtaining the POI embedding and timestamp embedding, they are input into a concatenation layer to generate a joint embedding representation. The specific processing formula is as follows:

[0033] ;

[0034] in, is a joint embedding, and , It is a connection operation.

[0035] As a preferred solution of the method for city POI recommendation and timestamp prediction based on user historical check-in sequence of the present invention, the timestamp prediction module specifically implements the following steps:

[0036] First, we will start from Joint representation learned Input into a Transformer model,

[0037] in, is the learned information representation, It is the basic framework for sequential POI recommendation;

[0038] Then further learn the timestamp representation:

[0039] ;

[0040] in, is the further learned joint representation, Represents a deep model based on the Transformer architecture;

[0041] Then, the timestamp feature extractor is used to extract the features related to the timestamp:

[0042] ;

[0043] in, is the feature extracted for the timestamp prediction task, is a timestamp feature extractor;

[0044] Then, decoded by the decoder , convert the extracted features into timestamp representation:

[0045] ;

[0046] in, is the decoded representation, It is a decoder;

[0047] Finally, the activation function and scaling operation are used to map To represent the discrete value of timestamp information:

[0048] ;

[0049] in, is the timestamp information of the final prediction, and Represents scaling and activation operations respectively.

[0050] As a preferred solution of the city POI recommendation and timestamp prediction method based on the user historical check-in sequence of the present invention, in step S3, the constructed TAPT model updates the model parameters by the gradient descent method to minimize the loss function in the training process, thereby achieving effective model training. The loss function used in the training process is as follows:

[0051] ;

[0052] in, is the loss for sequential POI recommendation:

[0053] ;

[0054] in, represents the total number of samples, is the Sigmoid function, It is The score of the positive samples, It is The score of negative samples, is a hyperparameter used to balance the two loss functions;

[0055] The mean square error MSE loss is used for optimization:

[0056] ;

[0057] in, is the length of each user's timestamp sequence, It is The real timestamp of each sample, It is The predicted timestamp of each sample.

[0058] Compared with the prior art, the present invention has the following beneficial effects: the present invention effectively overcomes the deficiencies of the prior art in user behavior modeling and time information processing by proposing a time-aware POI recommendation and timestamp prediction framework (TAPT). First, data filtering technology is used to exclude inactive users and unpopular POIs to ensure data quality and representativeness. Subsequently, through carefully designed timestamp encoding and feature connection, the model can more comprehensively capture the user's spatiotemporal preferences and improve the accuracy of recommendations. In addition, the application of a multi-task learning framework enables POI recommendation and timestamp prediction to complement each other, further enhancing the overall performance of the system. These advantages enable TAPT to perform well in real-time recommendation and prediction tasks, improving user experience and satisfaction. BRIEF DESCRIPTION OF THE DRAWINGS

[0059] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the present invention will be described in detail below in combination with the accompanying drawings and detailed embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative labor. Among them:

[0060] Figure 1 It is a flow chart of the city POI recommendation and timestamp prediction method based on the user's historical check-in sequence of the present invention;

[0061] Figure 2 A flow chart of the TAPT model provided by the present invention;

[0062] Figure 3 A flow chart of the POI recommendation module provided by the present invention;

[0063] Figure 4 This is a flow chart of the timestamp prediction module provided by the present invention. DETAILED DESCRIPTION

[0064] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, the specific embodiments of the present invention are described in detail below with reference to the accompanying drawings.

[0065] To solve the problems existing in the background technology, the present invention proposes a time-aware POI recommendation and timestamp prediction framework (TAPT). First, the present invention collects the historical check-in data of users and sorts the check-in records of each user in chronological order. Subsequently, a filtering technology is used to exclude the interaction of inactive users and unpopular POIs. Finally, a valid POI sequence and timestamp sequence are extracted from the processed user check-in sequence. This step ensures the quality and representativeness of the data and provides a reliable foundation for the subsequent recommendation model. The present invention first designs an efficient timestamp encoder, performs fine processing on the timestamp information in the user check-in sequence, decomposes the timestamp into a multi-dimensional vector, and converts it into a trainable embedding representation. This method effectively captures the irregularity of the time interval and enhances the model's temporal understanding of user behavior. Subsequently, the present invention designs an embedding connection layer to fuse the POI embedding with the timestamp embedding to generate a joint embedding representation, thereby improving the model's ability to capture user historical preferences and spatiotemporal features. After that, the joint embedding is input into the existing POI recommendation backbone model, and then the joint prediction of the next POI and its corresponding timestamp is realized through the connected timestamp predictor. The present invention uses a multi-task learning framework for end-to-end training and uses a unified loss function for model optimization to ensure that the visit time is accurately predicted while efficiently recommending POIs. The task flowchart is shown in the figure below. Figure 1 shown.

[0066] Step 1: Data Collection

[0067] Using mobile applications and GPS sensors, users' historical check-in data, including each POI and its corresponding timestamp, is collected regularly, and the data is transmitted to the server in real time and stored.

[0068] Step 2: Feature extraction

[0069] The feature extraction step includes three sub-steps: data cleaning, feature extraction, and data set construction:

[0070] 1. First, perform data cleaning, remove user and POI interaction data with missing values ​​or outliers, and sort each user's historical check-in data in chronological order. The formula is as follows:

[0071] (1)

[0072] in, It is The historical check-in sequence of users after sorting, It is The original input sequence of users, represents the sorting function, Indicates sorting by timestamp.

[0073] 2. Use filtering technology to remove inactive users and unpopular POI interactions to ensure data quality and representativeness. The operation is as follows:

[0074] (2)

[0075] (3)

[0076] in, and Represent the filtered active user set and popular POI set respectively. and Respectively represent users Check-ins and points of interest The number of interactions, and They represent the check-in number threshold and interaction number threshold respectively.

[0077] 3. Extract valid POI sequence and timestamp sequence from the processed user check-in sequence, expressed as and , thus constructing a data set.

[0078] Step 3: Construct TAPT model The TAPT model consists of three modules: encoder module (including POI encoder and timestamp encoder), POI recommendation module and timestamp prediction module. The flowchart of TAPT is as follows Figure 2 shown.

[0079] ①Encoder module

[0080] The encoder is a commonly used structure in machine learning to convert input data into a format suitable for model processing. In the TAPT model, the encoder module consists of a POI encoder and a timestamp encoder. When processing POI sequences, standard POI embedding techniques are used to convert them into ,in represents the sequence length, Represents the embedding dimension of POI. In this way, the semantic relationship between POIs can be effectively captured, providing a solid foundation for subsequent recommendation and timestamp prediction. The POI sequence is converted into the corresponding embedding as shown in formula (4):

[0081] (4)

[0082] in, represents POI embedding, is the POI encoder, is the input POI sequence.

[0083] In order to capture the daily behavior patterns of users, the timestamp sequence is decoupled into three components: hours, minutes, and seconds. Each part is a discrete value and can be converted into an embedding representation, so that the time characteristics of each check-in can be captured at a finer granularity. For example, the hour variable has only 24 discrete values ​​(ranging from 0 to 23). This decoupling process not only enhances the expressiveness of time information, but also provides rich contextual information for subsequent timestamp predictions. The decoupling process is as follows:

[0084] (5)

[0085] in, is a decoupled timestamp sequence, is the timestamp decoupler, is the input timestamp sequence. Encoding into learnable embeddings:

[0086] (6)

[0087] in, is timestamp embedding, is the timestamp embedding dimension, is the timestamp encoder. After obtaining the POI embedding and timestamp embedding, they are input into a concatenation layer to generate a joint embedding representation. This joint embedding is used to learn the strong correlation between POI and timestamp, thereby supporting subsequent multi-task learning. This process not only enhances the model's understanding of the relationship between time and place, but also lays the foundation for more accurate recommendations and time predictions. The specific processing flow is as follows:

[0088] (7)

[0089] in, is a joint embedding, and , It is a connection operation.

[0090] ②POI recommendation module

[0091] In the POI recommendation system, the model can recommend places of interest to users by analyzing the user's historical behavior data. After obtaining the joint embedding through formula (7), it is input into the backbone structure of the basic POI recommendation model. This process enables the model to learn the user's preference for POI and capture the preliminary user behavior timestamp features to provide support for subsequent timestamp prediction tasks. The flowchart of this module is shown in the figure. Figure 3 shown.

[0092] This process specifically includes the following steps:

[0093] (8)

[0094] in, is the learned information representation, It is the basic architecture for sequential POI recommendation.

[0095] In recommendation systems, feature filtering techniques are widely used to extract features related to recommendation tasks to improve the performance and accuracy of the model. In this process, feature filters are applied to extract POI-related features from the joint embedding. These extracted features are not only trained by the basic backbone structure, but also influenced by the timestamp features. This multi-level feature interaction enables the model to understand the user's preferences more comprehensively, thereby improving the accuracy and relevance of POI recommendations. The specific steps are as follows:

[0096] (9)

[0097] in, It represents the features related to POI learned and extracted by the model. is a POI feature extractor. An effective training goal is to align the user preference vector with the corresponding true vector as closely as possible. This alignment can help the model understand the user's needs more accurately and provide more relevant recommendations. To achieve this goal, it is necessary to calculate the user preference score. The specific steps are as follows:

[0098] (10)

[0099] in, It is the first The preference score of each POI, is the inner product function, is the position in the sequence User preferences, It is the first in the candidate set After calculating the user's preference scores for all POIs in the candidate set and ranking them, the system can recommend a Top-K POI list to the user.

[0100] ③Timestamp prediction module

[0101] The flowchart of this module is as follows Figure 4 shown.

[0102] First, the joint representation learned from formula (8) Input into a Transformer model to further learn timestamp representation:

[0103] (11)

[0104] in, is the further learned joint representation, Represents a deep model based on the Transformer architecture. Subsequently, the timestamp feature extractor is used to extract features related to the timestamp:

[0105] (12)

[0106] in, is the feature extracted for the timestamp prediction task, is a timestamp feature extractor. Then, it is decoded by the decoder , convert the extracted features into meaningful timestamp representation:

[0107] (13)

[0108] in, is the decoded representation, is the decoder. Finally, through the activation function and scaling operation mapping To represent the discrete value of timestamp information:

[0109] (14)

[0110] in, is the timestamp information of the final prediction, and Represent the zoom operation and activation operation respectively. After the above steps, the final hours, minutes, and seconds can be predicted to infer the timestamp information of the user's next interaction.

[0111] Model Training

[0112] The model parameters are updated by the gradient descent method to minimize the loss function during the training process, thereby achieving effective model training. The gradient descent method is a widely used optimization algorithm that gradually adjusts the parameter values ​​by calculating the gradient of the loss function relative to the model parameters so that the model moves in the direction of reducing the loss in each iteration. This iterative update mechanism enables the model to gradually approach the optimal solution and improve the accuracy and reliability of the prediction. The loss function used in the present invention is as follows.

[0113] During the model training process, the TAPT model, as a multi-task learning system, contains two different loss functions:

[0114] (15)

[0115] in, It is the loss used for sequential POI recommendation. For example, the cross entropy loss function is a commonly used loss function for optimizing sequential recommenders:

[0116] (16)

[0117] in, represents the total number of samples, is the Sigmoid function, It is The score of the positive samples, It is The score of negative samples. In formula (15), is a hyperparameter used to balance the two loss functions. We focus on the timestamp prediction task, aiming to narrow the gap between the predicted and true timestamps. We use the mean square error (MSE) loss to optimize this task:

[0118] (17)

[0119] in, is the length of each user's timestamp sequence, It is The real timestamp of each sample, It is The predicted timestamp of each sample.

[0120] Step 4: Model application

[0121] When the TAPT model of the present invention is applied, the user's historical check-in data is first used as input. Through data cleaning and feature extraction, a valid POI sequence and timestamp sequence are generated. Subsequently, the POI embedding and the timestamp embedding are fused to form a joint embedding using an efficient timestamp encoder and an embedding connection layer. Finally, the joint embedding is input into the POI recommendation model and the timestamp predictor to achieve a joint prediction of the next POI and its corresponding timestamp. In this way, the model can efficiently recommend POIs and accurately predict visit times. The specific application flow chart is shown in the figure below. Figure 1 Shown on the right.

[0122] Although the present invention has been described above with reference to the embodiments, various modifications may be made thereto and parts thereof may be replaced by equivalents without departing from the scope of the present invention. In particular, as long as there is no structural conflict, the various features in the embodiments disclosed in the present invention may be used in combination with each other in any manner, and the fact that these combinations are not exhaustively described in this specification is only for the sake of omitting space and saving resources. Therefore, the present invention is not limited to the specific embodiments disclosed herein, but includes all technical solutions falling within the scope of the claims.

Claims

1. A city POI recommendation and timestamp prediction method based on user historical check-in sequences, characterized in that: The steps include: S1. Collect the user's historical check-in data and sort each user's check-in record in chronological order to form a user check-in sequence, and then transmit the data to the server in real time and store it; S2, preprocess the data, and then extract the valid POI sequence and timestamp sequence from the preprocessed user check-in sequence, which are expressed as and ; S3, input the POI sequence and timestamp sequence into the constructed TAPT model, and use the efficient encoder module and embedding connection layer in the TAPT model to fuse the POI embedding with the timestamp embedding to form a joint embedding. Finally, the joint embedding is input into the POI recommendation model and timestamp predictor to achieve joint prediction of POI and its corresponding timestamp; The TAPT model includes an encoder module, a POI recommendation module, and a timestamp prediction module; The encoder module consists of a POI encoder and a timestamp encoder; When processing a POI sequence, convert it to ,in represents the sequence length, Represents the embedding dimension of POI. The POI sequence is converted into the corresponding embedding. The formula is as follows: ； in, represents POI embedding, is the POI encoder, is the input POI sequence; In order to capture the daily behavior patterns of users, the timestamp sequence is decoupled into three components: hours, minutes, and seconds. The decoupling formula is as follows: ； in, is a decoupled timestamp sequence, is the timestamp decoupler, is the input timestamp sequence, Afterwards, Encoded as a learnable embedding, the formula is as follows: ； in, is timestamp embedding, is the timestamp embedding dimension, is the timestamp encoder; After obtaining the POI embedding and timestamp embedding, they are input into a concatenation layer to generate a joint embedding representation. The specific processing formula is as follows: ； in, is a joint embedding, and , It is a connection operation.

2. The method for city POI recommendation and timestamp prediction based on user historical check-in sequence according to claim 1 is characterized in that: In step S1, the user's historical check-in data, including each POI and its corresponding timestamp, is collected regularly using a mobile application and a GPS sensor, and the data is transmitted to a server in real time and stored.

3. The method for city POI recommendation and timestamp prediction based on user historical check-in sequence according to claim 1 is characterized in that: In step S2, the specific steps of preprocessing the data are as follows: First, data cleaning is performed to remove user and POI interaction data with missing values ​​or outliers, and each user's historical check-in data is sorted in chronological order. The formula is as follows: ； in, It is The historical check-in sequence of users after sorting, It is The original input sequence of users, represents the sorting function, Indicates sorting by timestamp; Then, filtering technology is used to remove inactive users and unpopular POI interactions to ensure the quality and representativeness of the data. The operation is as follows: ； ； in, and Represent the filtered active user set and popular POI set respectively. and Respectively represent users Check-ins and points of interest The number of interactions, and They represent the check-in number threshold and interaction number threshold respectively.

4. The method for city POI recommendation and timestamp prediction based on user historical check-in sequence according to claim 1 is characterized in that: The timestamp prediction module implements the following steps: First, we will start from Joint representation learned Input into a Transformer model, in, is the learned information representation, It is the basic framework for sequential POI recommendation; Then further learn the timestamp representation: ； in, is the further learned joint representation, Represents a deep model based on the Transformer architecture; Then, the timestamp feature extractor is used to extract the features related to the timestamp: ； in, is the feature extracted for the timestamp prediction task, is a timestamp feature extractor; Then, decoded by the decoder , convert the extracted features into timestamp representation: ； in, is the decoded representation, It is a decoder; Finally, the activation function and scaling operation are used to map To represent the discrete value of timestamp information: ； in, is the timestamp information of the final prediction, and Represents scaling and activation operations respectively.

5. The method for city POI recommendation and timestamp prediction based on user historical check-in sequence according to claim 1 is characterized in that: In step S3, the constructed TAPT model updates the model parameters by the gradient descent method to minimize the loss function in the training process, thereby achieving effective model training. The loss function used in the training process is as follows: ； in, is the loss for sequential POI recommendation: ； in, represents the total number of samples, is the Sigmoid function, It is The score of the positive samples, It is The score of negative samples, is a hyperparameter used to balance the two loss functions; The mean square error MSE loss is used for optimization: ； in, is the length of each user's timestamp sequence, It is The real timestamp of each sample, It is The predicted timestamp of each sample.

Images