A location service recommendation method based on deep semantic extraction and diffusion model

Abstract

The application relates to a location service recommendation method based on a deep semantic extraction and diffusion model, which is used for next point of interest (POI) recommendation. A location service recommendation model DSDRec is constructed and trained, the DSDRec comprises a deep semantic feature extraction model, an encoder-decoder model and a diffusion model, rich semantic texts are freely obtained from original trajectory sequences through the deep semantic feature extraction model, a PLM is used in the encoder-decoder model to obtain sentence representation as high-quality deep semantic features of each trajectory. Finally, the diffusion model is used to help the recommendation model to learn more abstract interaction information. Thorough experiments are conducted by using two real-world datasets, and the results are visualized. The analyses prove the effectiveness of the method.

Description

Technical Field

[0001] This invention relates to the field of point-of-interest recommendation technology, and in particular to a location service recommendation method based on deep semantic extraction and diffusion model. Background Technology

[0002] With the increasing use of location-based social networks (LBSNs) and applications providing location-based services, there is a growing enthusiasm for creating Points of Interest (POI) recommendation models. These models are used to provide personalized recommendations to users, covering a wide range of services and products. Recommendation models around POIs have become an important feature in various check-in applications such as Yelp, Weibo, and Foursquare. Location-based platforms track users' check-in records and provide personalized POI recommendations, reducing information overload on online platforms. Unlike typical recommendation tasks, POI recommendation focuses on understanding the geographical aspects of POIs and how their location influences user behavior. Currently, many researchers are considering using deep learning methods to learn the sequential features of historical trajectories for better predictions, such as sequential recommendation models and attention models. In LBSNs, spatiotemporal information plays a crucial role in the decisions of both users and the recommendation system as a key feature. Therefore, many studies utilize spatiotemporal information in trajectories to reveal users' personalized movement preferences and improve recommendation performance; most of these studies capture spatiotemporal information by separately calculating the geographical distance between two interactions and the check-in time interval.

[0003] Current research on POI recommendation tasks primarily focuses on representation learning. This involves acquiring diverse representations of trajectory features and integrating these representations for accurate predictions. Each feature can provide a different perspective on the trajectory, and the lack of certain features can degrade recommendation performance. Semantics plays a crucial role in many artificial intelligence tasks, particularly in Natural Language Processing (NLP). They are also valuable in various recommendation tasks, including electronics and news recommendations. In POI recommendation tasks, some research has also demonstrated the potential of semantics. The semantics of user activity are often conveyed through brief textual descriptions. Understanding the semantics of a user's current activity is essential for predicting their subsequent location. To address this challenge, SERM, a method capable of capturing semantically conscious spatiotemporal transition patterns, has been introduced, ultimately improving the accuracy of location prediction. Similarly, spatiotemporal relational path embedding, which incorporates spatiotemporal and semantic elements, has been applied to advance the next POI recommendation task. These studies primarily focus on leveraging specific independent textual attributes from the LBSN dataset as semantic features.

[0004] However, current research has several limitations that restrict the performance of POI recommendation models. First, for POI recommendation tasks, obtaining high-quality deep semantic features is a significant challenge due to the lack of high-quality text data in the LBSN dataset. Current research primarily utilizes some independent text attributes from the LBSN dataset as semantic features, such as review and location categories, but these methods are insufficient in capturing the overall semantics of the trajectory. Therefore, the acquired semantic signals remain at a superficial level, lacking a comprehensive understanding of the underlying trajectory semantics. Second, most existing research typically focuses only on the spatiotemporal relationship between two interactions, but in real-world scenarios, user check-in behavior is not continuous (i.e., the sparsity of check-in records). That is, when the space or time interval between two check-in records is large—the cutoff point—the user's movement preferences may have changed significantly after the gap. Therefore, considering only the local spatiotemporal interval between two interactions ignores this situation and makes it difficult for the model to learn the user's true movement preferences. Summary of the Invention

[0005] In view of the above-mentioned problems in the existing technology, the technical problem to be solved by the present invention is: a method with high recommendation accuracy that can reflect the user's real mobile preferences.

[0006] To solve the above-mentioned technical problems, the present invention adopts the following technical solution: a location service recommendation method based on deep semantic extraction and diffusion model, comprising the following steps:

[0007] Define the user set and the location set as U = (u1, u2, ..., u3) and U4) respectively. N ) and V = (v1, v2, ..., v M The check-in sequence is represented as ci = (vid, cid, cate, ct, lon, lat), where N represents the total number of users and M represents the total number of locations or points of interest (POIs). A check-in point can be represented as ci = (vid, cid, cate, ct, lon, lat), where vid and cid are the location identifier and category, respectively, cate represents the name of cid, and ct represents the time the check-in action occurred. The variables lon and lat represent the longitude and latitude of the check-in location, indicating its geographical location. The user's check-in sequence, i.e., the trajectory sequence, is represented by S. u It is represented as S. u = (c1, c2, ..., c n ), where ci represents check-in in chronological order at a specific location, and n represents the timestamp.

[0008] The location service recommendation model DSDRec is constructed and trained. DSDRec includes a deep semantic feature extraction model, an encoder-decoder model, and a diffusion model.

[0009] S1: A deep semantic feature extraction model that uses cue engineering to create continuous natural language text from discrete trajectory sequences. This natural language text is called a cue sentence, which includes the user u's long-term preferences. and short-term preferences

[0010] Using PLM to respectively and The corresponding transformation is into a continuous context feature representation V L and V S .

[0011] S2: The encoder-decoder model obtains high-quality place context representations of sequence embeddings and learns trajectory sequences by fusing deep semantics and spatiotemporal information.

[0012] S2-1: From S u Select the time c when the check-in action occurs at each check-in point. t As a time series S u At each check-in point, the longitude (lon) and latitude (lat) of the check-in location are selected as the spatial sequence. in use Spatial encoding vector, using This represents the time-coded vector.

[0013] S2-2: Using an embedding layer to obtain a compact representation of Su, let... and These represent the location embedding matrix, spatial embedding matrix, temporal embedding matrix, and location category embedding matrix, respectively. According to E... V E S and E T From S u Retrieve location feature representation from location ID from Obtaining time feature representation According to S u The category ID in the data is used to obtain the feature representation of the venue category. Will and The fusion yields the final representation of the input trajectory sequence. in, C represents the number of location categories, L represents the length of Su, and d represents the dimension of the vector.

[0014] S2-3: Using a sequence encoder The context information is used to model and obtain the context representation of the user trajectory sequence.

[0015] S2-4: Incorporating long-term mobility preference semantics and short-term mobility preference semantics Integrate Obtain a contextual representation that integrates the semantics of users' long-term mobile preferences.

[0016] S2-5: Polymerization Obtain high-quality place context representation

[0017] S3: Enhanced using a diffusion model Obtain the site context representation reconstructed through diffusion.

[0018] S4: Training DSDRec begins by minimizing the objective function. Train the encoder-decoder model, then freeze all parameters in the encoder-decoder model, and minimize the objective function. To train the diffusion model. The parameters of the PLM are not updated during these two training steps. The training of the diffusion model includes training in the forward process and training in the backward process.

[0019] S5: For a new user, the user's trajectory sequence is input into the trained DSDRec to obtain the user's location context representation reconstructed through diffusion. The last element is then extracted from the user's location context representation reconstructed through diffusion. Will As a representation of the trajectory of new users, Perform matrix multiplication with the embedding matrix of all locations of the new user, calculate the score as shown in formula (26), and take the location with the maximum score as the next POI recommendation;

[0020]

[0021] Among them, E V ′ represents the embedding matrix of all locations for the new user.

[0022] As an improvement, V is obtained in S1. L and V S The steps are as follows:

[0023] A set of text templates was created, and the input slots were filled using the `cate` keyword to construct prompt sentences. and Describes the user's global movement behavior. and It describes the user's local mobile behavior patterns.

[0024] for The five most frequently visited location categories by users were analyzed; for Based on CT, the day is divided into two categories: weekdays and weekends, and the three most frequently visited venue categories are analyzed for each category; for The day is divided into three categories based on CT (Category A), and the three most frequently visited place categories by users in each category are analyzed; for Calculate a value τ representing how many of a user's most recently visited locations were not visited in their historical activity. Then define a threshold Γ: if τ >= Γ, it indicates a change in the user's short-term preferences; otherwise, no change has occurred. Fill the slots with the five most recently visited place categories by the user.

[0025] Finally, and The sentences that are connected together to form a prompt for u's long-term preference are denoted as follows: Will and The phrases connected together to form a short-term preference for u are denoted as follows:

[0026] Using PLM and The transformation is converted into a continuous feature representation, and the calculation of this transformation is as shown in formula (9):

[0027]

[0028]

[0029] in, and It is a tokenized version of the prompt sentence. It is a PLM, and φ is a parameter of the PLM. and They are and After PLM processing, the context feature representation, l L and l S They are and The length.

[0030] As an improvement, the calculation in S2-1 and The steps are as follows:

[0031] S2-1-1: Input: Column L, Haversine algorithm Spatial threshold Δ S Time threshold ΔT

[0032] S2-1-2: Initialize the time zone encoding list Spatial region coding Variables k1 = 0, k2 = 0;

[0033] S2-1-3: The spatiotemporal region coding steps are as follows:

[0034] a) Calculate the distance between the two locations based on their latitude and longitude. Among them lon i and lat i Let L and L represent the latitude and longitude of the i-th location, respectively. The location index starts from i=1 and ends at L-1, where L is the length of the trajectory sequence. i-1 and lat i-1 Let represent the latitude and longitude of the (i-1)th location, respectively.

[0035] Using Haversine formula f H The distance between two check-in locations is calculated using the · symbol. The calculation process is as follows:

[0036]

[0037] b) Calculate the time interval ΔT = |ct| between the timestamps of the user's check-in at the two locations. i -ct i-1 |, where ct i ct represents the timestamp recorded when a user checks in at the i-th location. i-1 This represents the timestamp recorded when the user checks in at location i-1;

[0038] c) Perform threshold judgment operation, when When the time interval is greater than the spatial threshold ΔS, let k1 = (k1+1)%2, and when the time interval is greater than the spatial threshold ΔS, let k1 = (k1+1)%2. When the time threshold ΔT is greater than 2, let k2 = (k2 + 1) % 2; d) Add k1 to the time threshold ΔT. Add k2 to middle.

[0039] S2-1-4: Output the spatial region code for user u User u's time zone encoding

[0040] As an improvement, in S2-2, the following is obtained: The steps are as follows:

[0041] will and Combined together The specific calculations are as follows:

[0042]

[0043] in, This represents the trajectory characteristics of u. γ is a hyperparameter. ⊙ represents vector dot addition, ⊙ represents vector dot multiplication, and drop represents vector dot multiplication. sp This indicates the dropout operation.

[0044] Using a learnable position embedding matrix The positions between check-ins are marked to avoid losing sequence position information, where Lmax is the maximum length of the trajectory. Then, the final representation of the input trajectory sequence is shown in Equation (12).

[0045]

[0046] in The position embedding matrix represents the learnable parameters.

[0047] As an improvement, the step of S2-3 to obtain the context representation of the user trajectory sequence is as follows:

[0048] Construct three vectors Q, K, and V as shown in the following formula (13):

[0049]

[0050]

[0051]

[0052] Among them, W Q W K and These are learnable parameters. Next, the output of MHAM is calculated and expressed as:

[0053]

[0054] head i =Attention(QW i q KW i k VW i v (15)

[0055]

[0056] Where [;] represents a join operation. and These are learnable parameters. It is the new latitude after the original latitude d is divided by H, where H is a hyperparameter, head1, head2, and head h These represent the 1st, 2nd, and hth attention modules in the multi-head attention mechanism, respectively. This represents the sequence context features output after processing by the multi-head self-attention module.

[0057] In MHAM, a causal mask was chosen as the attention mask. A residual connection mechanism was used, i.e. right Using PWFL, the format is as follows:

[0058]

[0059] Here, and These are learnable parameters; ReLU(·) represents the modified linear activation function, and Drop(·) represents the dropout operation. This is the final output.

[0060] As an improvement, S2-4 yields a high-quality contextual representation of the trajectory sequence. The steps are as follows:

[0061] set up and These represent the downsampling matrix, the upsampling matrix, and the matrix mapping multimodal features to a unified space, respectively. These are three d-dimensional bias vectors. The feature fusion operation is represented as:

[0062]

[0063]

[0064]

[0065] Where Φ is the LeakyReLU function with a negative slope of 0.1.

[0066] As an improvement, S2-5 is obtained As in formula (21):

[0067]

[0068] in, These are learnable parameters.

[0069] As an improvement, when training the encoder-decoder model in S4, Su is split into two parts: As the input sequence As a positive sample, where y i =c i +1 indicates the next point in Su. Sampling and Different negative samples for each point in

[0070] Training objectives Defined as:

[0071]

[0072] Among them, O (1) This represents a tensor filled with 1s, with a shape similar to... Same; while O (0) Represents a tensor filled with zeros, with a shape similar to... Same. c i It is the i-th check-in point in the trajectory sequence, z i The negative sample z corresponding to the i-th check-in point i ≠c i , It is S u After processing by the encoder-decoder model, c i The corresponding contextual feature representation, L is the trajectory S u The length.

[0073] As an improvement, S3 is obtained The steps are as follows:

[0074] During the forward process: Perform a space transformation operation on the pair The distribution is normalized, and its value range is expanded:

[0075] for set up This represents its mean. Express its variance, and then... Normalize and scale:

[0076]

[0077] Where λ is the hyperparameter of the temperature coefficient, and σ(.) is the variance calculation function.

[0078] Next, randomly select a set of diffusion steps. make For each diffusion step in T^, sample l Gaussian noises. Then, formula (2) is executed to obtain noise data. ε t The t-th diffusion step is the introduced random noise. It is a noise step sample set from which one sample is sampled from T.

[0079] At the end of the back diffusion, its output Performing a spatial inverse transformation operation is specifically represented as follows:

[0080]

[0081] in, It is a location context representation reconstructed through diffusion, and μ represents the formula for calculating the average value.

[0082] As an improvement, the steps for training the S4 diffusion model are as follows:

[0083] Training during the positive process:

[0084] Semantics of users' short-term mobile patterns Introducing a diffusion model as a personalized semantic signal for users, let... These are trainable parameters used for dimensionality transformation, and are set as follows: The training process in the forward pass is as follows: (This refers to the timestamp embedding matrix in the diffusion stream.)

[0085] S4-1-1: Input: Neural network f θ .

[0086] S4-1-2: Trainable network parameters: Θ={θ,W S E T}

[0087] S4-1-3: First calculate and and along the vector 0th dimension extension Next, align its dimensions.

[0088] S4-1-4: Loop until the model converges:

[0089] a) From the time step set Samples are taken at time step t and random Gaussian noise is sampled based on t. b) Calculate according to formula (2) c) Calculate the eigenvector e of the time step t =E T (t), along vector e t 0th dimension extension align its dimensions; d) and e t e) Concatenate vectors along the last dimension and use the result as model input; e) Calculate the predicted value of random noise. f) Calculate the loss value based on formula (8) And update the model parameters Θ={θ,W} based on the loss value. S E T}

[0090] The training process in the reverse process is as follows: εT represents the prior constraints.

[0091] Input: Model parameters Θ = {θ, W} trained via a forward pass. S E T}, T,

[0092] S4-2-1: First calculate and and along the vector 0th dimension extension Next, align its dimensions;

[0093] S4-2-2: Sampling random Gaussian noise

[0094] S4-2-3: Input and ε T And calculate using formula (2)

[0095] S4-2-4: Loop t = T-1, ..., 1: a′) Calculate the eigenvector e of the time step. t =E T (t), along vector e t 0th dimension extension Next, align its dimensions; b′) Sample random noise. c′) Predicting noise using a diffusion model d′) Calculate the mean of the data distribution using formula (5). Input is and t;e′) sampled random noise f′) Calculates the data distribution at the t-1 diffusion step using the reparameter technique.

[0096] S4-2-5: Return

[0097] Compared with the prior art, the present invention has at least the following advantages:

[0098] 1. The model proposed in this invention has high prediction accuracy; 2. The deep semantic extraction paradigm proposed in this invention can provide reference and guidance for other AI fields, and provides a novel approach to solving the application of deep semantic extraction in the field of sequence recommendation. Moreover, the deep semantic features obtained through this paradigm have good transferability.

[0099] 2. Deep Semantic Extraction Pipeline: Utilizing prompt techniques from the field of natural language processing combined with data analysis, discrete numerical trajectories are transformed into semantically rich text; the semantically rich text is input into a pre-trained language model to obtain deep semantic features for each trajectory sequence; and the acquired semantic information is integrated into the contextual representation of the trajectory using a deep semantic fusion algorithm.

[0100] 3. Encoder-decoder framework: It consists of two stacked two-layer transformer encoder modules to obtain contextual representations of trajectory sequences and perform result prediction.

[0101] 4. Diffusion Model: The diffusion model is used to adjust the data distribution of the encoder-decoder output to better fit the true distribution of the data and incorporate the user's personalized characteristics. Attached Figure Description

[0102] Figure 1 This is a schematic diagram of the DSDRec model.

[0103] Figure 2 An example of spatiotemporal region encoding.

[0104] Figure 3 An overview of the diffusion process in DSDRec.

[0105] Figure 4 This is an example of space transformation.

[0106] Figure 5 For the neural network f in the diffusion model θ The structure.

[0107] Figure 6 Visualization of attention distribution.

[0108] Figure 7 To investigate the removal of semantic signals in a diffusion model on two datasets. Performance comparison.

[0109] Figure 8 The impact of different dimensions d on the recommendation accuracy of the two datasets. Detailed Implementation

[0110] The present invention will now be described in further detail.

[0111] This invention proposes the following solutions to overcome the difficulty of obtaining deep semantic features from trajectory sequences due to the scarcity of high-quality text data. First, cue engineering techniques are employed to extract rich semantic text from the original discrete sequence to describe the trajectory. Then, a pre-trained language model (PLM) is used to obtain sentence representations as deep semantic features of the trajectory. This method can establish deep semantic connections within the trajectory. To capture global spatiotemporal interaction information, a simple yet effective method called Spatio-Temporal Area Encoding is proposed. Specifically, to inform the recommendation model about the locations of these cut points in the trajectory, the trajectory sequence is divided into different spatiotemporal areas based on spatiotemporal thresholds, and then different codes are assigned to different spatiotemporal areas. Experience shows that user movement behavior within the same spatiotemporal area exhibits similar characteristics, while different spatiotemporal areas show diversity. Therefore, it is hoped that spatiotemporal area encoding can better help the recommendation model capture users' personalized movement preferences. These proposed solutions address the limitations of existing methods by introducing deep semantic features and capturing global spatiotemporal interaction information. By enhancing the understanding of trajectory semantics and considering the impact of tangent points, the performance of POI recommendation models can be improved, providing users with more accurate and personalized recommendations.

[0112] By counteracting the perturbations caused by random Gaussian noise introduced during the diffusion process, the contextual representation of the trajectory sequence is enhanced, enabling the recommendation model to learn more abstract interaction information. To achieve this goal, an improved diffusion model is proposed as a submodule within the recommendation model. Specifically, (1) the forward and reverse processes of the diffusion model are made more stable and controllable by introducing semantic signals and prior constraints. By utilizing semantic signals and prior knowledge, the diffusion process can be guided to generate more coherent and meaningful trajectories. (2) A spatial transformation operation is proposed to make the diffusion model more suitable for recommendation models. This spatial transformation operation allows the diffusion model to be adjusted to adapt to the specific features and requirements of the recommendation model. By introducing these improvements into the diffusion model, the aim is to enhance the contextual representation learning of the trajectory sequence and improve the overall performance of the recommendation model.

[0113] The diffusion model can be divided into two steps: a forward process and a backward process. The forward process gradually adds Gaussian noise to the original data, while the backward process gradually reduces the noise and recovers the original data from the noisy data. Given the original data... During the forward propagation process, the diffusion model gradually adds Gaussian noise to x0 through T diffusion steps. Then, β t ∈( 0 , 1The noise schedule is defined as a series of predefined noise schedules used to control the amount of noise added in each diffusion step t, where t is in the range [0, T]. The forward process q(x) t |x t-1 It can be defined as:

[0114]

[0115] Based on previous research, x can be calculated directly from x0. t ,

[0116]

[0117] in, And α t =1-β t Where x0 represents the original data input to the diffusion model, i.e. x t and x t-1 Let β represent the data at diffusion steps t and t-1, respectively, under the condition of random Gaussian noise ε. t ∈(0,1) is defined as a series of predefined noise schedules to control the amount of noise added in each diffusion step t, where t is in the range [0,T], q(.|.) denotes the conditional probability function, and β t ∈(0,1) is a hyperparameter used to control the time step. I represents the identity matrix. α t It has no real meaning.

[0118] In the reverse process, the target is q(x) t-1 |x t Unfortunately, complete data is required before calculating this posterior distribution. t-1 |x t (x0) can be expressed by the following formula: Then, according to Bayes' theorem, we can obtain:

[0119]

[0120] Based on the Gaussian probability density function It can be calculated:

[0121]

[0122] Then we can calculate q(x) t-1 |x t The mean and variance of x0 are shown below:

[0123]

[0124]

[0125] According to formulas 5 and 6, let q(x) t-1 |x t )≈q θ (x t-1 |x t ), and each transition x t →x t-1 The definition is as follows:

[0126]

[0127] Where θ is the neural network f θ The learnable parameters, and the variance σ θ (x t ,t) 2 It is set to be untrainable to ensure stability. The ultimate goal is to train f. θ To adapt

[0128] The variational lower bound (VLB) can be used as the objective function. Furthermore, the parameter θ is transformed into noise ∈ θ This is to facilitate calculation.

[0129]

[0130] in, It is Gaussian noise at the diffusion step t. The network f trained using reparameterization techniques... θ It can be used to recover the original data. By utilizing the random Gaussian noise introduced in the diffusion process, it is hoped that the recommendation model can learn more abstract interaction information in the trajectory sequence by resisting these random perturbations.

[0131] See Figures 1-8 A location service recommendation method based on deep semantic extraction and diffusion model includes the following steps:

[0132] Define the user set and the location set as U = (u1, u2, ..., u3) and U4) respectively. N ) and V = (υ1, υ2, ..., υ M The check-in sequence is represented as ci = (vid, cid, cate, ct, lon, lat), where N represents the total number of users and M represents the total number of locations or points of interest (POIs). A check-in point can be represented as ci = (vid, cid, cate, ct, lon, lat), where vid and cid are the location identifier and category, respectively, cate represents the name of cid, and ct represents the time the check-in action occurred. The variables lon and lat represent the longitude and latitude of the check-in location, indicating its geographical location. The user's check-in sequence, i.e., the trajectory sequence, is represented by S. uIt is represented as S. u = (c1, c2, ..., c n Let ), where ci represents check-in at a specific location in chronological order, and n represents the timestamp. Given a set of locations V, the goal is to predict which location v∈V a user u might visit at timestamp n+1, called the next POI. For ease of description, this paper assumes a default batch size of 1.

[0133] The location service recommendation model DSDRec is constructed and trained. DSDRec includes a deep semantic feature extraction model, an encoder-decoder model, and a diffusion model.

[0134] S1: A deep semantic feature extraction model that uses cue engineering to create continuous natural language text from discrete trajectory sequences. This natural language text is called a cue sentence, which includes the user u's long-term preferences. and short-term preferences

[0135] Using PLM to respectively and The corresponding transformation is into a continuous context feature representation V L and V S .

[0136] S2: The encoder-decoder model obtains high-quality place context representations of sequence embeddings and learns trajectory sequences by fusing deep semantics and spatiotemporal information.

[0137] S2-1: Trajectory sequences carry important spatial and temporal information. Generally, 1) user movement patterns within the same geographical area have certain characteristics and vary between different geographical areas; 2) compared to check-in behavior at a specific moment, user movement behavior over a continuous period shows certain regularities. In real-world scenarios, user check-in behavior is not continuous; that is, when the spatial or time interval between two check-in records is large, the user's movement preferences may change significantly after the blank period (in this paper, these check-in points are referred to as tangent points). Considering the above, this invention proposes a simple yet effective method to describe the spatial and temporal relationships between check-ins from a global perspective, and to pass the location information of tangent points in the trajectory to the recommendation model, such as... Figure 2 As shown, specifically, the trajectory sequence is divided into different spatial and temporal regions based on spatiotemporal thresholds (ΔS and ΔT). Within the same spatiotemporal region, the user's movement behavior will exhibit similar characteristics, while differences will appear between different spatiotemporal regions. Figure 2In this context, ΔT represents the time threshold, and ΔS represents the spatial threshold. Based on the timestamp of each check-in and the geographical location of each location, the trajectory can be divided into a series of time regions (T-Area) and spatial regions (S-Area), defined by ΔT and ΔS, respectively. Then, each spatiotemporal region is alternately assigned a value from 0 to 1 as an identifier. In this example, for the time region encoding, c4, c4, c8, and c9 are cut-off points; for the spatial region encoding, c3, c4, c5, c6, c8, and c9 are cut-off points.

[0138] From S u Select the time c when the check-in action occurs at each check-in point. t As a time series At each check-in point in Su, the longitude (lon) and latitude (lat) of the check-in location are selected as the spatial sequence. in use Spatial encoding vector, using This represents the time-coded vector. It is a spatial encoding vector. Spatial eigenvectors can be obtained through the embedding matrix ES. It is a time-coded vector. The time feature vector can be obtained through the embedding matrix ET.

[0139] S2-2: An embedding layer is used to obtain a compact representation of Su. In this component, sequence and spatiotemporal information are encoded, capturing the essence of the input data. Specifically, let... and] These represent the location embedding matrix, spatial embedding matrix, temporal embedding matrix, and location category embedding matrix, respectively. According to E... V E S and E T From S u Retrieve location feature representation from location ID from Obtaining time feature representation Based on the category ID in Su, obtain the location category feature representation. Will and The fusion yields the final representation of the input trajectory sequence. in, C represents the number of location categories, L represents the length of Su, i.e., how many location IDs (the locations visited by this user over a period of time) are in a trajectory sequence, and d represents the dimension of the vector. The location ID represents the index of each point of interest (POI), such as ck for location k. For example, S...u = (c1, c2, ..., c n Su is the trajectory sequence of user u, which consists of many check-in points (POIs, i.e., location IDs).

[0140] If there are 100 locations in the dataset, then the numbers 1 to 100 correspond to the 100 location IDs.

[0141] Each location has a category. For example, if location id = 5, its category C is "coffee shop". If there are 100 locations in the dataset, and these 100 locations have a total of 20 categories, and the category is represented by the numbers 1-20, then the category id corresponds to one of the numbers 1-20.

[0142] EV, Es, and ET are user-defined, and the weights of these matrices are trainable. It is a spatial encoding vector. Through the embedding matrix E S Able to obtain spatial feature vectors It is a time-coded vector. Through the embedding matrix E T Able to obtain time feature vectors It is a location category feature vector (for example, a location with location id=5 has a category C of coffee shop). Through the embedding matrix E C get.

[0143] S2-3: Using a sequence encoder The context information is used to model and obtain the context representation of the user trajectory sequence.

[0144] S2-4: Incorporating long-term mobility preference semantics and short-term mobility preference semantics Integrate Obtain a contextual representation that integrates the semantics of users' long-term mobile preferences.

[0145] S2-5: Polymerization Obtain high-quality place context representation

[0146] S3: Enhanced using a diffusion model Obtain the site context representation reconstructed through diffusion.

[0147] Diffusion models can continuously add Gaussian noise and train neural networks to recover noisy data, enabling the model to learn from the noisy data and recover the distribution of the original data, thus giving diffusion models excellent generative capabilities. This invention utilizes diffusion models to help recommendation models learn more abstract interaction information. However, compared to images with pixel values ​​ranging from 0 to 255, adding a few steps of Gaussian noise does not cause excessive interference to their data distribution. However, the original data entering the diffusion model... These are dense eigenvectors, highly sensitive to Gaussian noise. This means that even adding a small amount of noise, or performing a single diffusion step, can disrupt their distribution, making the diffusion model difficult to work and converge. To address this issue, this invention proposes a simple yet effective operation: spatial transformation and inverse spatial transformation.

[0148] S4: Training DSDRec begins by minimizing the objective function. Train the encoder-decoder model, then freeze all parameters in the encoder-decoder model, and minimize the objective function. To train the diffusion model. During these two training steps, the PLM parameters will not be updated, as this has been found not to harm DSDRec's performance and would actually speed up the computation process.

[0149] Training of the diffusion model includes training in the forward process and training in the reverse process.

[0150] S5: For a new user, the user's trajectory sequence is input into the trained DSDRec to obtain the user's location context representation reconstructed through diffusion. The last element is then extracted from the user's location context representation reconstructed through diffusion. Will As a representation of the trajectory of new users, Perform matrix multiplication with the embedding matrix of all locations of the new user, calculate the score as shown in formula (26), and take the location with the maximum score as the next POI recommendation;

[0151]

[0152] Among them, E V ′ represents the embedding matrix of all locations for the new user.

[0153] Specifically, in S1, V is obtained. L and V S The steps are as follows:

[0154] Prompt engineering is used to create continuous natural language text, called prompt sentences, from discrete sequences of trajectories. These prompt sentences summarize the user's long-term and short-term preferences, providing a semantic understanding of user behavior.

[0155] A set of text templates was created, as shown in Table 1, and prompt sentences were constructed by using cate to fill the input slots through data analysis of user historical trajectories. and It describes the user's global movement behavior, showing the times and places the user typically prefers to go. and It describes the user's local movement behavior patterns, showing the user's recent trajectory and whether the user's recent movement patterns have changed.

[0156] Table 1 Prompt Statements

[0157]

[0158] Based on the user's historical trajectory sequence, for The five most frequently visited location categories by users were analyzed; for Based on CT, the day is divided into two categories: weekdays and weekends, and the three most frequently visited venue categories are analyzed for each category; for The day is divided into three categories based on CT (time zone): 6:00 AM to 12:00 PM, 12:00 PM to 7:00 PM, and 7:00 PM to 6:00 AM. The three most frequently visited location categories within each category are then analyzed. We calculate a value τ, representing how many of a user's most recently visited locations were not visited in their historical trajectory. Then, we define a threshold Γ: if τ >= Γ, it indicates a change in the user's short-term preferences; otherwise, no change has occurred. Note that in this work, Γ is set to 0.3; for... Fill the slots with the five most recently visited place categories by the user.

[0159] Finally, and The sentences that are connected together to form a prompt for u's long-term preference are denoted as follows: Will and The phrases connected together to form a short-term preference for u are denoted as follows: It is important to note that a space marker [EOS] should be inserted at the end of each sentence, and... and Insert a space marker [CLS] at the beginning.

[0160] Using PLM and The transformation is converted into a continuous feature representation to capture the semantic information of the trajectory. The calculation of this transformation is shown in formula (9):

[0161]

[0162]

[0163] in, and It is a tokenized version of the prompt sentence. It is a PLM, and φ is a parameter of the PLM. and They are and After PLM processing, the context feature representation, l L and l S They are and The length.

[0164] It's important to note that a spatial marker [CLS] is added at the beginning of each prompt sentence to aggregate the global semantics of the sequence. Therefore, V is chosen. L and V S The feature representation of [CLS] serves as the semantic representation of a user's long-term mobile preferences. and short-term mobility preference semantics It is the output feature dimension of PLM.

[0165] Specifically, the calculation in S2-1 and The steps are as follows:

[0166] S2-1-1: Input: Column L, Haversine algorithm Spatial threshold Δ S Time threshold Δ T

[0167] S2-1-2: Initialize the time zone encoding list Spatial region coding Variables k1 = 0, k2 = 0;

[0168] S2-1-3: The spatiotemporal region coding steps are as follows:

[0169] a) Calculate the distance between the two locations based on their latitude and longitude. Among them lon i and lat iLet L and L represent the latitude and longitude of the i-th location, respectively. The location index starts from i=1 and ends at L-1, where L is the length of the trajectory sequence. i-1 and lat i-1 Let represent the latitude and longitude of the (i-1)th location, respectively.

[0170] Using Haversine formula f H The distance between two check-in locations is calculated using the · symbol. The calculation process is as follows:

[0171]

[0172] b) Calculate the time interval ΔT = |ct| between the timestamps of the user's check-in at the two locations. i -ct i-1 |, where ct i ct represents the timestamp recorded when a user checks in at the i-th location. i-1 This represents the timestamp recorded when the user checks in at location i-1.

[0173] c) Perform threshold judgment operation, when When the time interval is greater than the spatial threshold ΔS, let k1 = (k1+1)%2, and when the time interval is greater than the spatial threshold ΔS, let k1 = (k1+1)%2. When the time threshold ΔT is greater than 2, let k2 = (k2 + 1) % 2; d) Add k1 to the time threshold ΔT. Add k2 to middle.

[0174] S2-1-4: Output the spatial region code for user u User u's time zone encoding

[0175] Specifically, in S2-2, the following is obtained: The steps are as follows:

[0176] will and Combined together The specific calculations are as follows:

[0177]

[0178] in, This represents the trajectory characteristics of u. γ is a hyperparameter. ⊙ represents vector dot addition, ⊙ represents vector dot multiplication, and drop represents vector dot multiplication. sp This represents the dropout operation, used to mitigate overfitting in deep neural networks.

[0179] Using a learnable position embedding matrix The positions between check-ins are marked to avoid losing sequence position information, where Lmax is the maximum length of the trajectory. Then, the final representation of the input trajectory sequence is shown in Equation (12).

[0180]

[0181] in The position embedding matrix represents the learnable parameters.

[0182] Specifically, the steps in S2-3 to obtain the context representation of the user trajectory sequence are as follows:

[0183] A sequence encoder is used to process an input sequence. The context information is modeled and consists of two parts: a multi-head attention module (MHAM) and a pointwise feedforward layer (PWFL). In MHAM, this capability allows DSDRec to simultaneously attend to the context in different representation subspaces (i.e., attention heads) located at different positions in the input sequence. So far, the input has been obtained through the embedding layer. To calculate the attention distribution, three vectors Q, K, and V are constructed as shown in the following formula (13):

[0184]

[0185]

[0186]

[0187] Among them, W Q W K and These are learnable parameters. Next, the output of MHAM is calculated and expressed as:

[0188]

[0189] head i =Attention(QW i q KW i k VW i v (15)

[0190]

[0191] Where [;] represents a join operation. and These are learnable parameters. This is the new latitude after the original latitude d is divided by H, where H is a hyperparameter representing the number of divisions in the multi-head attention mechanism. head1, head2, and head... h These represent the 1st, 2nd, and hth attention modules in the multi-head attention mechanism, respectively. This represents the sequence context features output after processing by the multi-head self-attention module.

[0192] In MHAM, a causal mask was chosen as the attention mask. To avoid network degradation, a residual connection mechanism was used, i.e. Although MHAM can aggregate all historical sequence information using dynamic weights, it remains a linear module. To introduce non-linearity and consider the interactions between different dimensions, [the following is omitted as it is not relevant to the translation]. PWFL was applied, in the following form:

[0193]

[0194] Here, and These are learnable parameters; ReLU(·) represents the modified linear activation function, and Drop(·) represents the dropout operation. This is the final output.

[0195] Specifically, steps S2-4 yield a high-quality contextual representation of the trajectory sequence. The steps are as follows:

[0196] In order to align different modalities and Features, which need to be mapped to the same feature space, then interact to maintain semantic consistency. Let... and These represent the downsampling matrix, the upsampling matrix, and the matrix mapping multimodal features to a unified space, respectively. These are three d-dimensional bias vectors. Note that they are repeated along the 0-dimensional axis. L times, that is Feature fusion operation is represented as:

[0197]

[0198]

[0199]

[0200] Where Φ is the LeakyReLU function with a negative slope of 0.1, and for the MaxPooling operation, the stride is set to 2 and the kernel size is set to 2.

[0201] Specifically, S2-5 obtains As in formula (21):

[0202]

[0203] in, These are learnable parameters.

[0204] Specifically, in S4, when training the encoder-decoder model, Su is split into two parts: As the input sequence (context), As positive samples (the label of each point), where y i =c i +1 indicates the next point in Su. Sampling and Different negative samples for each point in

[0205] Training objectives Defined as:

[0206]

[0207] Among them, O (1) This represents a tensor filled with 1s, with a shape similar to... Same; while O (0) Represents a tensor filled with zeros, with a shape similar to... Same. c i It is the i-th check-in point in the trajectory sequence, z i The negative sample z corresponding to the i-th check-in point i ≠c i , It is S u After processing by the encoder-decoder model, c i The corresponding contextual feature representation, L is the trajectory S u The length of L is the number of check-in points (ci). (Note: L = n here for ease of representation.)

[0208] Specifically, S3 obtains The steps are as follows:

[0209] During the forward process: perform space transformation operations (such as...) Figure 4 As shown, Figure 4 The image illustrates the effect of spatial transformation. Through this transformation, the original feature space is scaled into a noise-robust space. As shown above, for noise-sensitive data distributions, even a small amount of Gaussian noise can disrupt the original distribution, making multi-step operations difficult in diffusion models. The distribution is normalized, and its value range is expanded:

[0210] for set up This represents its mean. Express its variance, and then... Normalize and scale:

[0211]

[0212] Where λ is the hyperparameter of the temperature coefficient, and σ(.) is the variance calculation function.

[0213] Next, randomly select a set of diffusion steps. make For each diffusion step in T^, sample l Gaussian noises. Then, formula (2) is executed to obtain noise data. ε t The t-th diffusion step introduces random noise (following a Gaussian distribution N(0,1)). It is a noise step sample set from which one sample is sampled from T.

[0214] At the end of the back diffusion, its output Performing a spatial inverse transformation operation is specifically represented as follows:

[0215]

[0216] in, It is a location context representation reconstructed through diffusion, where μ represents the average value calculation formula. It integrates the user's short-term movement semantic signals with more abstract interaction information between each interaction.

[0217] Specifically, the steps for training the S4 diffusion model are as follows:

[0218] Training during the positive process:

[0219] The original diffusion model is uncontrollable; that is, as the number of diffusion steps increases, the original data distribution will be disrupted into an isotropic Gaussian distribution, causing the recovered original data to lose the user's personalized preferences. This necessitates semanticizing the user's short-term movement patterns. A diffusion model is introduced as a personalized semantic signal for users to mitigate the loss of personalized preferences caused by the introduction of Gaussian noise. Let... These are trainable parameters used for dimensionality transformation, and are set as follows: The training process in the forward pass is as follows: (This refers to the timestamp embedding matrix in the diffusion stream.)

[0220] S4-1-1: Input: Neural network f θ .

[0221] S4-1-2: Trainable network parameters: Θ={θ,W S E T}

[0222] S4-1-3: First calculate and and along the vector 0th dimension extension Next, align its dimensions.

[0223] S4-1-4: Loop until the model converges:

[0224] a) From the time step set Samples are taken at time step t and random Gaussian noise is sampled based on t. b) Calculate according to formula (2) c) Calculate the eigenvector e of the time step t =E T (t), along vector e t 0th dimension extension align its dimensions; d) and e t e) Concatenate vectors along the last dimension and use the result as model input; e) Calculate the predicted value of random noise. f) Calculate the loss value based on formula (8) And update the model parameters Θ={θ,W} based on the loss value. S E T}

[0225] The training process in the reverse process is as follows:

[0226] The inverse process aims to recover the original data distribution from noisy data. It does not recover the original sequence representation from pure noise, thus preserving some personalized information. εT represents the prior constraints.

[0227] Input: Model parameters Θ = {θ, W} trained via a forward pass. S E T}, T,

[0228] S4-2-1: First calculate and and along the vector 0th dimension extension Next, align its dimensions;

[0229] S4-2-2: Sampling random Gaussian noise

[0230] S4-2-3: Input and ε TAnd calculate using formula (2)

[0231] S4-2-4: Calculate the eigenvector e of the time step in the loop t = T-1, ..., 1: a′) t =E T (t), along vector e t 0th dimension extension Next, align its dimensions; b′) Sample random noise. c′) Predicting noise using a diffusion model d′) Calculate the mean of the data distribution using formula (5). Input is and t;e′) sampled random noise f′) Calculates the data distribution at the t-1 diffusion step using the reparameter technique.

[0232] S4-2-5: Return

[0233] The neural network f used in the diffusion model θ It is a stack of multilayer perceptrons (MLPs) and PWFLs, and the model structure is as follows: Figure 5 As shown, SiLU was chosen as f. θ The activation function in [the context].

[0234] experiment:

[0235] 1. Dataset

[0236] The performance of the DSDRec model was evaluated using real-world datasets from New York City (NYC) and Tokyo (TKY). To ensure data quality, inactive users and locations were filtered out from both datasets, and user records with fewer than 10 check-ins and locations with fewer than 10 visits were removed. Cue engineering was performed using trajectory sequences from NYC and TKY, transforming discrete check-ins into dense text cue sentences (i.e., users' long-term and short-term preferences). Table 2 shows the details for the two datasets. For each user trajectory of length n, training was performed using the initial 1 to n-2 check-ins; (n-2)-nd check-ins were used for validation; and (n-1)-st check-ins were used for testing.

[0237] Table 2 Dataset

[0238]

[0239] 2. Baseline Model

[0240] The following existing models are used as benchmark models for comparison with the model of this invention.

[0241] SERM: This model focuses on predicting the next POI and incorporates semantic features during the prediction process.

[0242] STAN: This is a self-attention-based model that utilizes the relative spatiotemporal information of all check-ins and uses a spatiotemporal attention network to aggregate all relevant access records in the user's trajectory to recommend the next POI.

[0243] SASRec: This recommendation model utilizes a self-attention mechanism and employs transformers as sequence encoders.

[0244] TiSASRec: This model is a sequential approach for recommending the next POI.

[0245] SSE-PT: This is a self-attention-based recommendation model. During training, it introduces random factors by replacing one embedding with another according to predefined probabilities, thereby integrating personalized information.

[0246] BERT4Rec: This is a widely used PLM-based pre-trained recommendation model that uses masked item prediction as a pre-training task to learn contextual information about sequences.

[0247] 3. Evaluation Indicators

[0248] Top-k accuracy metrics, namely ACC@1, ACC@5, ACC@10, and ACC@20, are used to evaluate recommendation performance. These metrics are widely used in recommendation systems. ACC@k measures whether the true result appears in the top k results. The calculation can be expressed as:

[0249]

[0250] Where X is a batch of samples, It is an indicator function.

[0251] Table 3 compares the overall performance of the baseline model and DSDRec on the two datasets. Best results are marked in bold, and second-best results are marked with an underline. % improvement indicates the relative improvement of DSDRec compared to the best baseline result.

[0252] Table 3

[0253]

[0254] 4. Implementation details

[0255] In the experiment, the dimension d = 64, and the dimension of PLM is... The spatial threshold ΔS = 5, the temporal threshold ΔT = 1, the maximum trajectory length Lmax = 100, the number of diffusion steps T = 20, the number of attention heads H = 2, λ = 20, the layer normalization is set to 1e-8, and the selected PLM is BERT base version 2. For all dropout operations, the value is set to 0.1, except for dropsp, which is set to 0.4. In the diffusion model, the noise scheme β is set to a linear noise scheme, starting from β1 = 10. -4 The hyperparameters were linearly increased to βT = 0.02. The training process of DSDRec consisted of two independent phases. Using the Adam optimizer with a learning rate of 1e-2 and γ = 0.6, the first phase lasted 500 epochs with a batch size of 128. In the second phase, using the Adam optimizer with a learning rate of 1e-3 and γ = 0.5, the second phase lasted 200 epochs with a batch size of 64. Results were recorded every 10 epochs, and the best record was selected as the final result. Regarding the baseline models, the hyperparameters remained unchanged from the original settings in their respective papers. Notably, in the case of SERM, a pre-trained version of the GloVe vectors, denoted as glove.6B.50d, was used. 3 .

[0256] 5. Results and Analysis

[0257] (1) Performance Recommendation. Table 3 provides a performance comparison of the DSDRec model with baseline models on the New York (NYC) and Tokyo (TKY) datasets. By evaluating the ACC@k metric, it can be observed that DSDRec performs better on all baseline models except for ACC@5 on the TKY dataset. DSDRec improves ACC@1 by 10.98% on the New York dataset and by 7.07% on the Tokyo dataset, indicating that the model provides more accurate predictions. In addition to modeling the contextual information of the sequence, TiSASRec also focuses on temporal information, outperforming SASRec and SSE-PT, but underperforming DSDRec due to a lack of spatial and semantic features. STAN utilizes spatiotemporal features in the trajectory to extract contextual features. However, it ignores deep semantic knowledge in the trajectory, resulting in performance lower than DSDRec. It can also be observed that BERT4Rec outperforms TiSASRec on the Tokyo dataset, but it does not consider temporal features. Statistical analysis of the two datasets reveals that the data distribution in TKY is more uniform than in NYC, meaning that the time intervals for user check-ins are relatively consistent. This means that, to some extent, the absolute position encoding of a sequence can be aligned with the temporal features of the check-in. Compared to SERM, DSDRec has a significant performance advantage because it only considers the shallower semantic features in the trajectory, ignoring the deeper semantics hidden within them.

[0258] For each sample, the input consists of a check-in sequence from 0 to L, where L represents the total number of check-ins (i.e., points of interest). The x-axis and y-axis represent the positions of the check-ins in the trajectory sequence, with indices from 0 to L. This value corresponds to the attention score, indicating the degree to which the current item contributes to the final prediction. Brighter colors indicate higher attention scores. w / o indicates that long-term preference semantics are not fused. W / indicates that it has been merged. These samples were randomly selected from the New York dataset.

[0259] (2) Analysis of deep semantic features. In the encoder-decoder model, it is used to align with the context representation of the trajectory sequence and is used in the feature fusion layer; In diffusion models, short-term semantic signals from users are used to aid prediction. In this section, the goal is to analyze... and The impact on the model. To analyze... The impact on DSDRec was investigated by randomly selecting three samples from the New York dataset and visualizing the attention score of each sample in the decoder. Figure 6The three graphs at the top show that, without incorporating semantic signals, the recommendation model tends to allocate more attention to the head subsequence. Empirically, a user's current movement behavior is primarily influenced by recent movement preferences, meaning the recommendation model should focus more on the tail subsequence. On the other hand, from... Figure 6 The three corresponding graphs at the bottom layer show a very smooth attention distribution, indicating that the model allocates similar attention to all items in the sequence, which leads to the neglect of some locations that have a significant impact on prediction (i.e., some locations frequently visited by users). Conversely, after fusing the semantic signals, we can observe that: 1) the model allocates more attention to the tail data, indicating that the model can be biased towards focusing on users' recent movement preferences; 2) the attention distribution is uneven, indicating that the model can focus more on some specific locations. To analyze short-term preference semantics... To investigate the impact, a comparative experiment was conducted, namely, removing [something] from the diffusion model. Let's observe its impact on prediction. The results are as follows: Figure 7 As shown, it is clear that fusing semantic signals can further improve model performance. Figure 7 In the text, w / osemantic means to remove.

[0260] (3) Ablation Experiment. An ablation study was conducted to evaluate the impact of each proposed component on the overall model performance, using the New York and Tokyo (TKY) datasets. The results are shown in Table 4. This section focuses on the contributions of three components: 1) the diffusion model, 2) the spatiotemporal region encoding (STA), and 3) the feature fusion layer (FFL). As can be seen from Table 4, the complete model achieved the highest performance, indicating that the above three components made significant contributions to the final prediction. In particular, fusing these three components improved the ACC@1 of the TKY dataset by 13.55%, while improving it by 7.16% in the New York dataset. In both datasets, the stacking of the diffusion model contributed more to ACC@1 and ACC@5 than to ACC@10 and ACC@20. These results suggest that, by incorporating the semantics of users' short-term mobility preferences, the diffusion model can better capture users' short-term preferences, thus providing more accurate predictions.

[0261] In Table 4, '-' indicates 'none'. STA represents spatiotemporal region coding, used to represent spatial and temporal features. MD represents diffusion model. FFL represents feature fusion layer, used to fuse semantic signals. DSDRec★ represents 'DSDRec-DM'.

[0262] Table 4 Ablation experiments of DSDRec on two datasets.

[0263]

[0264] 6. Hyperparameters

[0265] This section aims to analyze the following aspects: 1) the influence of dimension d; 2) the influence of diffusion step number T; and 3) the influence of temperature coefficient λ.

[0266] To determine a suitable value for d, d was selected from [10, 32, 64, 100, 128, 256], and experiments were conducted on two datasets. Notably, for convenience, the performance of DSDRec with the diffusion model removed was recorded in this experiment. The results are as follows... Figure 8 As shown, it can be observed that the model's performance gradually improves with increasing d, reaching its best performance when d is approximately 64, and then slightly decreases. These results also indicate that: a) if the dimensionality is too small, the model will underfit; b) the model's performance is not linearly related to the increase of d; and c) if the dimensionality is too large, there is a risk of overfitting, which will harm the model's performance. Therefore, in this experiment, d = 64 was chosen for both datasets.

[0267] Table 5 shows the impact of hyperparameters T and λ on the New York dataset (*similar results on the TKY dataset). In the inverse process of the diffusion model, to preserve some individual information, the original sequence representation is not recovered from pure noise. This strategy allows for good results with only a few diffusion steps, reducing the computational cost under large diffusion steps. It can be seen that the best results are achieved when T = 20. λ is used to control the scaling of the original data distribution and reduce sensitivity to noise. It can be seen that when λ = 1, the results are poor, severely damaging the performance of DSDRec. λ = 1 means not expanding the original data distribution, and the results demonstrate the effectiveness of the proposed method (spatial transformation and inverse spatial transformation). Generally, the best results are achieved when λ = 20. It is found that when λ >= 40, the diffusion model converges rapidly, and performance no longer improves. This is because when λ is too large, adding noise will no longer disturb the original data distribution, causing the model to lose its randomness and making it difficult to further explore more abstract interaction information in the trajectory sequence.

[0268] In Table 5, HP is an abbreviation for hyperparameter.

[0269] Table 5 shows the relationship between recommendation accuracy and different diffusion steps T and scaling values ​​λ on the New York dataset.

[0270]

[0271] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit it. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical solutions of the present invention without departing from the spirit and scope of the technical solutions of the present invention, and all such modifications or substitutions should be covered within the scope of the claims of the present invention.

Claims

1. A location service recommendation method based on deep semantic extraction and diffusion model, characterized in that: Includes the following steps: Define the user set and the location set as U = (u1, u2, ..., u3) and U4) respectively. N ) and V = (υ1, υ2, ..., υ M The check-in sequence is defined as follows: ), where N represents the total number of users, and M represents the total number of locations or Points of Interest (POIs). A check-in point can be represented as ci = (vid, cid, cate, ct, lon, lat), where vid and cid are the location identifier and category, respectively, cate represents the name of cid, and ct represents the time when the check-in action occurs. The variables lon and lat represent the longitude and latitude of the check-in location, indicating its geographical location. The user's check-in sequence, i.e., the trajectory sequence, is represented by S. u It is represented as S. u = (c1, c2, ..., c n ), where ci represents check-in in chronological order at a specific location, and n represents the timestamp; The location service recommendation model DSDRec is constructed and trained. DSDRec includes a deep semantic feature extraction model, an encoder-decoder model, and a diffusion model. S1: A deep semantic feature extraction model that uses cue engineering to create continuous natural language text from discrete trajectory sequences. This natural language text is called a cue sentence, which includes the user u's long-term preferences. and short-term preferences Using PLM to respectively and The corresponding transformation is into a continuous context feature representation V L and V S ; S2: The encoder-decoder model obtains high-quality place context representations of sequence embeddings and learns trajectory sequences by fusing deep semantics and spatiotemporal information; S2-1: From S u Select the time c when the check-in action occurs at each check-in point. t As a time series S u At each check-in point, the longitude (lon) and latitude (lat) of the check-in location are selected as the spatial sequence. in use Spatial encoding vector, using Represents the time-coded vector; S2-2: Using an embedding layer to obtain a compact representation of Su, let... and These represent the location embedding matrix, spatial embedding matrix, temporal embedding matrix, and location category embedding matrix, respectively; according to E V E S and E T From S u Retrieve location feature representation from location ID from Obtaining time feature representation According to S u The category ID in the data is used to obtain the feature representation of the venue category. Will and The fusion yields the final representation of the input trajectory sequence. in, C represents the number of location categories, L represents the length of Su, and d represents the dimension of the vector. S2-3: Using a sequence encoder The context information is used to model and obtain the context representation of the user trajectory sequence. S2-4: Semantics of long-term mobility preferences and short-term mobility preference semantics Integrate Obtain a contextual representation that integrates the semantics of users' long-term mobile preferences. S2-5: Polymerization Obtain high-quality place context representation S3: Enhanced using a diffusion model Obtain the site context representation reconstructed through diffusion. S4: Training DSDRec begins by minimizing the objective function. Train the encoder-decoder model, then freeze all parameters in the encoder-decoder model, and minimize the objective function. To train the diffusion model; in these two training steps, the parameters of the PLM will not be updated. The training of the diffusion model includes training in the forward process and training in the backward process. S5: For a new user, the user's trajectory sequence is input into the trained DSDRec to obtain the user's location context representation reconstructed through diffusion. The last element is then extracted from the user's location context representation reconstructed through diffusion. Will As a representation of the trajectory of new users, Perform matrix multiplication with the embedding matrix of all locations of the new user, calculate the score as shown in formula (26), and take the location with the maximum score as the next POI recommendation; Among them, E V′ This represents the embedding matrix of all locations for new users.

2. The location service recommendation method based on deep semantic extraction and diffusion model as described in claim 1, characterized in that: The V obtained in S1 L and V S The steps are as follows: A set of text templates was created, and the input slots were filled using the `cate` keyword to construct suggestion sentences; and Describes the user's global movement behavior. and It describes the user's local mobile behavior patterns; for The five most frequently visited location categories by users were analyzed; for Based on CT, the day is divided into two categories: weekdays and weekends, and the three most frequently visited venue categories are analyzed for each category; for The day is divided into three categories based on CT (Category A), and the three most frequently visited place categories by users in each category are analyzed; for Calculate a value τ, representing how many of the user's most recently visited locations were not visited in their historical trajectory; then define a threshold Γ, if τ >= Γ, it indicates that the user's short-term preferences have changed, otherwise they have not changed; for Fill the slots with the five most recently visited place categories by the user; Finally, and The sentences that are connected together to form a prompt for u's long-term preference are denoted as follows: Will and The phrases connected together to form a short-term preference for u are denoted as follows: Using PLM and The transformation is converted into a continuous feature representation, and the calculation of this transformation is as shown in formula (9): in, and It is a tokenized version of the prompt sentence. It is a PLM, and φ is a parameter of the PLM; and They are and After PLM processing, the context feature representation, l L and l s They are and The length.

3. The location service recommendation method based on deep semantic extraction and diffusion model as described in claim 2, characterized in that: The calculation in S2-1 and The steps are as follows: S2-1-1: Input: Column L,Haversine algorithm Spatial threshold Δ S Time threshold Δ T S2-1-2: Initialize the time zone encoding list Spatial region coding Variables k1 = 0, k2 = 0; S2-1-3: The spatiotemporal region coding steps are as follows: a) Calculate the distance between the two locations based on their latitude and longitude. Among them lon i and lat i Let L and L represent the latitude and longitude of the i-th location, respectively. The location index starts from i=1 and ends at L-1, where L is the length of the trajectory sequence. i-1 and lat i-1 Let represent the latitude and longitude of the (i-1)th location, respectively. Using Haversine formula f H The distance between two check-in locations is calculated using the · symbol. The calculation process is as follows: b) Calculate the time interval ΔT = |ct| between the timestamps of the user's check-in at the two locations. i -ct i-1 |, where ct i ct represents the timestamp recorded when a user checks in at the i-th location. i-1 This represents the timestamp recorded when the user checks in at location i-1; c) Perform a threshold judgment operation, when... When the time interval is greater than the spatial threshold ΔS, let k1 = (k1+1)%2, and when the time interval is greater than the spatial threshold ΔS, let k1 = (k1+1)%2. When the time threshold ΔT is greater than 1, let k2 = (k2 + 1) % 2; d) Add k1 to the time threshold ΔT. Add k2 to middle; S2-1-4: Output the spatial region code for user u User u's time zone encoding 4. The location service recommendation method based on deep semantic extraction and diffusion model as described in claim 3, characterized in that: In S2-2, The steps are as follows: will and Combined together The specific calculations are as follows: in, This represents the trajectory characteristics of u. γ is a hyperparameter, ⊕ represents vector dot addition, ⊙ represents vector dot multiplication, and drop... sp This indicates a dropout operation; Using a learnable position embedding matrix The positions between check-ins are marked to avoid losing sequence position information, where Lmax is the maximum length of the trajectory; then, the final representation of the input trajectory sequence is shown in formula (12); in The position embedding matrix represents the learnable parameters.

5. The location service recommendation method based on deep semantic extraction and diffusion model as described in claim 4, characterized in that: The steps in S2-3 to obtain the context representation of the user trajectory sequence are as follows: Construct three vectors Q, K, and V as shown in the following formula (13): Among them, W Q W K and These are learnable parameters; next, the output of MHAM is calculated and expressed as: Where [;] represents a join operation. and These are learnable parameters. It is the new latitude after the original latitude d is divided by H, where H is a hyperparameter, head1, head2, and head h These represent the 1st, 2nd, and hth attention modules in the multi-head attention mechanism, respectively. This represents the sequence context features output after processing by the multi-head self-attention module; In MHAM, a causal mask was chosen as the attention mask; a residual connection mechanism was used, i.e. right Using PWFL, the format is as follows: Here, and These are learnable parameters; ReLU(·) represents the modified linear activation function, and Drop(·) represents the dropout operation. This is the final output.

6. The location service recommendation method based on deep semantic extraction and diffusion model as described in claim 5, characterized in that: S2-4 yields a high-quality contextual representation of the trajectory sequence. The steps are as follows: set up and These represent the downsampling matrix, the upsampling matrix, and the matrix mapping multimodal features to a unified space, respectively. These are three d-dimensional bias vectors; the feature fusion operation is represented as: Where Φ is the LeakyReLU function with a negative slope of 0.

1.

7. The location service recommendation method based on deep semantic extraction and diffusion model as described in claim 6, characterized in that: The S2-5 obtained As in formula (21): in, These are learnable parameters.

8. The location service recommendation method based on deep semantic extraction and diffusion model as described in claim 7, characterized in that: When training the encoder-decoder model in S4, Su is split into two parts: As the input sequence As a positive sample, where y i =c i +1 indicates the next point in Su; sampling and... Different negative samples for each point in Training objectives Defined as: Among them, O (1) This represents a tensor filled with 1s, with a shape similar to... Same; while O (0) Represents a tensor filled with zeros, with a shape similar to... Same; c i It is the i-th check-in point in the trajectory sequence, z i The negative sample z corresponding to the i-th check-in point i ≠c i , It is S u After processing by the encoder-decoder model, c i The corresponding contextual feature representation, L is the trajectory S u The length.

9. The location service recommendation method based on deep semantic extraction and diffusion model as described in claim 8, characterized in that: The S3 obtained The steps are as follows: During the forward process: Perform a space transformation operation on the pair The distribution is normalized, and its value range is expanded: for set up This represents its mean. Express its variance, and then... Perform normalization and scaling: Where λ is the hyperparameter of the temperature coefficient, and σ(.) is the variance calculation function; Next, randomly select a set of diffusion steps. make For each diffusion step in T^, sample l Gaussian noises. Then, formula (2) is executed to obtain noise data. ε t The t-th diffusion step is the introduced random noise. It is a noise step sample set from which l samples are sampled from T; At the end of the back diffusion, its output Performing a spatial inverse transformation operation is specifically represented as follows: in, It is a location context representation reconstructed through diffusion, and μ represents the formula for calculating the average value.

10. The location service recommendation method based on deep semantic extraction and diffusion model as described in claim 9, characterized in that: The steps for training the S4 diffusion model are as follows: Training during the positive process: Semantics of users' short-term mobile patterns Introducing a diffusion model as a personalized semantic signal for users, let... These are trainable parameters used for dimensionality transformation, and are set as follows: The training process in the forward pass is as follows: (This refers to the timestamp embedding matrix in the diffusion stream.) S4-1-1: Input: Neural network f θ ; S4-1-2: Trainable network parameters: Θ={θ,W S E T } S4-1-3: First calculate and and along the vector 0th dimension extension Next, align its dimensions. S4-1-4: Loop until the model converges: a) From the time step set Samples are taken at time step t and random Gaussian noise is sampled based on t. b) Calculate according to formula (2) c) Calculate the eigenvector e of the time step t =E T (t), along vector e t 0th dimension extension align its dimensions; d) and e t Concatenate the vectors along the last dimension and use the result as the model input. e) Calculate the predicted value of random noise f) Calculate the loss value based on formula (8) And update the model parameters Θ={θ,W} based on the loss value. S E T }; The training process in the reverse process is as follows: εT represents the prior constraints; Input: Model parameters Θ = {θ, W} trained via a forward pass. S E T }, T, S4-2-1: First calculate and and along the vector 0th dimension extension Next, align its dimensions; S4-2-2: Sample random Gaussian noise S4-2-3: Input and ε T And calculate using formula (2) S4-2-4: Loop t = T-1, ..., 1: a') Calculate the eigenvector e of the time step. t =E T (t), along vector e t 0th dimension extension Next, align its dimensions; b') Sample random noise. c') Predicting noise using a diffusion model d') Calculate the mean of the data distribution using formula (5). Input is and t;e') sampled random noise f') Calculate the data distribution at the t-1 diffusion step using the reparameter technique. S4-2-5: Return

Images