Location recommendation method based on graph mask and space-time diffusion

Abstract

The invention relates to a place recommendation method MSDRec based on graph masks and space-time diffusion, which comprises two core modules, the first module is a global trajectory flow graph mask self-supervision module, and the module constructs a global trajectory flow graph and carries out a graph mask self-supervision learning method suitable for POI recommendation. The strategy focuses on the core structure information of the reserved graph and deeply recovers the key edge, so that the noise caused by structural damage can be effectively reduced, and the accuracy and robustness of the model during data sparsity processing are improved. And the second module is a space-time diffusion enhancement module which encodes the space-time relation characteristics into low-dimensional embedding by constructing a global space-time transfer graph, and further de-noising and refining are carried out through a diffusion model. The diffusion process is performed within the representation space, including multi-step noise increase and removal. This process handles various noise levels, enhancing the ability of the model in identifying and eliminating noise in spatio-temporal data, thereby optimizing and refining spatio-temporal POI representations.

Description

Technical Field

[0001] The invention relates to a method for recommending points of interest, and in particular to a method for recommending places based on image masking and spatiotemporal diffusion. Background Art

[0002] The surge in the use of location-based social networks (LBSNs) and applications that provide location-based services has stimulated growing interest in point-of-interest (POI) recommendation models. These models are used to provide users with personalized recommendations covering a wide range of services and products. POI-centric recommendation models have become an indispensable feature in various check-in applications such as Yelp, Weibo, and Foursquare, aiming to provide users with places they are likely to visit in the future based on historical data predictions. Such recommendations are crucial in fields such as business analytics, urban planning, and consumer behavior research. However, POI recommendation systems are often limited by the inherent sparsity of user-driven trajectory data and its complex spatiotemporal relationships, which increase the noise interference and uncertainty of recommendation models when processing these data, thereby reducing the accuracy and personalization level of recommendations.

[0003] In the POI recommendation system, two major challenges - data sparsity and difficulty in extracting complex spatiotemporal relationships - jointly restrict the accuracy and generalization of recommendations. Data sparsity mainly stems from the limited interaction between users and places: most users usually only visit a limited number of places, while a large number of places become information-poor due to the lack of sufficient user interaction. This limitation of interaction not only hinders the learning of effective user preferences and place characteristics, but also increases the uncertainty and overfitting risk of the model during training. At the same time, the spatiotemporal relationships between POIs are highly complex in nature. Data sparsity and the low quality of trajectory datasets jointly exacerbate the difficulty of extracting valuable information from these relationships: data sparsity leads to a lack of effective information, while the low quality of trajectory data leads to the extracted spatiotemporal relationships often containing a lot of noise and unreliable information, which further increases the difficulty of understanding these complex spatiotemporal relationships.

[0004] In the existing POI recommendation system, graph neural network and self-supervised learning are the two mainstream methods to solve data sparsity. Many studies have used graph structures to model global user trajectories and extract common mobility patterns to alleviate data sparsity, but this may introduce or aggravate noise problems when dealing with a large number of low-quality nodes and edges. Figure 1 The low check-in POIs p1, p6, etc. and edges p8→p7, p 12 →p 11. For example, some studies have captured a wide range of user behavior patterns by constructing global user trajectory flow graphs. Although these methods can improve the recommendation system, they may also make the noise problem more significant due to the large amount of data from occasional interactions. At the same time, self-supervised learning, as an emerging machine deep learning paradigm, enhances the learning ability of the model through contrast tasks and generative tasks. Although it can be trained without a large amount of labeled data, this method may also introduce additional noise. Although a large number of studies have shown that contrast strategies can effectively extract user preferences, random data augmentation may destroy the structural information of the user sequence structure or the core of the graph structure, such as Figure 1 Randomly cutting out POIs with high frequency of check-ins or reordering POI interaction sequences in the comparison strategy will seriously distort the structural information. Generative self-supervised learning, such as predicting the attributes of masked nodes or edges, helps the model understand the details of the sequence or graph structure, but it also faces the risk of increased noise due to the accumulation of random mask errors, such as Figure 1 Masking a large number of high-check-in POIs in the generative formula will destroy the original structural information. In addition, the incompleteness and uneven spatiotemporal distribution of user-driven trajectory datasets often lead to a lot of noise. These noises directly affect the model's ability to extract effective signals from complex spatiotemporal relationships, especially in data-sparse environments. Although some studies have attempted to capture spatiotemporal relationships through timestamps and geotags, this method mainly relies on simple, linear spatiotemporal correlation analysis, which often fails to deeply explore the complex geographic and temporal dependencies in the data. At the same time, this simplified processing method also fails to effectively distinguish between signals and noise, which may lead to the misinterpretation of noise as effective signals. Summary of the invention

[0005] In view of the above problems existing in the prior art, the technical problem to be solved by the present invention is: the impact of data sparsity on the accuracy of POI recommendation.

[0006] To solve the above technical problems, the present invention adopts the following technical solution: a location recommendation method based on image mask and spatiotemporal diffusion, comprising the following steps: constructing MSDRe c ,MSDRe c It includes global trajectory flow graph mask self-supervised learning module, spatiotemporal diffusion denoising module, global POI representation fusion layer, embedding layer and Transforme r Encoder;

[0007] S1: The global trajectory flow graph mask self-supervised learning module constructs a global trajectory flow graph, identifies and masks key connections through node importance sampling and multi-hop mask edge sampling to obtain a global transfer mask graph. Afterwards, key connections are reconstructed through three self-supervised decoding tasks.

[0008] S1-1: Construct a global trajectory flow graph, given all historical trajectories of global users Define a directed weighted graph as the global trajectory flow graph in represents the node set, that is, the POI set, E m represents the edge set, Ω m Indicates the edge The number of times it appears in .

[0009] S1-2: Node importance sampling, for Any POI node u∈V m , calculate the embedding changes of node u and its k-th order neighbors to get the importance score of node u Pick The top M nodes with the highest scores are added as candidate nodes to the anchor node set V a In, V a =Top M {v|v∈V}, where TopM represents The top M nodes.

[0010] S1-3: Multi-hop masked edge sampling, defining the global sampling process, the edge sampling probability P(E) between any two node pairs is expressed as:

[0011]

[0012] where p u，v represents the sampling probability of the path from node u to node v, p0 represents the basic sampling probability, λ represents the attenuation factor of the control information attenuation, 0<λ≤1, k u，v Represents the number of hops between node u and node v.

[0013] After sampling, we get a global transfer mask map in Indicates that the edge set after removing the masked edge is used as the target of the subsequent decoding task, Represents the edge weight set after removing mask edges.

[0014] S1-4: obtained by GCN-based graph encoder The mask representation H m ,Three decoders including degree decoder, edge existence decoder and edge weight decoder are designed for decoding.

[0015] Degree decoder for prediction The edge existence decoder is used to determine the number of connections for each node in The edge weight decoder is used to predict whether there is an edge between any two nodes in The weights of the edges that already exist in .

[0016] S2: The spatiotemporal diffusion denoising module constructs a global time map and a global space map, and uses a diffusion model to denoise and enhance the spatiotemporal representation to finally obtain the denoised global spatiotemporal data

[0017] S2-1: Construct a global time graph and a global space graph. Both the global time graph and the global space graph are directed weighted graphs obtained by integrating all user trajectories S. reflects a global view of the spatial position transformation pattern, while the global time map The time-position transition patterns are collected, where V s and V t are all POI check-in node sets, E s and E t is the set of weighted edges that connect two different nodes, Ω s and Ω t Represented and The weight of .

[0018] S2-2: Forward diffusion and reverse diffusion of the diffusion model, using the graph convolutional network GCN encoder to process the global time graph and the global spatial graph respectively to independently extract the spatial POI representation H s and time POI representation H t , for H s and H t Fusion is performed to obtain the global spatiotemporal POI representation H st ,

[0019] Diffusion process: In the forward process, Gaussian noise gradually increases to H st In, until H st Finally, it is completely transformed into Gaussian noise E t In contrast, the reverse diffusion process uses a trainable denoiser to remove noise, and a trained diffusion model is obtained through a continuous iterative diffusion process.

[0020] S2-3: Utilization of diffuse reasoning process For global spatiotemporal information H st Denoising

[0021] S3: Global POI representation fusion layer, which will get Hm and Fusion is performed to obtain Hf.

[0022] S4: Embedding layer, each POI are all assigned a final embedding h obtained by the mask module and the diffusion enhancement module of the present invention.p ∈H f Any user Historical access sequence Initial embedding of Obtained by:

[0023]

[0024] Among them, h l Indicates user Access Sequence Check-in location The embedding representation of p l express The location of the first check-in location is embedded in the .

[0025] S5: Recommending users using Transformer encoder Final sequence preference embedding according to Score the next possible POI and make a prediction.

[0026] Use a self-supervised method to train MSDRec. When no longer changing, the model's current parameters are retained.

[0027] S6: For a new user z, obtain z’s historical trajectory and will After inputting the optimal MSDRec after training, the optimal MSDRec will be calculated based on its historical trajectory. Calculate the trajectory preference of z And the embedding representation H of all N POIs f =(h1, h2, ..., h n ), and then calculate Get the preference scores of all POIs Then Score N The POI with the highest score is the POI recommended by the new user z.

[0028] Furthermore, in S1-2, the importance score is calculated The process is as follows:

[0029]

[0030] Sub k (u) represents the k-order subgraph of node u. For any node v in the u subgraph, represents its embedding representation at time step t, and ||·||2 represents the Euclidean norm.

[0031] Further, in S1-4, we obtain The mask representation H m The process is as follows:

[0032] right The adjacency matrix of Perform Laplace normalization, where N represents the number of POIs, and normalization The calculation process is as follows:

[0033]

[0034] Where D is is the degree matrix of , and I is the identity matrix.

[0035] The standardized multi-layer graph convolutional network GCN is used Encoding, each layer of GCN updates the node embedding by neighborhood aggregation, and the formula is:

[0036]

[0037] in, is the node feature matrix of the lth layer, is the weight matrix of the lth layer.

[0038] The node embeddings output by each layer are accumulated to obtain the final representation of node u:

[0039]

[0040] in, represents the final mask representation of node u, is the representation of node u after the l-th layer of graph convolution, so The final mask is denoted as H m , any node u∈V m , whose embedding representation

[0041] Furthermore, in S1-4, the decoding process of the three decoders is as follows:

[0042] Degree Decoder: For Any node u∈V m , define the degree decoding as follows:

[0043]

[0044] in, Represents the degree prediction result of node u, Decoder D It stands for node degree decoder, ReLU is the activation function, and MLP is multi-layer perceptron.

[0045] Edge existence decoder: For The mask edge (u, v) in is represented by and Use MLP to process the embedding of node pairs and use the Sigmoid activation function to predict the probability of the existence of the edge:

[0046]

[0047] in, Represents the prediction of the existence of edge (u, v), Decoder E represents the edge existence decoder, and ⊙ is the element-wise product, expressing the strength of the interaction between nodes.

[0048] Edge weight decoder: For The mask edge (u, v) in is represented by and Use MLP to process the embedding of node pairs, and choose to use the ReLu activation function to predict the weight of the edge:

[0049]

[0050] in, Represents the prediction of the edge (u, v) weight, Decoder E represents the edge weight decoder and ⊙ represents the element-wise product.

[0051] Furthermore, in S2-1, Ω is constructed s and Ω t The process is as follows:

[0052] Traverse all tracks If two POI and If they appear continuously in the sequence, it means that a position transition has occurred. Collect the position transitions in all user trajectories, evaluate the spatial and temporal intervals, and construct and

[0053]

[0054] in Represents any edge (u, v) Weight, Represents any edge (u, v) Weight, Δd represents the spatial interval threshold, represents the time interval threshold, Respectively represent users The spatial interval and time interval of the transfer from node i to node ,.

[0055] Furthermore, in S2-2, the forward diffusion and reverse diffusion processes of the diffusion model are as follows:

[0056] During the forward diffusion process, T noise steps are applied iteratively, where T is a hyperparameter and the data at step t is denoted by E t , the data of step 0 is the original data, that is, let E0 = H st .

[0057] Use one in s min to max The linear interpolation sequence s between two hyperparameters generates β t :

[0058] β t =1-s t / s t-1 , s=(1,s max ,......,s min ) (twenty three)

[0059] E t It is obtained by the following formula:

[0060]

[0061] in, Reparameterization

[0062] Reverse Diffusion Process: Use a learnable neural network to estimate the following conditional probabilities:

[0063] p(E t-1 |E t )=N(E t-1 ;μ θ (E t , t), ∑ θ (E t , t)) (25)

[0064] Among them, μ θ (·),∑ θ (·) represents a neural network with learnable parameters θ, which is used to estimate the parameters of the Gaussian distribution. The embedding at the tth time step is concatenated with the embedding at a specific time step as input. The network consists of two fully connected layers:

[0065] μ θ (e t , t) = FC 2 (e t |h t), FC(x)=δ(Wx+b) (26)

[0066] Among them, FC 2 Represents two consecutive fully connected layers. represents the node embedding vector in the t-th step diffusion, and represents the embedding at the tth time step. δ(·), W, and b refer to the activation function, the weight of the linear transformation, and the bias vector of the fully connected layer, respectively.

[0067] Further, in S2-3, we obtain The process is as follows:

[0068] The spatiotemporal information enhancer is based on H st As input, skip the forward noise diffusion process, and in the denoising step, update To iterate the diffusion enhancement, The definition is as follows:

[0069]

[0070] according to and t and use To predict use For space-time information H st Denoising is performed to obtain a denoised spatiotemporal representation

[0071] Further, in S3, H f The process is as follows:

[0072]

[0073] Where W represents the learnable aggregation matrix,

[0074] Furthermore, in S5, Transforme r The multi-head attention mechanism in the encoder is as follows:

[0075] The formula of the multi-head self-attention module is expressed as follows:

[0076]

[0077] Where L and H represent the number of multi-head attention modules and heads, respectively. h Represents the score of the hth head. Indicates user In the feature representation at layer l, is the projection matrix of the h-th head corresponding to the query, key, and value in the attention mechanism. Concat(·) is the concatenation operation. Representative User The final representation of layer l. W1, W2, b1, b2 form a feedforward neural network. The final sequence embedding is obtained through residual connection

[0078] Furthermore, in S5, the loss The calculation process is as follows:

[0079] The degree decoder is optimized by the mean square error loss function. The degree decoding loss for:

[0080]

[0081] in, represents the degree decoder's prediction of the node u, D u is the actual degree of the node, and N is the total number of nodes in the graph.

[0082] The edge existence decoder uses a binary cross entropy loss function, the edge existence decoding loss for:

[0083]

[0084] Among them, E neg Represents the negative sample set of the selected non-existent edges, represents the actual existence score of edge (u, v), is the negative sample score.

[0085] The edge weight decoder uses the mean square error loss function to optimize the weight decoding loss for:

[0086]

[0087] in, is the weight of edge (u, v) predicted by the edge weight decoder, is the actual weight of the edge, |E| is The total number of sides.

[0088]

[0089] Among them, log p(e0) is ELBO, is the reconstruction term, p θ (e t |e0) is the trained denoiser, q(e T |e0) is the true distribution, and T is the pre-set diffusion step hyperparameter.

[0090] The purpose of the denoising matching term is to minimize the true distribution q(e t-1 |e t , e0) and denoiser p θ (e t-1 |e t ) is the KL divergence between .

[0091] The denoised match term at the t-th time step is defined as follows, It is expressed as:

[0092]

[0093] in, represents the prediction of the initial embedding e0, and the reconstruction term is represented as the predicted value The square loss between the real embedding vector e0, i.e., the diffusion loss The definition is as follows:

[0094]

[0095] For Users Optimizing the loss for recommendation tasks Defined as:

[0096]

[0097] in Represents the user u trajectory sequence Embedded, Represents real sign-in of embedding, and A randomly selected user Negative samples that have not been visited;

[0098]

[0099] Compared with the prior art, the present invention has at least the following advantages:

[0100] The present invention provides a novel POI recommendation method MSDRec, which combines self-supervised learning based on graph masks and spatiotemporal diffusion technology to alleviate the sparsity of user-driven trajectory datasets and accurately extract complex spatiotemporal relationships therein. This method effectively integrates the core structural information in the global trajectory graph with refined spatiotemporal embedding to ensure robust and accurate recommendations in sparse and noisy data environments. Through comprehensive experiments on real-world datasets, it is demonstrated that MSDRec surpasses existing advanced models. By integrating innovative components - graph mask self-supervised learning and spatiotemporal diffusion technology, MSDRec shows special advantages in dealing with sparse data and noise problems, and can provide deeper and more accurate recommendations. BRIEF DESCRIPTION OF THE DRAWINGS

[0101] Figure 1 Modeling global policies and self-supervised methods can introduce or exacerbate additional noise.

[0102] Figure 2 This is a diagram of the overall model architecture of the method of the present invention.

[0103] Figure 3 is the spatiotemporal diffusion process in the implicit space.

[0104] Figure 4 Figure 5 is an experimental diagram of the number of anchor nodes hyperparameter, where (a) and (b) respectively represent the performance changes of MSDRec under different parameters in the NYC and TKY datasets, where the horizontal axis represents the value range of the number of anchor nodes, and the vertical axis represents the values ​​of HR@10 and NDCG@10.

[0105] Figure 5 This is a mask step hyperparameter experiment, where (a) and (b) respectively show the performance changes of MSDRec under different parameters in the NYC and TKY datasets, where the horizontal axis represents the value range of the mask step number, and the vertical axis represents the values ​​of HR@10 and NDCG@10.

[0106] Figure 6 This is a mask depth hyper-parameter experiment, where (a) and (b) respectively show the performance changes of MSDRec under different parameters in the NYC and TKY datasets, where the horizontal axis represents the value range of mask depth, and the vertical axis represents the values ​​of HR@10 and NDCG@10.

[0107] Figure 7 It is a hyper-parameter experiment of the number of diffusion steps, where (a) and (b) respectively show the performance changes of MSDRec under different parameters in the datasets NYC and TKY, where the horizontal axis represents the value range of the number of diffusion steps, and the vertical axis represents the values ​​of HR@10 and NDCG@10.

[0108] Figure 8 This is an experiment on the anti-sparseness of MSDRec, where (a) and (b) respectively show the performance changes of MSDRec under different sparsity of NYC and TKY, where the horizontal axis represents the degree of sparsity of the dataset, and the vertical axis represents the value of NDCG@20.

[0109] Fig. 9 Figure 1 is the anti-noise experiment of MSDRec, where (a) and (b) respectively represent the performance changes of MSDRec under the influence of different noises of NYC and TKY, where the horizontal axis represents the degree of noise interference of the data set, and the vertical axis represents the value of NDCG@20. DETAILED DESCRIPTION

[0110] The present invention is described in further detail below.

[0111] See also Figure 2 , a place recommendation method based on graph mask and spatiotemporal diffusion, MSDRec, consists of two core modules: (1) the global trajectory flow graph mask self-supervision module, which proposes an adaptive masking strategy for POI recommendation and designs three self-supervised reconstruction tasks to deeply mine the high-order structural information of the graph. (2) The spatiotemporal diffusion denoising module, which constructs a global spatiotemporal graph and applies a diffusion model to identify and eliminate noise from the encoded spatiotemporal representation. The two modules work together to effectively cope with the challenges of data sparsity and complex spatiotemporal relationships, and enhance the robustness of the recommendation system. Experimental results on two real datasets show that our proposed MSDRec model outperforms recent sequential recommendation models.

[0112] 1. Problem definition: We assume is the user set, where N is the number of users. is the set of POI locations, M is the number of locations. And is a timestamp set. M, N, K are positive integers. Represents a detailed description of the i-th location, including its ID, longitude, and latitude information.

[0113] Define sign-in, which is a triple Indicates that user u visited

[0114] user The travel trajectory is defined as The goal of the next POI recommendation task is to The historical trajectory Recommend the location with the highest probability The goal is to predict the next location a user is most likely to visit based on their past interaction history. To maximize the likelihood of predicting the next location, we minimize the negative log-likelihood of the true next location given the user’s historical trajectory. is the probability distribution of the possible next positions, the optimization objective can be expressed as follows:

[0115]

[0116] where θ represents the model parameters used for next location prediction.

[0117] 2. The diffusion model consists of two components: forward and reverse processes:

[0118] The goal of the forward process is to inject Gaussian noise into the original data. Given a data sample x0, the diffusion model will gradually add Gaussian noise of different scales to it, and after T steps, x T Specifically, from x t-1 to x t The formula for adding noise is:

[0119]

[0120] where t = {1, 2, ..., T} is the current step, β t ∈(0, 1) is the noise scale at step t, I is the identity matrix, is a Gaussian distribution, indicating that x t is sampled from this distribution. Based on the additivity of independent Gaussian noise and the reparameterization technique, x can be directly calculated from x0 t :

[0121]

[0122] in α s =1-β s .

[0123] The inverse process aims to transform the noisy data x into T Transform to x0, following the sequence x T →x T-1 →x T-2 →……→x0. According to, when q(x t |x t-1 ) conforms to the Gaussian distribution and β t If it is small enough, the distribution q(x t-1 |x t ) also exhibits Gaussian properties. Therefore, a neural network can be used to predict the inverse distribution:

[0124]

[0125] Where θ is the parameter of the neural network, μ θ (x t , t) and ∑ θ (x t , t) are the mean and covariance matrix of the Gaussian distribution respectively.

[0126] In order to train the diffusion model, it is crucial to obtain reliable μ θ (x t , t) and ∑ θ (x t , t) value to guide the reverse process to achieve accurate denoising. To this end, it is necessary to optimize the model to predict the denoising distribution p θThe variational lower bound (VLB) of the negative log-likelihood of (x0) is:

[0127]

[0128] in,

[0129]

[0130] Among them, D KL (·) is the KL divergence, t∈{1, 2, …, T-1}. Since there are no learnable parameters in the forward process, L can be ignored during training. T , and L0 represents the negative logarithmic probability of the original data sample x0 given the first step noise data x1. T The goal is to distribute p θ (x t |x t+1 ) and the computable posterior distribution q(x) in the reverse process t |x t+1 , x0).

[0131] A place recommendation method based on graph mask and spatiotemporal diffusion includes the following steps: constructing MSDRe c ,MSDRe c It includes global trajectory flow graph mask self-supervised learning module, spatiotemporal diffusion denoising module, global POI representation fusion layer, embedding layer and Transforme r Encoder.

[0132] S1: The global trajectory flow graph mask self-supervised learning module constructs a global trajectory flow graph, identifies and masks key connections through node importance sampling and multi-hop mask edge sampling to obtain a global transfer mask graph. Afterwards, the key connections are reconstructed through three self-supervised decoding tasks. This allows for deep mining of deeper structural information in the trajectory flow graph. This approach can not only enhance the model’s ability to learn from low-quality data samples, but also improve the overall generalization and robustness of the recommendation system.

[0133] S1-1: Construct a global trajectory flow graph, given all historical trajectories of global users Define a directed weighted graph as the global trajectory flow graph in represents a node set, i.e., a POI set, and Em represents an edge set. If a user visits a certain trajectory sequence and Right now User appears The trajectory Then in the trajectory flow graph, there will be nodes and Draw a directed edge between them. Count all the trajectories of global users to get their edge set E m ,Ω m Indicates the edge The number of times it appears in . Any edge Weight equal exist The number of times it appears in all trajectories.

[0134] In this way, the global trajectory flow graph effectively constructs the spatial pattern and fluidity of user behaviors, providing rich structured information for subsequent graph mask self-supervised learning.

[0135] S1-2: Node importance sampling. In the global trajectory flow graph, correctly identifying key nodes is a prerequisite for key edge discovery, which is crucial to reducing the noise impact caused by a large number of occasional interaction edges. Based on this, the present invention proposes a GCN-based node importance sampling method, which aims to accurately capture the POI embedding changes within the node and its high-order subgraph, and accurately evaluate the importance of the node in the graph structure. This method not only quantifies the embedding changes of the node and its direct and indirect neighbors in the global trajectory flow graph, but also extends to the high-order connections of the node to capture its influence in the entire network.

[0136] for Any POI node u∈V m , calculate the embedding changes of node u and its k-th order neighbors to get the importance score of node u Pick The top M nodes with the highest scores are added as candidate nodes to the anchor node set V a In, V a =Top M {v|v∈V}, where Top M express The top M nodes.

[0137] Specifically, calculate the importance score The process is as follows:

[0138]

[0139] Sub k (u) represents the k-order subgraph of node u. For any node v in the u subgraph, represents its embedding representation at time step t, and ||·||2 represents the Euclidean norm. This method not only reveals the core role of certain POI nodes in the graph structure, but also indirectly identifies key edges by focusing on important nodes, thereby alleviating the data noise caused by minor interactions to a certain extent.

[0140] S1-3: Multi-hop mask edge sampling, after sampling the anchor node set V a Finally, the present invention proposes a path-based multi-hop masking edge sampling strategy to capture the cross-sequence important transition relationship between nodes in the global trajectory flow graph while retaining the core structure information of the graph as much as possible. Specifically, the multi-hop masking edge sampling strategy includes two main stages: node extension and hierarchical sampling. In the node extension stage, the present invention selects the anchor node set V from the anchor node set V. a Starting from the above, we recursively expand its connection range, so that the sampled nodes can: 1) not only cover the short-term and long-term relationships within a single user sequence, but also capture the potential connections between global users across sequences; we construct the k-hop neighborhood set ρ of the node k .

[0141] From V a Start by recursively expanding V a Connect the range, build node V a The k-hop neighboring node set ρ k , the construction process is described as:

[0142]

[0143] Among them, ρ k-1 That is, the set of k-1 hop neighbor nodes, Representation and ρ k-1 The set of nodes to which the node is directly connected.

[0144] In the hierarchical sampling stage, the present invention uses a random walk strategy to carefully select transfer paths from the extended multi-hop node set. The path sampling probability p from node u to node v is u，v As the number of hops k between nodes decreases exponentially, it is expressed as follows:

[0145] p u，u =p0·λ k-1 (9)

[0146] Among them, p0 is the basic sampling probability, λ is the attenuation factor that controls information attenuation (0<λ≤1), and k is the minimum number of hops from node u to node v. This strategy ensures that edges close to the anchor node have a higher sampling probability, while the sampling probability of distant edges decreases with the increase of hops, simulating the natural attenuation of information propagation in the network, thereby highlighting the key cross-sequence transfer paths in the graph.

[0147] Define the global sampling process, ρ k The edge sampling probability P(E) between any two node pairs is expressed as:

[0148]

[0149] where p u，vrepresents the sampling probability of the path from node u to node v, p0 represents the basic sampling probability, λ represents the attenuation factor of the control information attenuation, 0<λ≤1, k u，v represents the number of hops between node u and node v. The edges are sampled and the final set of sampled edges is determined by multiplying the probabilities of each candidate edge pair.

[0150] After sampling, we get a global transfer mask map in Indicates that the edge set after removing the masked edge is used as the target of the subsequent decoding task, Represents the edge weight set after removing masked edges. These key edges are subsequently reconstructed through the decoder, thereby strengthening the model's ability to model important structural relationships in the graph. When restoring key edges, the model can better capture hidden structural patterns and complex node interactions, enabling the model to effectively capture complex dependencies between nodes, thereby improving the ability of graph neural networks to capture key information in graph structures.

[0151] S1-4: obtained by GCN-based graph encoder The mask representation Hm is taken as the image, and three decoders, namely, degree decoder, edge existence decoder and edge weight decoder, are designed for decoding. The goal of these decoders is to reconstruct and predict key graph structure features. The joint design of these decoders enables the model to not only learn more comprehensive and fine-grained graph structure information, but also optimize its characterization of common behaviors and preferences of global users.

[0152] Degree decoder for prediction The edge existence decoder is used to determine the number of connections for each node in The edge weight decoder is used to predict whether there is an edge between any two nodes in The weights of the edges that already exist in .

[0153] Specifically, get The mask representation H m The process is as follows:

[0154] In the generation Afterwards, the present invention Encode to fully capture the structural features and provide high-quality input support for the subsequent reconstruction of key side information. The adjacency matrix of Laplace normalization is performed, where N represents the number of POIs, to ensure that the information is The transmission between the nodes is more balanced. Standardization The calculation process is as follows:

[0155]

[0156] Where D is The degree matrix is ​​, and I is the unit matrix. After Laplace normalization, the model can relatively reduce the impact of degree differences on information propagation and achieve smoother information flow.

[0157] The standardized multi-layer graph convolutional network GCN is used Encoding, each layer of GCN updates the node embedding by neighborhood aggregation, and the formula is:

[0158]

[0159] in, is the node feature matrix of the lth layer, is the weight matrix of the lth layer. This multi-layer convolution design enables the node embedding in each layer to not only retain its own characteristics, but also gradually integrate the structural information of multi-order neighbors, thereby enhancing the model's ability to capture the position and relationship of nodes in the graph.

[0160] In order to obtain the final node representation, the present invention accumulates the node embeddings output by each layer, and obtains the final representation of node u as:

[0161]

[0162] in, represents the final mask representation of node u, is the representation of node u after the l-th layer of graph convolution, so The final mask is denoted as H m , any node u∈V m , whose embedding representation

[0163] A residual connection is added in the last layer to alleviate the gradient vanishing problem. Specifically, the present invention sums up the representations of all hidden layers to obtain the final embedding of node u Finally, the mask representation H of the global POI is obtained m With this GCN-based graph encoder, the proposed model is able to encode the structural information of the mask graph into item embeddings for recommendation and subsequent reconstruction of mask edges.

[0164] Specifically, the decoding process of the three decoders is as follows:

[0165] Degree decoder: The degree decoder predicts the number of connections of each node, that is, the degree of each node in the graph. The degree of the node reflects the importance and centrality of the POI in the user's trajectory. Any node u∈V m , define the degree decoding as follows:

[0166]

[0167] in, Represents the degree prediction result of node u, Decoder D It stands for node degree decoder, ReLU is the activation function, and MLP is multi-layer perceptron.

[0168] Edge Existence Decoder: The edge existence decoder is responsible for determining whether there is an edge between any two nodes in the graph. This is the basic part of graph structure decoding and is crucial for correctly simulating and capturing global user behavior patterns. The mask edge (u, v) in is represented by and Use MLP to process the embedding of node pairs and use the Sigmoid activation function to predict the probability of the existence of the edge:

[0169]

[0170] in, Represents the prediction of the existence of edge (u, v), Decoder E represents the edge existence decoder, and ⊙ is the element-wise product, expressing the strength of the interaction between nodes.

[0171] Edge Weight Decoder: The edge weight decoder is used to predict the weight of existing edges in the graph. The weight represents the frequency of global users’ transfer from one POI to another or the overall preference strength. This is particularly important for capturing the behavior patterns and preferences of global users. The mask edge (u, v) in is represented by and Use MLP to process the embedding of node pairs. The difference is that since the weight of the edge ω ≥ 1, the ReLu activation function is chosen to predict the weight of the edge:

[0172]

[0173] in, Represents the prediction of the edge (u, v) weight, Decoder E represents the edge weight decoder and ⊙ represents the element-wise product.

[0174] Through three self-supervised learning objectives, the model can learn high-quality graph representations without explicit labels, and can show obvious advantages when processing sparse interaction data. For example, the edge existence decoder reveals the underlying patterns of user behavior; the edge weight decoder reflects the strength of user preferences; and the degree decoder provides insights into key nodes in the network. These decoding tasks together improve the model's understanding of the global trajectory flow graph, enabling it to better capture the potential behavior patterns of users between POIs.

[0175] Loss function: The loss functions of three self-supervised learning tasks are integrated to comprehensively learn the global trajectory flow graph. First, the degree decoder is optimized by the mean square error loss function. The function measures the deviation between the node degree predicted by the decoder and the true degree:

[0176]

[0177] in, represents the degree decoder's prediction of the node u, D u is the actual degree of the node, and N is the total number of nodes in the graph.

[0178] The edge existence decoder uses a binary cross entropy loss function, the edge existence decoding loss This is useful for predicting the presence or absence of an edge in a graph:

[0179]

[0180] Among them, in order to accelerate the optimization, the present invention adopts negative sampling, E neg Represents the negative sample set of the selected non-existent edges, represents the actual existence score of edge (u, v), is the negative sample score.

[0181] The edge weight decoder predicts the weights of the edges in the graph and uses the mean square error loss function to optimize the weight decoding loss. To ensure the accuracy of the prediction weights:

[0182]

[0183] in, is the weight of edge (u, v) predicted by the edge weight decoder, is the actual weight of the edge, |E| is The total number of sides.

[0184] In this way, the model of the present invention is not only able to learn the connectivity and structure of nodes, but also to optimize the learning process through these detailed loss function feedbacks.

[0185] S2: The spatiotemporal diffusion denoising module constructs a global time map and a global space map, and uses a diffusion model to denoise and enhance the spatiotemporal representation to finally obtain the denoised global spatiotemporal data

[0186] S2-1: Construct a global time graph and a global space graph. Both the global time graph and the global space graph are directed weighted graphs obtained by integrating all user trajectories S. reflects a global view of the spatial position transformation pattern, while the global time map The time-position transition patterns are collected, where V s and V t are all POI check-in node sets, E s and E t is the set of weighted edges that connect two different nodes, Ω s and Ω t Represented and The weight of .

[0187] Specifically, construct Ω s and Ω t The process is as follows:

[0188] Traverse all tracks Similar to the construction of the global trajectory flow graph, if two POIs and If they appear continuously in the sequence, it means that a position transition has occurred. Collect the position transitions in all user trajectories, evaluate the spatial and temporal intervals, and construct and

[0189]

[0190] in Represents any edge (u, v) Weight, Represents any edge (u, v) Weight, Δd represents the spatial interval threshold, represents the time interval threshold, Respectively represent users The spatial interval and time interval of the transfer from node i to node , set two spatial interval thresholds Δd1 = 1 km and Δd2 = 100 km, and two time interval thresholds and To further construct the adjacency matrix of the spatial and temporal graphs.

[0191] S2-2: Forward diffusion and reverse diffusion of the diffusion model. After constructing the global time graph and the global space graph, the present invention processes the global time graph and the global space graph respectively by applying the graph convolution network GCN encoder to independently extract the spatial POI representation H s and time POI representation H t The powerful capability of GCN can be used to capture high-order dependencies in spatial and temporal dimensions by aggregating the neighborhood features of each POI node. Next, the present invention fuses the global spatiotemporal POI representation to obtain Therefore, the diffusion model is used to denoise the spatiotemporal information, such as Figure 3 shown.

[0192] The global temporal graph and the global spatial graph are processed separately using the graph convolutional network (GCN) encoder to independently extract the spatial POI representation H. s and time POI representation H t , for H s and H t Fusion is performed to obtain the global spatiotemporal POI representation H st ,

[0193] Diffusion process: Based on H st , the present invention designs the forward and reverse processes of diffusion in the implicit space. In the forward process, Gaussian noise gradually increases to H st In, until H st Finally, it is completely transformed into Gaussian noise E t In contrast, the reverse diffusion process uses a trainable denoiser to remove noise, and a trained diffusion model is obtained through a continuous iterative diffusion process.

[0194] Specifically, the forward diffusion and reverse diffusion processes of the diffusion model are as follows:

[0195] During the forward diffusion process, T noise steps are applied iteratively, where T is a hyperparameter and the data at step t is denoted by E t , the data of step 0 is the original data, that is, let E0 = H st The data of step t is calculated from the data of step t-1 as follows:

[0196]

[0197] in represents a Gaussian distribution, with parameter β t Control the size of the noise, β t As t=1,2,……,T increases, E tgradually converges to completely Gaussian noise. This property allows the noise diffusion process of the present invention to cover a wide range of noise levels in the data. min to max The linear interpolation sequence s between two hyperparameters generates β t :

[0198] β t =1-s t / s t-1 , s=(1,s max ,......,s min ) (twenty three)

[0199] Due to the additive nature of the Gaussian distribution, the data for step t can be obtained using only the t The E0 and precomputed values ​​of the serial correlation are directly calculated. This greatly speeds up the forward process by avoiding iterative calculations. t It is obtained by the following formula:

[0200]

[0201] Among them, α t =1-β t , Reparameterization

[0202] Reverse Diffusion Process: Use a learnable neural network to estimate the following conditional probabilities:

[0203] p(E t-1 |E t )=N(E t-1 ;μ θ (E t , t), ∑ θ (E t , t)) (25)

[0204] Among them, μ θ (·),∑ θ (·) represents a neural network with learnable parameters θ, which is used to estimate the parameters of the Gaussian distribution. The embedding at the tth time step is concatenated with the embedding at a specific time step as input. The network consists of two fully connected layers:

[0205] μ θ (e t , t) = FC 2 (e t |h t ), FC(x)=δ(Wx+b) (26)

[0206] Among them, FC 2 Represents two consecutive fully connected layers. represents the node embedding vector in the t-th step diffusion, and represents the embedding at the tth time step. δ(·), W, and b refer to the activation function, the weight of the linear transformation, and the bias vector of the fully connected layer, respectively.

[0207] Loss function: The learnable denoising process of latent space spatiotemporal information diffusion is optimized by maximizing the evidence lower bound (ELBO) of the input spatiotemporal information. This ELBO term can be decomposed as follows:

[0208]

[0209] Among them, logp(e0) is ELBO, is the reconstruction term, p θ (e t |e0) is the trained denoiser, q(e T |e0) is the true distribution, and T is the pre-set diffusion step hyperparameter.

[0210] Since the prior matching term is a constant, it can be omitted in the loss function for optimization. The purpose of the denoised matching term is to minimize the true distribution q(e t-1 |e t , e0) and denoiser p θ (e t-1 |e t ). Based on previous work, the present invention simplifies the learning process by omitting the learning of the standard deviation network and assuming that ∑ θ (e t , t) = σ 2 (t)I.

[0211] The denoised match at the t-th time step is defined as follows:

[0212]

[0213] Among them, μ θ (e t , t) represents the embedding vector e t The average predictor output, μ(e t , e0,t) represents the average of the true probabilities. These average terms can be decomposed into t , e0 and the prediction network of the real data e0 The output of It is expressed as:

[0214]

[0215] in, represents the prediction of the initial embedding e0, and the reconstruction term is represented as the predicted value The square loss between the real embedding vector e0, i.e., the diffusion loss The definition is as follows:

[0216]

[0217] S2-3: Utilization of diffuse reasoning process For global spatiotemporal information H st Denoising

[0218] Specifically, get The process is as follows:

[0219] The spatiotemporal information enhancer is based on H st As input, skip the forward noise diffusion process, and in the denoising step, update To iterate the diffusion enhancement, The definition is as follows:

[0220]

[0221] according to and t and use To predict use For space-time information H st Denoising is performed to obtain a denoised spatiotemporal representation

[0222] S3: Global POI representation fusion layer, which will get H m and Fusion to obtain H f .

[0223] Specifically, we get H f The process is as follows:

[0224]

[0225] Where W represents the learnable aggregation matrix,

[0226] S4: Embedding layer, each POI are all assigned a final embedding h obtained by the mask module and the diffusion enhancement module of the present invention. p ∈H f In the Transformer, each item at position i in the input sequence has a corresponding learnable position embedding p i , added to the initial embedding. Any user Historical access sequence Initial embedding of Obtained by:

[0227]

[0228] Among them, h represents the user Access Sequence Check-in location The embedding representation, pl represents The location of the first check-in location is embedded in the .

[0229] S5: Recommending users using Transformer encoder Final sequence preference embedding according to Score the next possible POI and make predictions. c Perform training, and when the loss L no longer changes, keep the current parameters of the model.

[0230] Specifically, Transforme r The multi-head attention mechanism in the encoder is as follows:

[0231] In Transforme r In the multi-head self-attention module, the multi-head self-attention module optimizes the processing ability of sequence data and is its key component. The formula of the multi-head self-attention module is expressed as follows:

[0232]

[0233] Where L and H represent the number of multi-head attention modules and heads, respectively. h Represents the score of the hth head. Indicates user In the feature representation at layer l, is the projection matrix of the h-th head corresponding to the query, key, and value in the attention mechanism. Concat(·) is the concatenation operation. Represents the final representation of user u at layer l. W1, W2, b1, b2 form a feedforward neural network. The final sequence embedding is obtained through residual connection To enhance model performance, layer normalization and attention value dropout are applied on the input of each block.

[0234] Specifically, loss The calculation process is as follows:

[0235] In order to predict the interaction probability between the user and the candidate POI, the present invention obtains the final embedding of the user sequence from the multi-head self-attention block and combines this embedding with the target POI Embed To improve the generalization performance of the model, the present invention performs negative sample sampling and uses the cross entropy loss function to optimize the recommendation task. Specifically, for users Optimizing the loss for recommendation tasks Defined as:

[0236]

[0237] in Represents the user u trajectory sequence Embedded, Represents real sign-in of embedding, and A randomly selected user In addition, in order to further enhance the recommendation effect, the present invention combines the self-supervisory signal and integrates the signal generated by the graph mask autoencoder through a multi-task learning framework to optimize the model training process.

[0238]

[0239] S6: For a new user z, obtain z’s historical trajectory and will After inputting the optimal MSDRec after training, the optimal MSDRec will be calculated based on its historical trajectory. Calculate the trajectory preference of z And the embedding representation H of all N POIs f =(h1, h2, ..., h n ), and then calculate Get the preference scores of all POIs Then Score N The POI with the highest score is the POI recommended by the new user z.

[0240] Experiment and analysis

[0241] The following section describes the details of the experimental configuration and provides a comprehensive analysis of the experimental results. The proposed method is tested on four real datasets, and the experimental design focuses on the following four questions.

[0242] MSDR c How does the performance of compare with the most advanced sequence recommendation models in recent years?

[0243] MSDR c What specific role does each component play?

[0244] How do certain unique hyperparameter configurations affect the model?

[0245] MSDR c How does it perform on sparse, noisy datasets?

[0246] 1. Dataset

[0247] Four real-world location datasets NYC and TKY from the FourSquare platform are used for experiments. The FourSquare platform is a mobile service website centered on user location information. The NYC and TKY datasets record the check-in behavior of users in New York (NYC) and Tokyo (TKY), and the user check-in data from April 2012 to February 2013 are divided.

[0248] In order to maintain data quality during the experiment, we adopted a filtering criterion to exclude users with less than 10 check-in records and locations with less than 10 visitors. After filtering, the statistics of each dataset are shown in Table 1, including the number of users, number of locations, location categories, number of check-ins, and average trajectory length.

[0249] Table 1 Statistical summary

[0250]

[0251] 2. Baseline

[0252] In order to evaluate the effectiveness of the proposed model, this paper compares it with the excellent sequence recommendation models in recent years. In addition, in order to further verify the impact of the model on sparsity and capturing and refining high-order spatiotemporal relationships, this paper compares it with the sequence recommendation model and the sequence recommendation model that relies on global modeling and self-supervised learning. The selected baselines are as follows

[0253] Sequential Recommendation Model:

[0254] SASRec(ICDM2018): A sequential recommendation model that effectively captures long-term semantics and balances model complexity by using self-attention.

[0255] SSEPT20(RecSys2020): A personalized Transformer model designed to enhance sequential recommendations by incorporating personalized user embeddings.

[0256] BERT4Rec (ACM2019): A deep bidirectional Transformer model that captures user contextual behavior preferences through masked self-supervision tasks.

[0257] CL4SRec(ICDE2022): The first contrastive learning paradigm applied to sequence recommendation tasks, alleviating the inherent sparsity of sequence data through contrastive self-supervisory signals.

[0258] DuoRec (ACM2022): Improves sequential recommendation models by addressing the problem of representation degradation in contrastive self-supervised tasks through novel contrastive regularization and model-level enhancement.

[0259] TiCoSeRec (TKDE2023): It designs time-aware contrastive data augmentation to alleviate the problem of destroying the original structural information of the sequence in sequence comparison tasks.

[0260] GETNext (SIGIR2022): Model global user trajectory flows through graph structures to capture common and common mobility patterns, and improve POI recommendation models through contextual learning and multi-task learning.

[0261] MAERec(SIGIR2023): An Adaptive Graph Masking Self-Supervised Task Paradigm for Sequential Recommendation, which aims to alleviate the negative impact of noise through an adaptive graph masking strategy.

[0262] DSDRec (Information Sciences 2024): A POI recommendation model enhanced by combining deep semantics and diffusion, which aims to improve the POI recommendation model by mining the deep semantic and spatiotemporal information between sequences.

[0263] 3. Evaluation Metrics

[0264] The present invention uses the evaluation indicators NDCG@K and HR@K, which are commonly used in current recommendation systems, to evaluate the quality of the model, where the value of K is 5, 10 and 20.

[0265] Normalized Discounted Cumulative Gain (NDCG@K) is a commonly used evaluation metric in recommendation systems to measure the quality of item ranking in recommendation lists. It normalizes the cumulative gain by considering the relevance of recommended items at different positions and the impact of their ranking on user satisfaction. i represents the relevance of the i-th item, DCG represents the discounted cumulative gain, and the calculation formula is as follows:

[0266]

[0267] The final NDCG expression is as follows:

[0268]

[0269] Where IDCG@K represents the ideal discounted cumulative gain based on descending relevance sorting.

[0270] Hit rate (HR@K) is defined as the ratio of the number of correctly predicted samples to the total number of samples in the prediction result list. In essence, it measures the consistency between the recommended items and the user's preferences, emphasizing the accuracy of the prediction process. Its calculation formula is as follows:

[0271]

[0272] Among them, hit(i) indicates whether the recommendation result hits user i, where 1 indicates a hit and 0 indicates a miss.

[0273] 4. Implementation Details

[0274] The present invention uses PyTorch to implement the MSDRec proposed in the present invention, and uses the Adam optimizer for parameter inference, with a learning rate of 0.001, a learning rate of the diffusion model set to 0.005, a batch size of 20, and an embedding dimension set to 128. In view of the different trajectory lengths in different data sets, the present invention sets a trajectory length threshold of 50. Trajectories exceeding this threshold will be ignored, and trajectories below this threshold will be filled with zeros to 50. In the graph mask self-supervision module, the present invention sets the mask depth range to {1,3,5}, and performs an optimization on V every {1,5,8,10} time steps. a A sampling is performed to search the number of anchor nodes for each masking step from {10, 50, 100}. The present invention sets the number of GNN layers in the range of {1, 2, 3}. The dropout rate is 0.3. In the spatiotemporal diffusion part, the number of diffusion steps ranges from {10, 20, 32, 50}, the number of Transformer layers is set to 2, and 4 heads are used for multi-dimensional representation.

[0275] 5. Results Analysis

[0276] 1) Recommended performance

[0277] Table 2 shows the comparative experimental results of the MSDRec method and several high-quality methods in recent years on the NYC and TKY real data sets. The bold format is used to represent the best performance data for each indicator, while the underline represents the second-best performance data for each indicator. The table clearly shows the superior performance of the model of the present invention on all evaluation indicators on the four data sets. It is worth noting that the method of the present invention has a significant improvement on the NYC and TKY data sets, with an average improvement of about 3%.

[0278] Table 2 Comparative experiments on NYC, TKY, SG, and GB datasets

[0279]

[0280] In the face of the challenges of data sparsity and complex spatiotemporal relationships in POI recommendation systems, experimental results reveal the obvious advantages of self-supervised learning (such as CL4SRec, DuoRec, BERT4Rec, MAERec, TiCoSeRec) and graph global modeling (such as GETNext) over basic models (such as SASRec, SSEPT). Specifically, self-supervised learning methods such as BERT4Rec achieved an improvement of about 8.75% compared to SASRec on NDCG@20, while the graph global modeling method GETNext achieved an improvement of about 13.42% on the same indicator, showing the significant effect of these methods in improving the model's ability to handle sparse data. Self-supervised learning methods enhance the model's ability to recognize user behavior patterns by simulating learning tasks (such as mask prediction and contrastive learning). For example, the masked language model task of BERT4Rec predicts missing behaviors in user sequences, enabling it to achieve an improvement of about 1.21% on HR@10 compared to SASRec. CL4SRec optimizes the understanding of the model in dealing with sparse environments through contrastive learning, and its NDCG@20 performance is about 7.91% higher than SASRec. Graph global modeling methods such as GETNext achieve about 5.90% improvement in HR@10 compared with SASRec by modeling extensive connections between user trajectories, showing its ability to improve the accuracy and generalization of recommendations at the global level. This method is particularly suitable for dealing with nonlinear and multi-level dependency problems in POI recommendation, effectively improving the efficiency of spatiotemporal data. For the processing of complex spatiotemporal relationships, the DSDRec model combines deep semantic analysis and spatiotemporal information diffusion enhancement to achieve an improvement of about 8.84% in NDCG@20 indicators compared with SASRec, demonstrating its ability to extract clear and useful information from noisy spatiotemporal data. DSDRec verifies that the diffusion model can more accurately identify and remove noise in the data while capturing deep spatiotemporal dependencies. Finally, the MSDRec model of the present invention shows an additional improvement of about 2.26% compared with GETNext on NDCG@20 by combining the advantages of self-supervised learning and spatiotemporal diffusion technology, which proves the excellent ability of MSDRec in handling high-order complex spatiotemporal relationships and the noise they bring, marking a major improvement over the current POI recommendation system methods.

[0281] 2) Effects of different components

[0282] In the study of the present invention, the present invention conducted a series of ablation experiments to evaluate the impact of each component in the MSDRec model proposed by the present invention on the overall prediction performance. These experiments are specially designed to reveal the role of each component in the model and its specific contribution to the performance. The specific ablation experiments include: (1) the complete MSDRec model; (2) removing diffusion denoising (w / o diff); (3) removing graph mask self-supervised learning (w / omasking); (4) removing node importance sampling (w / o NodeImp-Sampling); (5) removing high-order temporal relationship extraction (w / o T); (6) removing high-order spatial relationship extraction (w / o S); (7) removing spatiotemporal relationship diffusion denoising (w / o ST_diff); (8) using a single decoder (w / single_decoder); (9) removing adaptive fusion (w / o weight_fusion). The experimental results are shown in Table 3.

[0283] In the ablation experiment of the present invention, the present invention pays special attention to the key components of the MSDRec model, namely, graph mask self-supervised learning, diffusion denoising, complex spatiotemporal relationship extraction, and decoding self-supervised tasks. The experimental results show that after removing the mask learning function, all performance indicators have declined, especially on NDCG@20, from 0.7603 to 0.7426, proving the importance of mask learning in helping the model learn and understand complex user behaviors and POI relationships. Similarly, after canceling the diffusion denoising process, the performance also declined, with NDCG@20 dropping from 0.7603 to 0.7520, emphasizing the core role of denoising in clarifying and optimizing spatiotemporal signals. In addition, removing the spatiotemporal context processing led to a further significant drop in NDCG@20 to 0.7364, highlighting the key role of spatiotemporal context in enriching POI embedding and enhancing the model's predictive ability. For the decoder component, using only a single decoder has a negative impact on both the reconstruction of the graph structure and the mining of deep information, further confirming their importance in maintaining and understanding the graph structure in the model.

[0284] Table 3 Ablation experiment results of different modules

[0285]

[0286] 3) Impact of different hyperparameters

[0287] See also Figure 4-Figure 7, the present invention systematically analyzes the impact of hyperparameters mask candidate number M, mask update frequency S, mask depth k, and diffusion time step T on model performance. These hyperparameter experiments aim to find the optimal configuration to improve the performance of the model on different datasets. Specifically, the setting range of mask candidate number M is {10, 50, 100}, the mask update frequency S considers {1, 5, 8, 10}, the mask depth k considers {1, 3, 5}, and the setting range of diffusion time step T is {10, 20, 32, 50}. Through detailed experimental analysis of these parameters, the present invention observed some findings on two datasets (NYC and TKY).

[0288] For the number of mask candidates M, see Figure 4 , the present invention found that on the NYC dataset, the model performance first improved with the increase of the number of candidates and reached a peak at M = 50, and then the performance began to decline, which shows that appropriately increasing the number of candidates can help the model capture richer information, but after exceeding a certain threshold, too many candidates may cause the subsequent masking edge to destroy the original structural information of the graph to a certain extent. In contrast, the TKY dataset shows a higher robustness to the increase of the number of candidates, and the performance does not change much. For the mask update frequency S, see Figure 5 , the two datasets show similar trends: as the mask update frequency increases, the model performance first slightly improves or remains stable. This phenomenon shows that moderately increasing the mask update frequency helps the model learn more complex data structures, but too many mask steps may introduce unnecessary complexity and lead to performance degradation. In addition, see Figure 6 , the increase of mask depth k leads to a significant drop in performance in the NYC dataset, indicating that deep masking may destroy the original structure of the graph, while the TKY dataset is less sensitive to this change, showing that different dataset characteristics may affect the optimal choice of hyperparameters. Figure 7 ,Experiments on the diffusion time step T show that a moderate increase in the number of diffusion steps in the NYC dataset can improve the performance at the initial stage, but after exceeding a certain number of steps, the performance decreases, suggesting that too much diffusion may lead to excessive smoothing of information. In the TKY dataset, the increase in the number of diffusion steps has a relatively small impact on the performance, showing its different responses to the diffusion treatment.

[0289] In summary, the present invention selects the best hyperparameter configuration for the two datasets: the number of mask candidates M is 40, the mask step S is 8, the mask depth k is 3, and the diffusion time step T is set to 32. This configuration can maximize the performance while maintaining the model efficiency, avoid the risk of overfitting, and is suitable for processing the complex data features in the two datasets.

[0290] 4) Impact of model anti-sparseness

[0291] Table 4. Changes of NDCG@20 under different sparsity of different models NYC and TKY

[0292]

[0293] In the experiment of the present invention, the present invention simulates the common data sparsity problem of the recommendation system. The model is trained by using only part of the training data (25%, 50%, 75%, and 100%) during the training process, and the test data is kept unchanged to evaluate the performance of the model under different sparsity. Figure 8 The present invention selects two datasets, NYC and TKY, which are helpful for evaluating the model's trajectory data processing capabilities in different urban contexts.

[0294] As baseline models, the present invention uses GETNext and MAERec. GETNext uses graphs to model the processing of global user optimization for time series data, while MAERec focuses on enhancing the model's ability to handle sparse data through an adaptive graph mask mechanism. In the experimental results, the present invention observed that the performance of all models declined as the training data decreased. However, MSDRec consistently outperformed GETNext and MAERec at all levels of data sparsity, and its performance declined at a slower rate. For example, on the NYC dataset, even when the training data was only 75%, the performance of MSDRec was close to that of GETNext and MAERec when using the full amount of training data. In addition, among all models, when the NYC training data was reduced to 50%, the performance of GETNext and MAERec decreased by 13% and 12%, respectively, while MSDRec only decreased by about 10%, showing its stronger robustness in dealing with data sparsity problems. These experimental results not only verify the effectiveness of MSDRec in alleviating data sparsity problems, but also show the differences in adaptability of different models when facing data loss.

[0295] 5) Model’s resistance to noise

[0296] Table 5. Changes of NDCG@20 of different models under different noises of NYC and TKY

[0297]

[0298] Real-world POI recommendation systems often face noisy user interactions. In order to evaluate the robustness of the proposed MSDRec algorithm to noisy data, we replaced a certain proportion (i.e., 0%, 10%, 20%, 30%, 40%, and 50%) of real user interactions with randomly sampled pseudo interactions and evaluated the model on a dataset with artificial noise perturbations. Fig. 9As shown, the proposed model outperforms the baseline at all noise levels on NYC and TKY. In addition, the performance degradation rate of the proposed model is always lower than that of the baseline.

[0299] Finally, it should be noted that the above embodiments are only used to illustrate the technical solution of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical solution of the present invention can be modified or replaced by equivalents without departing from the purpose and scope of the technical solution of the present invention, which should be included in the scope of the claims of the present invention.

Claims

1. A location recommendation method based on graph masking and spatiotemporal diffusion, characterized by: The steps include: constructing MSDRec, which includes a global trajectory flow graph mask self-supervised learning module, a spatiotemporal diffusion denoising module, a global POI representation fusion layer, an embedding layer, and a Transformer encoder; S1: The global trajectory flow graph mask self-supervised learning module constructs a global trajectory flow graph, identifies and masks key connections through node importance sampling and multi-hop mask edge sampling to obtain a global transfer mask graph. Afterwards, the key connections are reconstructed through three self-supervised decoding tasks; S1-1: Construct a global trajectory flow graph, given all historical trajectories of global users Define a directed weighted graph as the global trajectory flow graph in represents the node set, that is, the POI set, E m represents the edge set, Ω m Indicates the edge The number of times it appears in S1-2: Node importance sampling, for Any POI node u∈V m , calculate the embedding changes of node u and its k-th order neighbors to get the importance score of node u Pick The top M nodes with the highest scores are added as candidate nodes to the anchor node set V a middle; V a =Top M {v|v∈V} Among them, Top M express The top M nodes; S1-3: Multi-hop masked edge sampling, defining the global sampling process, the edge sampling probability P(E) between any two node pairs is expressed as: where p u,v represents the sampling probability of the path from node u to node v, p0 represents the basic sampling probability, λ represents the attenuation factor of the control information attenuation, 0<λ≤1, k u,v represents the number of hops between node u and node v; After sampling, we get a global transfer mask map in Indicates that the edge set after removing the masked edge is used as the target of the subsequent decoding task, represents the edge weight set after removing the masked edge; S1-4: obtained by GCN-based graph encoder The mask representation H m ,Design three decoders for decoding: degree decoder, edge existence decoder and edge weight decoder; Degree decoder for prediction The edge existence decoder is used to determine the number of connections for each node in The edge weight decoder is used to predict whether there is an edge between any two nodes in The weights of the existing edges in ; S2: The spatiotemporal diffusion denoising module constructs a global time map and a global space map, and uses a diffusion model to denoise and enhance the spatiotemporal representation to finally obtain the denoised global spatiotemporal data S2-1: Construct a global time graph and a global space graph. Both the global time graph and the global space graph are directed weighted graphs obtained by integrating all user trajectories S. reflects a global view of the spatial position transformation pattern, while the global time map The time-position transition patterns are collected, where V s and V t are all POI check-in node sets, E s and E t is the set of weighted edges that connect two different nodes, Ω s and Ω t Represented and The weight of S2-2: Forward diffusion and reverse diffusion of the diffusion model, using the graph convolutional network GCN encoder to process the global time graph and the global spatial graph respectively to independently extract the spatial POI representation H s and time POI indicates H t , for H s and H t Fusion is performed to obtain the global spatiotemporal POI representation H st , Diffusion process: In the forward process, Gaussian noise gradually increases to H st In, until H st Finally, it is completely transformed into Gaussian noise E t On the contrary, the reverse diffusion process uses a trainable denoiser to remove noise, and a trained diffusion model is obtained through a continuous iterative diffusion process. S2-3: Diffusion of reasoning processes For global spatiotemporal information H st Denoising S3: Global POI representation fusion layer, which will get H m and Fusion to obtain H f ; S4: Embedding layer, each POI are all assigned a final embedding h obtained by the mask module and the diffusion enhancement module of the present invention. p ∈H f Any user Historical access sequence Initial embedding of Obtained by: Among them, h l Indicates user Access Sequence Check-in location The embedding representation of p l express The position embedding of the lth check-in location in ; S5: Recommending users using Transformer encoder Final sequence preference embedding according to Score the next possible POI to achieve prediction; Use a self-supervised method to train MSDRec. When no longer changing, keep the current parameters of the model; S6: For a new user Get The historical trajectory and will After inputting the optimal MSDRec after training, the optimal MSDRec will be calculated based on its historical trajectory. Calculate Trajectory preference And the embedding representation H of all N POIs f =(h1, h2, ..., h n ), and then calculate Get the preference scores of all POIs Then Score N The POI with the highest score is the POI recommended by the new user z.

2. The location recommendation method based on graph mask and spatiotemporal diffusion as claimed in claim 1, characterized in that: In S1-2, the importance score is calculated The process is as follows: Sub k (u) represents the k-order subgraph of node u. For any node v in the u subgraph, represents its embedding representation at time step t, and ‖·‖2 represents the Euclidean norm.

3. The location recommendation method based on graph mask and spatiotemporal diffusion as claimed in claim 2, characterized in that: In S1-4, we get The mask representation H m The process is as follows: right The adjacency matrix of Perform Laplace normalization, where N represents the number of POIs, and normalization The calculation process is as follows: Where D is The degree matrix of , I is the identity matrix; The standardized multi-layer graph convolutional network GCN is used Encoding, each layer of GCN updates the node embedding by neighborhood aggregation, and the formula is: in, is the node feature matrix of the lth layer, is the weight matrix of the lth layer; The node embeddings output by each layer are accumulated to obtain the final representation of node u: in, represents the final mask representation of node u, is the representation of node u after the l-th layer of graph convolution, so The final mask is denoted as H m , any node u∈V m , whose embedding representation 4. The location recommendation method based on graph mask and spatiotemporal diffusion as claimed in claim 3, characterized in that: In S1-4, the decoding process of the three decoders is as follows: Degree Decoder: For Any node u∈V m , define the degree decoding as follows: in, Represents the degree prediction result of node u, Decoder D represents the node degree decoder, ReLU is the activation function, and MLP is the multi-layer perceptron; Edge existence decoder: For The mask edge (u,v) in is represented by its node as and Use MLP to process the embedding of node pairs and use the Sigmoid activation function to predict the probability of the existence of the edge: in, Represents the prediction of the existence of edge (u,v), Decoder E represents the edge existence decoder, ⊙ is the element-wise product, expressing the strength of the interaction between nodes; Edge weight decoder: For The mask edge (u,v) in is represented by its node as and Use MLP to process the embedding of node pairs, and choose to use the ReLu activation function to predict the weight of the edge: in, Represents the prediction of the edge (u,v) weight, Decoder E represents the edge weight decoder and ⊙ represents the element-wise product.

5. The location recommendation method based on graph mask and spatiotemporal diffusion as claimed in claim 4, characterized in that: In S2-1, Ω is constructed s and Ω t The process is as follows: Traverse all tracks If two POI and If it appears continuously in the sequence, it means that a position transition has occurred. The position transitions in all user trajectories are collected, the spatial and temporal intervals are evaluated, and the position transitions are constructed according to formulas (20) and (21). and in Represents any edge (u,v) Weight, Represents any edge (u,v) Weight, Δd represents the spatial interval threshold, represents the time interval threshold, Respectively represent users The spatial and temporal intervals of the transfer from node i to node j.

6. The location recommendation method based on graph mask and spatiotemporal diffusion as claimed in claim 5, characterized in that: In S2-2, the forward diffusion and reverse diffusion processes of the diffusion model are as follows: During the forward diffusion process, T noise steps are applied iteratively, where T is a hyperparameter and the data at step t is denoted by E t , the data of step 0 is the original data, that is, let E0 = H st ; Use one in s min to max The linear interpolation sequence s between two hyperparameters generates β t : β t =1-s t / s t-1 ,s=(1,s max ,……,s min ) (23) E t It is obtained by the following formula: Among them, α t =1-β t , Reparameterization Reverse Diffusion Process: Use a learnable neural network to estimate the following conditional probabilities: p(E t-1 |And t )=N(E t-1 ;μ θ (AND t ,t),∑ θ (AND t ,t)) (25) Among them, μ θ (·),∑ θ (·) represents a neural network with learnable parameters θ, which is used to estimate the parameters of the Gaussian distribution; the embedding at the tth time step is concatenated with the embedding at a specific time step as input; the network consists of two fully connected layers: μ θ (e t ,t)=FC 2 (e t |h t ), FC(x)=δ(Wx+b) (26) Among them, FC 2 Represents two consecutive fully connected layers; represents the node embedding vector in the t-th step diffusion, and represents the embedding at the tth time step; δ(·), W, and b refer to the activation function, the weight of the linear transformation, and the bias vector of the fully connected layer, respectively.

7. The location recommendation method based on graph mask and spatiotemporal diffusion as claimed in claim 7, characterized in that: In S2-3, we get The process is as follows: The spatiotemporal information enhancer is based on H st As input, skip the forward noise diffusion process, and in the denoising step, update To iterate the diffusion enhancement, The definition is as follows: according to and t and use To predict use For space-time information H st Denoising is performed to obtain a denoised spatiotemporal representation 8. The location recommendation method based on graph mask and spatiotemporal diffusion as claimed in claim 7, characterized in that: In S3, H f The process is as follows: Where W represents the learnable aggregation matrix, 9. The location recommendation method based on graph mask and spatiotemporal diffusion as claimed in claim 8, characterized in that: In S5, the multi-head attention mechanism in the Transformer encoder is as follows: The formula of the multi-head self-attention module is expressed as follows: Among them, L and H represent the number of multi-head attention modules and heads respectively; Head h represents the score of the hth head; Indicates user In the feature representation at layer l, is the projection matrix of the h-th head corresponding to the query, key, and value in the attention mechanism; Concat(·) is the concatenation operation, Representative User The final representation of layer l; W1, W2, b1, b2 form a feedforward neural network; the final sequence embedding is obtained through residual connection 10. The location recommendation method based on graph mask and spatiotemporal diffusion as claimed in claim 9, characterized in that: In S5, the loss The calculation process is as follows: The degree decoder is optimized by the mean square error loss function. The degree decoding loss for: in, represents the degree decoder's prediction of the node u, D u is the actual degree of the node, and N is the total number of nodes in the graph; The edge existence decoder uses a binary cross entropy loss function, the edge existence decoding loss for: Among them, E neg Represents the negative sample set of the selected non-existent edges, represents the score of the actual existence of the edge (u,v), is the negative sample score; The edge weight decoder uses the mean square error loss function to optimize the weight decoding loss for: in, is the weight of the edge (u,v) predicted by the edge weight decoder, is the actual weight of the edge, |E| is The total number of sides in the middle; Among them, logp(e0) is ELBO, is the reconstruction term, p θ (e t |e0) is the trained denoiser, q(e T |e0) is the true distribution, T is the pre-set diffusion step hyperparameter; The purpose of the denoising matching term is to minimize the true distribution q(e t-1 |e t , e0) and denoiser p θ (e t-1 |e t ) The denoised match term at the t-th time step is defined as follows, It is expressed as: in, represents the prediction of the initial embedding e0, and the reconstruction term is represented as the predicted value The square loss between the real embedding vector e0, i.e., the diffusion loss The definition is as follows: For Users Optimizing the loss for recommendation tasks Defined as: in Represents the user u trajectory sequence Embedded, Represents real sign-in of embedding, and A randomly selected user Negative samples that have not been visited;

Images