A multi-mode travel recommendation method based on a contrast debiased heterogeneous graph neural network

Abstract

The application discloses a multi-mode travel recommendation method based on a contrast debiasing heterogeneous graph neural network, relates to the technical field of public transportation information processing, and comprises the following steps: 1, acquiring multi-source data and constructing a heterogeneous graph according to the multi-source data; 2, extracting different types of node representations from the heterogeneous graph by using a heterogeneous graph Transformer algorithm; 3, removing multi-source data bias by using a contrast debiasing learning method, performing data enhancement, and obtaining a navigation scheme embedding vector by using a sequence model trained by self-supervised learning; and 4, fusing the node representations and the navigation scheme embedding vector by using a linear neural network to obtain a multi-mode travel recommendation result. By encoding and embedding the data and transforming the heterogeneous graph, the multi-modal features are extracted by using the heterogeneous graph Transformer algorithm and the debiasing contrast learning algorithm respectively, and then the multi-modal features are fused, so that the accurate and reliable multi-mode travel recommendation is realized, and the problem that the accuracy and reliability of the prior art are low is overcome.

Description

Technical Field

[0001] This invention relates to the field of public transportation information processing technology, and more specifically to a multimodal travel recommendation method based on a contrastive debiased heterogeneous graph neural network. Background Technology

[0002] With the continuous development of location-based services (LBS), research on transportation recommendation systems is increasing. This research includes next-stop recommendation, multimodal travel recommendation, and route recommendation. Among these recommendation systems, multimodal travel recommendation is one of the core functions of various navigation software. Common modes of transportation include public transport, subway, driving, cycling, and walking. Once a user inputs their origin and destination, navigation software can provide multiple options based on different travel routes. Multimodal travel recommendation systems search among these options for the most suitable travel plan for the user's needs and prioritize recommending the most appropriate plan. Furthermore, urban traffic managers can guide drivers to choose different modes of transportation by recommending different travel options, thereby achieving a balanced distribution of traffic flow and ultimately optimizing the urban transportation system.

[0003] However, research on multimodal travel recommendation is still in its early stages. Research on multimodal travel recommendation systems currently faces several challenges: (1) Biased data. When faced with multiple options, people tend to look at the highest-ranked option, even if it is not their preferred choice. This bias is prevalent in various recommendation systems, making it difficult for models to accurately understand user preferences during training. (2) Sparse and long-tailed data. In a large city, if it is divided into N traffic analysis zones, due to the exponential growth of potential traffic routes, there will be N... 2 There are OD (origin-destination) pairs. This means that a large number of travel OD demands may not appear in the existing dataset. In addition, a few popular origin and destination points (such as train stations and airports) dominate the vast majority of the data, while many other locations appear only a limited number of times. (3) Difficulty in representing large-scale graph structures. Numerous studies have shown that group behavior has an impact on individual travel. Therefore, modeling group travel characteristics as graph structures is a widely used approach. However, traditional graph convolution methods have a computational complexity of O(n^2) when constructing the adjacency matrix. 2 For example, when n users access a location simultaneously, it is necessary to construct n 2 Using edges to represent the interactions between them is unacceptable for large-scale city-level travel data. Due to these shortcomings, existing multimodal travel recommendation systems suffer from insufficient accuracy and reliability.

[0004] Therefore, improving the accuracy and reliability of multimodal travel recommendations is a problem that urgently needs to be solved by those skilled in the art. Summary of the Invention

[0005] In view of this, the present invention provides a multimodal travel recommendation method based on a contrastive debiased heterogeneous graph neural network. This method encodes and embeds data and transforms heterogeneous graphs, and uses the heterogeneous graph Transformer algorithm and the debiased contrastive learning algorithm to extract multimodal features, respectively. Then, multimodal feature fusion is performed to achieve accurate and reliable multimodal travel recommendations, overcoming the problems of low accuracy and reliability of existing methods.

[0006] To achieve the above objectives, the present invention adopts the following technical solution:

[0007] A multimodal travel recommendation method based on contrastive debiased heterogeneous graph neural networks includes the following steps:

[0008] Step 1: Acquire multi-source data and construct a heterogeneous graph based on the multi-source data;

[0009] Step 2: Use the Heterogeneous Graph Transformer algorithm to extract different types of node representations from the heterogeneous graph;

[0010] Step 3: Use a contrastive debiasing learning method to remove bias from multi-source data, perform data augmentation on the multi-source data, and input the augmented data into a sequence model trained by self-supervised learning to obtain the navigation scheme embedding vector;

[0011] Step 4: The extracted node representations and navigation scheme embedding vectors are fused using a linear neural network to obtain multimodal travel recommendation results.

[0012] The technical effect of the above solution is to construct a heterogeneous graph from multi-source data to achieve a more comprehensive expression of heterogeneous information networks; the heterogeneous graph Transformer algorithm represents the mutual attention on each edge based on the parameters of node type and edge type, and extracts different types of node representations; the fusion of the features extracted from the two parts can realize the fusion of multi-source data and give recommendation results for multi-modal travel.

[0013] Preferably, a heterogeneous graph has a node set and edge set The picture Satisfying two mapping functions:

[0014]

[0015]

[0016] The two mapping functions assign each node in the node set and each edge in the edge set to a specific type; where, and It is a predefined set of node types and edge types that satisfy heterogeneity.

[0017] The steps for constructing a heterogeneous graph include:

[0018] Step 11: Obtain navigation request data and geographic information from multi-source data. The navigation request data includes user ID, origin location, destination location, and access frequency. The node set in the heterogeneous graph includes user nodes and location nodes, where user ID is used as a user node, and origin location and destination location are used as location nodes.

[0019] Step 12: Connect the starting point relationship edge between the user ID and the originating location, and connect the ending point relationship edge between the user ID and the destination location;

[0020] Step 13: Use the access frequency of the originating location as an attribute of the starting point relation edge, and use the access frequency of the destination location as an attribute of the ending point relation edge.

[0021] Step 14: Connect the origin and destination points with edges when the same user ID accesses both the origin and destination locations simultaneously;

[0022] Step 15: Determine the surrounding locations based on geographic information. Select the surrounding locations whose Manhattan distance to the origin or destination location is less than the set adjacency threshold as adjacent nodes, and connect the adjacent nodes with the corresponding origin or destination locations using spatial adjacency edges.

[0023] Step 16: Add reverse edges to the starting point relation edges, ending point relation edges, starting point-end point pairs, and spatial adjacency edges to construct an undirected heterogeneous graph.

[0024] Preferably, the heterogeneous graph Transformer algorithm is used to represent the mutual attention on each edge based on parameters of node type and edge type, and to extract node representations of different types, specifically including:

[0025] Step 21: In the heterogeneous graph constructed in Step 1, for a given target node tgt and all its neighboring nodes src∈N(tgt), calculate the mutual attention HGT based on the meta-relations between the target node and its neighboring nodes. l [tgt] is represented as:

[0026]

[0027]

[0028]

[0029]

[0030]

[0031]

[0032]

[0033] Where e represents the edge between the neighboring node src and the target node tgt; Attention(src,e,tgt) represents the attention neural network; Message(src,e,tgt) represents the message-passing neural network; Aggregate represents the aggregation function; for the i-th attention head, the node will be mapped to three different vectors with meta-relation-specific linear projections. d is the original dimension of the node; h is the number of attention heads; d / h is the dimension of each attention head; μ represents the node similarity, which is calculated through a scaled dot product with an additional prior tensor. This indicates the overall importance of each meta-relation; <φ(src), ψ(ε), φ(tgt)> represents a triple consisting of the src type of the adjacent node, the tgt type of the target node, and the edge type between the two nodes, where φ(src) represents the src type of the adjacent node, φ(tgt) represents the tgt type of the target node, and ψ(ε) represents the edge type; H represents the parameters of the attention neural network and the information transmission neural network corresponding to the edge of type ψ(e), respectively; (l-1) [src]、H (l -1) [tgt] represents the mutual attention between adjacent nodes src and the mutual attention between the target node tgt in the (l-1)th layer, respectively;

[0034] Step 22: Aggregate messages from neighboring nodes and update the mutual attention HGT. l [tgt], as shown in the following formula:

[0035]

[0036]

[0037] Where σ represents the nonlinear activation function; information from all source nodes is aggregated to the target node and then projected linearly onto ALiner. φ(tgt) Update the parameters and perform residual connections in the heterogeneous graph;

[0038] Step 23: Based on the updated mutual attention, extract node representations using the heterogeneous graph neural network constructed according to the heterogeneous graph Transformer algorithm to obtain user embedding vectors and location feature embedding vectors.

[0039] Preferably, step 3 includes the following specific processes:

[0040] Step 31: The multi-source data also includes the navigation scheme corresponding to the navigation request data. The navigation scheme is augmented twice by shuffling the navigation scheme and setting the navigation scheme to zero, respectively, to obtain random augmented views P1 and P2.

[0041] Step 311: Sort all navigation schemes by time to obtain the navigation scheme sequence P;

[0042] Step 312: Zero out some information in the navigation scheme sequence P by randomly discarding a certain proportion of information in the navigation scheme; zeroing out the navigation scheme means randomly discarding a certain proportion of information in the navigation scheme, not completely discarding the scheme, and setting some features in the scheme to zero.

[0043] Step 313: Navigation scheme shuffling is a data augmentation operation that eliminates the impact of positional bias in the recommended schemes. First, all navigation schemes in the navigation scheme sequence P from Step 312 are padded to the same length using specific numbers. Then, the padded navigation schemes are randomly shuffled to obtain a randomly augmented view P1. Because positional bias comes not only from order but also from sequence length, navigation scheme shuffling can reduce the influence of both simultaneously. This is represented as:

[0044] P1 = Drippiut(Shuffle(P));

[0045] Step 314: Return to step 311 and execute 311-313 again to obtain the randomized enhanced view P2, represented as:

[0046] P2 = Dropout(Shuffle(P))

[0047] Dropout and Shuffle represent zeroing out the navigation scheme and shuffling the navigation scheme, respectively; since both data augmentation methods contain randomness, P1≠P2.

[0048] Step 32: Select a multilayer long short-term memory network (LSTM) as the sequence model, and embed it using a sequence model heap of random augmented views to obtain the navigation scheme embedding vectors P′1 and P′2:

[0049] P1′=LSTM(P1)

[0050] P′2=LSTM(P2)

[0051] Long Short-Term Memory (LSTM) neural networks can be represented as:

[0052] f t =σ(W f (h t-1 x t )+b f )

[0053] i t =σ(W i (h t-1 x t )+b i )

[0054]

[0055]

[0056] o t =σ(W o (h t-1 x t )+b o )

[0057] h t =o t *tanh(C t )

[0058] Among them, f t The forgetting gate represents the long short-term memory neural network; i t , This represents the input gate; C t It indicates the cell state; o t This indicates the output gate; h t The hidden state is represented by σ; σ is the sigmoid activation function; W and b represent the weight parameters and biases of the corresponding layers, respectively; the input at each time step is embedded into a hidden state {h1, h2, ..., hb}. t-1 h t In the last hidden state h, only the last hidden state is hidden. t It will be selected for use;

[0059] Step 33: Perform a projection transformation on the random augmented views P′1 and P′2. Let z1 and z2 represent the projected vectors. Then, the process of minimizing the symmetry loss can be expressed as:

[0060]

[0061] in,

[0062]

[0063] in, The value represents the loss; ||·||2 represents the l2-norm of the variable; the stopgrad(·) function indicates that the variable it indicates will be treated as a constant independent of gradient calculation; This represents the negative cosine similarity function, used to calculate the distance between two vectors;

[0064] Step 34: Optimize the sequence model parameters using the loss value, and return to step 32 to obtain the optimized navigation scheme embedding vector.

[0065] Preferably, the fusion step specifically includes:

[0066] The navigation scheme is summed with the encoded representation vectors of the nodes in the heterogeneous graph, and the result is output through a linear neural network. The linear neural network uses a multilayer perceptron, as shown in the following equation:

[0067]

[0068] Where MLP stands for Multilayer Perceptron; p i u i l i These represent the navigation scheme embedding vector, user embedding vector, and location feature embedding vector corresponding to the i-th navigation request data, respectively.

[0069] As can be seen from the above technical solution, compared with the prior art, this invention discloses a multi-modal travel recommendation method based on a contrastive debiasing heterogeneous graph neural network. This method includes four main steps: multi-source data heterogeneous graph construction, heterogeneous graph Transformer algorithm, contrastive debiasing learning algorithm, and fusion recommendation. In solving the multi-modal travel recommendation problem, this method fully considers heterogeneity, improving the accuracy and precision of predictions through hierarchical heterogeneous graph feature extraction and modeling. Simultaneously, the method employs a contrastive debiasing learning algorithm, effectively eliminating the influence of biases in multi-source data on the results, thereby enhancing the reliability of the results. Furthermore, this method is applicable not only to other urban spatial areas but also to different travel mode recommendations, exhibiting strong generalizability. The neural network framework used in this method has obvious framework and modular characteristics, allowing for the addition or removal of modules according to actual conditions and easy connection with other applications, possessing good scalability. In summary, this method not only demonstrates higher accuracy and reliability in multi-modal travel recommendation but also possesses good scalability, making it more suitable for widespread application. Attached Figure Description

[0070] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on the provided drawings without creative effort.

[0071] Figure 1 The attached figure is a flowchart of a multimodal travel recommendation method based on a contrastive debiased heterogeneous graph neural network provided by the present invention;

[0072] Figure 2 The attached figure is a schematic diagram illustrating the specific implementation process of the multi-modal travel recommendation method in the embodiments provided by the present invention. Detailed Implementation

[0073] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0074] Example 1

[0075] This embodiment defines the multimodal travel recommendation problem, including:

[0076] Definition 1 (Basic Symbols):

[0077] make Represents a collection of users, each user It includes the corresponding user attributes. Let Representative locations, each location l = can be represented by a tuple containing latitude and longitude.<lon,lat> .make A collection representing different timestamps.

[0078] Definition 2 (Navigation Request):

[0079] A navigation request consists of a tuple The diagram shows that o and d represent the start and end points of the navigation, respectively. A navigation request indicates that a user u is preparing to travel from location o to location d with timestamp t and has sent a request to the navigation software.

[0080] Definition 3 (Navigation Scheme):

[0081] Once a navigation request is received, the map service will provide multiple navigation options. Each navigation option is represented by p =<m,r1,r2,...> The tuple representation of `r1`, `r2`, ..., represents various information related to the navigation plan, such as price, distance, and estimated time of arrival (ETA). Users will receive multiple navigation plans in a specific order, which we use... This represents a sequence of multiple navigation schemes.

[0082] Definition 4 (Multimodal Travel Recommendation):

[0083] The goal of multimodal travel recommendation is to provide the optimal multimodal travel plan p by learning from existing user navigation request records. More specifically, the optimal mapping function f(·) learned from the data satisfies

[0084] m * =f(r, P) r ) = f(o, d, u, t, P) r ),

[0085] Where m * This indicates the travel mode corresponding to the optimal navigation route. In this invention, we assume that the travel mode actually selected by the user is the optimal travel mode.

[0086] This invention discloses a multi-modal travel recommendation method based on a contrastive debiased heterogeneous graph neural network, the process of which is as follows: Figure 1 As shown, it includes:

[0087] S1: Construction of heterogeneous graphs from multi-source data; By constructing heterogeneous graphs from multi-source data, a more comprehensive expression of heterogeneous information networks can be achieved, integrating multi-source data;

[0088] Heterogeneous graphs are graphs with a set of nodes. and edge set The picture Satisfying two mapping functions:

[0089]

[0090]

[0091] The two mapping functions assign each node in the node set and each edge in the edge set to a specific type; where, and It is a predefined set of node types and edge types that satisfy heterogeneity.

[0092] The steps for constructing a heterogeneous graph include:

[0093] S11: Obtain navigation request data and geographic information from multi-source data. The navigation request data includes user ID, origin location, destination location, and access frequency. The node set in the heterogeneous graph includes user nodes and location nodes, where user ID is used as a user node, and origin location and destination location are used as location nodes.

[0094] S12: Connect the starting point relationship edge between the user ID and the originating location, and connect the ending point relationship edge between the user ID and the destination location;

[0095] S13: Use the access frequency of the originating location as the attribute of the starting point relation edge, and use the access frequency of the destination location as the attribute of the ending point relation edge.

[0096] S14: Connect the origin and destination points of simultaneous visits by users with the same user ID.

[0097] S15: Determine the surrounding locations based on geographic information, and take the surrounding locations whose Manhattan distance to the origin and destination locations is less than the set adjacency threshold as adjacent nodes, and connect the spatial adjacency edges between the adjacent nodes and the corresponding origin or destination locations.

[0098] S16: Add reverse edges to the starting point relation edges, ending point relation edges, starting point-end point pairs, and spatial adjacency edges to construct an undirected heterogeneous graph;

[0099] S2: Heterogeneous graph Transformer algorithm; it represents the mutual attention on each edge based on the parameters of node and edge types, and extracts node representations of different types;

[0100] S21: For a given target node tgt and all its neighboring nodes src∈N(tgt) in a heterogeneous graph, compute the mutual attention HGT based on the meta-relations between the target node and its neighboring nodes. l [tgt] is represented as:

[0101]

[0102]

[0103]

[0104]

[0105]

[0106]

[0107]

[0108] Where e represents the edge between the neighboring node src and the target node tgt; Attention(src,e,tgt) represents the attention neural network; Message(src,e,tgt) represents the message-carrying neural network; Aggregate represents the aggregation function; for the i-th attention head, the node will be mapped to three different vectors with meta-relation-specific linear projections. d is the original dimension of the node; h is the number of attention heads; d / h is the dimension of each attention head; node similarity is calculated via a scaled dot product with an additional prior tensor. To represent the overall importance of each meta-relation; <φ(src), ψ(ε), φ(tgt)> represents a triple consisting of the types of nodes src and tgt and the edges between them; H represents the neural network parameters corresponding to the attention and information transmission of ψ(e) type edges, respectively; (l-1) [src]、H (l-1) [tgt] represents the mutual attention between the src and tgt nodes in the (l-1)th layer;

[0109] S22: Aggregate the corresponding messages from neighboring nodes and update the mutual attention HGT mentioned above. l [tgt], as shown in the following formula:

[0110]

[0111]

[0112] Where σ represents the nonlinear activation function; information from all source nodes is aggregated to the target node and then projected linearly onto ALiner. φ(tgt) The parameters were updated, and a residual connection was performed.

[0113] S23: Based on the updated mutual attention, extract node representations using a heterogeneous graph neural network constructed using the heterogeneous graph Transformer algorithm to obtain user embedding vectors and location feature embedding vectors;

[0114] S3: Contrastive debiasing learning algorithm; a contrastive learning method is used to remove bias from multi-source data, and a self-supervised learning scheme is introduced into the training process;

[0115] S31: Multi-source data also includes navigation schemes corresponding to navigation request data. Navigation schemes are augmented using navigation scheme shuffling and navigation scheme zeroing methods.

[0116] S311: Sort all navigation schemes by time to obtain a navigation scheme sequence P. For the navigation scheme sequence P, setting the navigation scheme to zero means randomly discarding a certain proportion of information in the navigation scheme; or, if the scheme is not completely discarded, setting some features of the scheme to zero.

[0117] S312: Navigation scheme shuffling is a data augmentation operation that eliminates the impact of positional bias in recommended schemes. First, all navigation schemes are filled to the same length using specific numbers, and then the filled navigation schemes are randomly shuffled. Because positional bias comes not only from order but also from sequence length, navigation scheme shuffling can reduce the influence of both simultaneously.

[0118] S313: By using the above data augmentation method twice, two randomly augmented views, P1 and P2, can be generated:

[0119]

[0120] Dropout and Shuffle represent zeroing out the navigation scheme and shuffling the navigation scheme, respectively; since both data augmentation methods contain randomness, P1≠P2.

[0121] S32: Select a multilayer long short-term memory network (LSTM) as the sequence model, and use the sequence model to embed random augmented views to obtain the navigation scheme embedding vectors P′1 and P′2:

[0122] P1′=LSTM(P1)

[0123] P′2=LSTM(P2)

[0124] Long Short-Term Memory (LSTM) neural networks can be represented as:

[0125] f t =σ(W f (h t-1 x t )+b f )

[0126] i t =σ(W i (h t-1 x t )+b i )

[0127]

[0128]

[0129] o t =σ(W o (h t-1 xt )+b o )

[0130] h t =o t *tanh(C t )

[0131] Among them, f t The forgetting gate represents the long short-term memory neural network; i t , This represents the input gate; C t It indicates the cell state; o t This indicates the output gate; h t The hidden state is represented by σ; σ is the sigmoid activation function; W and b represent the weight parameters and biases of the corresponding layers, respectively; the input at each time step is embedded into a hidden state {h1, h2, ..., hb}. t-1 h t In the last hidden state h, only the last hidden state is hidden. t It will be selected for use;

[0132] S33: Comparative learning and training to correct biases;

[0133] Projecting the random augmented views P′1 and P′2, let z1 and z2 represent the projected vectors. Then, the loss value in minimizing the symmetry loss can be expressed as:

[0134]

[0135] in,

[0136]

[0137] Where ||·||2 represents the calculation of the l2-norm of the variable; the stopgrad(·) function indicates that the variable it indicates will be treated as a constant independent of the gradient calculation; This represents the negative cosine similarity function, used to calculate the distance between two vectors;

[0138] S34: Optimize the sequence model parameters using the loss value, and return to step 32 to obtain the optimized navigation scheme embedding vector;

[0139] S4: Fusion Recommendation; Based on the heterogeneous graph and contrast-based bias removal learning of multi-source data, the features extracted from the two parts are fused to achieve the fusion of multi-source data and provide recommendation results for multi-modal travel;

[0140] The navigation scheme is summed with the encoded representation vectors of the nodes in the heterogeneous graph, and the result is output through a linear neural network. The linear neural network uses a multilayer perceptron, as shown in the following equation:

[0141]

[0142] Where MLP stands for Multilayer Perceptron; p i u i l i These represent the embedding vectors of the navigation scheme, user, and location features corresponding to the i-th navigation request data, respectively. The embedding vector of the navigation scheme is obtained from the sequence model of S3; the embedding vectors of the user and location features are obtained from the node mutual attention of S2.

[0143] Example 2

[0144] In one specific embodiment, City A is used as the recommended area for multimodal travel. This embodiment extracts travel navigation records from a navigation software in City A from October 1, 2018 to November 30, 2018. After data processing, multimodal travel data is obtained as an example.

[0145] This embodiment achieves this through a neural network method. Figure 1 Steps S1 through S4. A more detailed structure of the neural network is as follows: Figure 2 As shown in Table 1 below:

[0146] Table 1 Neural Network Parameter Settings

[0147]

[0148]

[0149] In addition, the aforementioned multimodal travel data was sorted in chronological order, with the first 70% of the data used to train the neural network; the first 70% to 80% of the data used to verify the effectiveness of the neural network training; and the last 20% of the data used to test the performance of the final neural network.

[0150] This embodiment uses a weighted F1 score, along with precision and recall, to evaluate the effectiveness of multimodal travel recommendations. The evaluation metrics can be expressed as:

[0151]

[0152]

[0153]

[0154]

[0155] In this context, TP, FP, and FN represent true positives, false positives, and false negatives for each travel mode category, respectively. A true positive represents the number of correctly predicted positive samples, a false positive represents the number of negative samples incorrectly predicted as positive samples, and a false negative represents the number of positive samples incorrectly predicted as negative samples. k It is calculated from the proportion of instances of category k.

[0156] The results of the tests on the test set are shown in Table 2 below:

[0157] Table 2 Test Results

[0158] 0.7675 0.7650 0.7908

[0159] It can be seen that the prediction method of the present invention has shown accuracy and reliability in recommending multiple evaluation indicators.

[0160] This invention discloses a multimodal travel recommendation method based on a contrastive debiasing heterogeneous graph neural network. Its core functions include a heterogeneous graph Transformer algorithm and a contrastive debiasing learning algorithm, aiming to achieve feature extraction for multimodal travel. This method constructs a heterogeneous graph from multi-source data and performs feature fusion prediction after feature extraction.

[0161] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on its differences from other embodiments. Similar or identical parts between embodiments can be referred to interchangeably. For the apparatus disclosed in the embodiments, since it corresponds to the method disclosed in the embodiments, the description is relatively simple; relevant parts can be referred to the method section.

[0162] The above description of the disclosed embodiments enables those skilled in the art to make or use the invention. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the invention. Therefore, the invention is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A multi-modal travel recommendation method based on a contrastive debiased heterogeneous graph neural network, characterized in that, Includes the following steps: Step 1: Acquire multi-source data and construct a heterogeneous graph based on the multi-source data; Step 2: Use the Heterogeneous Graph Transformer algorithm to extract different types of node representations from the heterogeneous graph; Step 3: Use a contrastive debiasing learning method to remove bias from multi-source data, perform data augmentation on the multi-source data, and input the augmented data into a sequence model trained by self-supervised learning to obtain the navigation scheme embedding vector; Step 4: The extracted node representations and navigation scheme embedding vectors are fused using a linear neural network to obtain multi-modal travel recommendation results; Step 3 includes the following specific steps: Step 31: The multi-source data includes the navigation scheme corresponding to the navigation request data. The navigation scheme is augmented twice by shuffling the navigation scheme and zeroing the navigation scheme, respectively, to obtain random augmented views P1 and P2. Step 32: Using a multi-layer long short-term memory network (LSTM) as the sequence model, embedding the navigation scheme using a heap of random augmented views from the sequence model to obtain the embedding vector. and : ; ； Step 33: Randomly augment the view and Perform a projection transformation, let Let the projected vector be the vector. Then, the process of minimizing the symmetry loss is expressed as: ; in, ; in, Indicates the loss value; Represents the calculated variable -norm; The function indicates that the variable it indicates will be treated as a constant independent of gradient calculation; Represents the negative cosine similarity function; Step 34: Optimize the sequence model parameters using the loss value, and return to step 32 to obtain the optimized navigation scheme embedding vector; The process of data augmentation of navigation schemes using navigation scheme shuffling and navigation scheme zeroing methods includes: Step 311: Sort all navigation schemes by time to obtain the navigation scheme sequence P; Step 312: Zero out some information in the navigation scheme sequence P using the navigation scheme; Step 313: Shuffling the navigation schemes. Using a set number, fill all the navigation schemes in the navigation scheme sequence P from step 312 to the same length, and then randomly shuffle the filled navigation schemes to obtain a randomized enhanced view P1, represented as: ； Step 314: Return to step 311, and execute steps 311-313 again to obtain the randomized enhanced view P2, represented as: ; Dropout and Shuffle represent zeroing out the navigation scheme and shuffling the navigation scheme, respectively.

2. The multi-modal travel recommendation method based on a contrastive debiased heterogeneous graph neural network according to claim 1, characterized in that, Heterogeneous graphs are graphs with a set of nodes. and edge set The picture It satisfies two mapping functions: ; The two mapping functions assign each node in the node set and each edge in the edge set to a specific type; where, and It is a predefined set of node types and edge types that satisfy heterogeneity. .

3. The multimodal travel recommendation method based on a contrastive debiased heterogeneous graph neural network according to claim 1, characterized in that, The steps involved in constructing a heterogeneous graph include: Step 11: Obtain navigation request data and geographic information from multi-source data. The navigation request data includes user ID, origin location, destination location, and access frequency. The node set in the heterogeneous graph includes user nodes and location nodes, where user ID is used as a user node, and origin location and destination location are used as location nodes. Step 12: Connect the starting point relationship edge between the user ID and the originating location, and connect the ending point relationship edge between the user ID and the destination location; Step 13: Use the access frequency of the originating location as an attribute of the starting point relation edge, and use the access frequency of the destination location as an attribute of the ending point relation edge. Step 14: Connect the origin and destination points with edges when the same user ID accesses both the origin and destination locations simultaneously; Step 15: Determine the surrounding locations based on geographic information. Select the surrounding locations whose Manhattan distance to the origin or destination location is less than the set adjacency threshold as adjacent nodes, and connect the adjacent nodes with the corresponding origin or destination locations using spatial adjacency edges. Step 16: Add reverse edges to the starting point relation edges, ending point relation edges, starting point-end point pairs, and spatial adjacency edges to construct an undirected heterogeneous graph.

4. The multimodal travel recommendation method based on a contrastive debiased heterogeneous graph neural network according to claim 3, characterized in that, The heterogeneous graph Transformer algorithm is used to represent the mutual attention on each edge based on node type and edge type, and to extract different types of node representations, specifically including: Step 21: In the heterogeneous graph constructed in Step 1, for a given target node tgt and all its neighboring nodes src∈N(tgt), calculate the mutual attention HGTl[tgt] based on the meta-relations between the target node and its neighboring nodes, expressed as: ; Where e represents the edge between the neighboring node src and the target node tgt; Attention(src,e,tgt) represents the attention neural network; Message(src,e,tgt) represents the message-carrying neural network; Aggregate represents the aggregation function; for the i-th attention head, the node will be mapped to three different vectors with meta-relation-specific linear projections. ; d is the original dimension of the node; h is the number of attention heads; d / h is the dimension of each attention head; Indicates node similarity. It is calculated by the scaled dot product with an additional prior tensor, representing the overall importance of each meta-relation; This represents a triple consisting of the src type of the adjacent node, the tgt type of the target node, and the edge type between the two nodes. Indicates the src type of adjacent nodes. Indicates the target node type tgt. Indicates the edge type; , They represent The parameters of the attention neural network and the parameters of the information transmission neural network corresponding to the type edge; , These represent the mutual attention between adjacent nodes src and the mutual attention between the target node tgt in the (l-1)th layer, respectively. Step 22: Aggregate messages from neighboring nodes and update mutual attention. As shown in the following formula: ; in, This represents a non-linear activation function; information from all neighboring nodes is aggregated to the target node through linear projection. Update the parameters and perform residual connections in the heterogeneous graph; Step 23: Based on the updated mutual attention, extract node representations using the heterogeneous graph neural network constructed according to the heterogeneous graph Transformer algorithm to obtain user embedding vectors and location feature embedding vectors.

5. The multimodal travel recommendation method based on a contrastive debiased heterogeneous graph neural network according to claim 1, characterized in that, The specific steps of the fusion include: The navigation scheme is summed with the encoded representation vectors of the nodes in the heterogeneous graph, and the result is output through a linear neural network. The linear neural network uses a multilayer perceptron, as shown in the following equation: ; MLP stands for Multilayer Perceptron; , , They represent the first The navigation scheme embedding vector, user embedding vector, and location feature embedding vector corresponding to each navigation request data.

Images