Multi-modal travel route personalized generation method and system based on deep learning

Abstract

The invention relates to the technical field of tourism big data, in particular to a multi-modal tourism route personalized generation method and system based on deep learning, and the method comprises the steps: constructing and incrementally updating a tourism knowledge hypergraph, extracting hidden time sequence preference and cross-entity cooperation signals of a user through a space-time perception graph neural network, and generating an interest drift model; performing deep fusion and intention analysis on the interest drift model and a context signal sensed in real time, and decoding to generate a candidate route concept skeleton by introducing an attention competition mechanism; and carrying out multi-dimensional simulation deduction and adaptability evaluation on the candidate route concept skeleton, carrying out iterative optimization through a reinforcement learning strategy based on deduction feedback, and finally outputting a personalized tourist route with optimal robustness, so that the robust tourist route which is highly personalized and adapts to environmental changes can be generated, and the robustness of the tourist route is improved. Therefore, recommendation accuracy and user experience are improved.

Description

Technical Field

[0001] This invention relates to the field of tourism big data technology, and in particular to a method and system for generating personalized multimodal tourism routes based on deep learning. Background Technology

[0002] In recent years, with the rapid development of the tourism industry, generating truly personalized travel routes for users has become a core challenge in the field of smart tourism. Traditional recommendation methods mostly rely on static historical behavior or single-modal data, making it difficult to effectively integrate multimodal information such as text, images, and sensor data. Furthermore, they lack unified modeling of dynamic user preferences, spatiotemporal context, and complex cross-entity relationships. Existing methods have significant shortcomings in semantic alignment accuracy, high-order interaction representation, and real-time adaptability, resulting in generated routes that often ignore user latent intentions, interest drift, and environmental disturbances, limiting their personalization and robustness. Therefore, there is an urgent need for an intelligent route generation method that can deeply integrate multimodal data, perceive temporal changes, and possess reasoning and simulation optimization capabilities. Summary of the Invention

[0003] This invention overcomes the shortcomings of existing technologies and provides a method and system for generating personalized multimodal travel routes based on deep learning.

[0004] To achieve the above objectives, the technical solution adopted by the present invention is as follows: The first aspect of this invention discloses a method for personalized generation of multimodal travel routes based on deep learning, comprising the following steps: Parallel preprocessing and semantic alignment are performed on multimodal tourism data streams from different data sources to extract primary entity features with consistent spatiotemporal annotations; Based on the aforementioned primary entity features, a tourism knowledge hypergraph is constructed and incrementally updated, wherein nodes represent aligned multimodal entities, and their hyperedges characterize higher-order interaction relationships between entities through event perception. Using the aforementioned tourism knowledge hypergraph, a predictive interest drift model is generated by extracting users' implicit temporal preferences and cross-entity collaborative signals through a spatiotemporally aware graph neural network. The interest drift model is deeply fused with real-time perceived contextual signals and intent analysis is performed. By introducing an attention competition mechanism, a set of candidate route concept skeletons that reflect the user's potential exploration intent are decoded and generated. The candidate route concept skeleton is subjected to multi-dimensional simulation and adaptive evaluation, and iterative optimization is carried out through reinforcement learning strategy based on the simulation feedback, finally outputting the most robust personalized travel route.

[0005] For example, parallel preprocessing and semantic alignment are performed on multimodal tourism data streams from different data sources to extract primary entity features with consistent spatiotemporal annotations, specifically: Receive multimodal tourism data streams from different data sources, including text, images, and sensor data; Synchronous and independent processing of various types of data: text data is segmented and named entity recognition is performed to extract text entities, image data is convolved to generate visual entities, and sensor data is parsed to obtain location entities; Semantic alignment analysis based on a shared tourism ontology is performed on the text entities, visual entities, and location entities, and semantically consistent fused entities are generated through concept mapping and relational reasoning. Extract timestamps and geographic coordinates from the raw data stream, and use these to attach a unified time stamp and spatial stamp to each fused entity to form a spatiotemporal anchor entity; Based on the spatiotemporal anchor entities, primary entity features with consistent spatiotemporal annotations are generated through a feature encoding layer.

[0006] For example, based on the primary entity features, a tourism knowledge hypergraph is constructed and incrementally updated, wherein nodes represent aligned multimodal entities, and their hyperedges characterize higher-order interaction relationships between entities through event awareness, specifically: Each primary entity feature is mapped to a high-dimensional vector to form a node embedding that includes multimodal semantics and spatiotemporal annotation; By employing a sliding time window and geographic grid partitioning, spatiotemporal clustering of nodes is performed to generate spatiotemporal co-occurrence event clusters centered on events; All nodes within each spatiotemporal co-occurrence event cluster are modeled for relationships using a multi-head hypergraph attention mechanism. The affinity between nodes is calculated, and event-aware hyperedges are generated. These event-aware hyperedges are attached with attribute vectors representing the event type and intensity. All event-aware hyperedges are combined into a sparse hypergraph adjacency tensor, and the node-relationship joint representation is learned through a hypergraph convolutional network using the sparse hypergraph adjacency tensor. When new data flows in, the new nodes are aligned with existing graph nodes, and local hyperedge recomputation is triggered. The affected event-aware hyperedges and their attribute vectors are updated using the temporal gating fusion unit, thereby realizing the dynamic updating and semantic maintenance of the tourism knowledge hypergraph.

[0007] For example, utilizing the aforementioned tourism knowledge hypergraph, a predictive interest drift model is generated by extracting users' implicit temporal preferences and cross-entity collaborative signals through a spatiotemporally aware graph neural network, specifically as follows: The time encoding layer in the spatiotemporally aware graph neural network encodes the node timestamps to generate time-enhanced node features, and at the same time, the spatial encoding layer calculates spatial attention weights based on geographic coordinates to output spatially aware node embeddings. The temporally enhanced node features and spatially aware node embeddings are fused to form a spatiotemporal context node representation, which is then input into a multi-layer hypergraph convolution module for message propagation. By aggregating the association relationships of nodes within hyperedges, node-level temporal pattern features are extracted. Based on the node-level temporal pattern features, a temporal attention mechanism is used to dynamically weight the user's historical interaction sequence to generate a user's implicit temporal preference vector. The spatiotemporal context node representation is then used for cross-entity collaborative analysis. By calculating the semantic and spatiotemporal correlation between entities, a cross-entity collaborative signal matrix is ​​output. The implicit temporal preference vector of users is concatenated with the cross-entity collaborative signal matrix and then input into a predictive recurrent unit network with a gating mechanism to model the temporal evolution of user interests and generate an interest drift model that represents the future changes in user preferences.

[0008] For example, the interest drift model is deeply fused with real-time perceived contextual signals and intent analysis is performed. By introducing an attention competition mechanism, a set of candidate route concept skeletons reflecting the user's potential exploration intent is decoded and generated, specifically: The user's future preference change representation output by the interest drifting model is concatenated with the real-time perceived context signal to form an initial fusion intent tensor; the influence weights of different context dimensions on the user's historical behavior and real-time state are calculated in parallel by multiple independent attention heads to generate a multi-faceted attention weight matrix. Based on the aforementioned multi-faceted attention weight matrix, a gating competition mechanism is used to perform saliency competition and sparsification screening on the weights output by different attention heads, generating the winning attention weight vector. The initial fusion intent tensor is weighted and pooled using the winning attention weight vector to output dynamic fusion features; The dynamic fusion features are subjected to multi-scale intent analysis. The dependencies between potential intent units are mined through a self-attention layer, and a set of candidate route concept skeletons that reflect the user's multi-dimensional potential exploration intent are generated through feedforward network transformation and decoding.

[0009] For example, based on the aforementioned multi-faceted attention weight matrix, a gating competition mechanism is used to perform saliency competition and sparsity filtering on the weights output by different attention heads to generate the winning attention weight direction, specifically: Based on the aforementioned multi-faceted attention weight matrix, a gated generative network is used to calculate the saliency score of each attention head, forming an attention head saliency score vector. The attention head saliency score vector is weighted and fused with the original multi-faceted attention weight matrix to generate a weighted attention weight matrix. The weighted attention weight matrix is ​​subjected to a temperature parameter-based sparsification transformation to calculate the competitive threshold of each weight unit and generate a sparsification mask. The sparse attention weight matrix is ​​obtained by element-wise filtering and resetting the weighted attention weight matrix using the sparse mask. The sparse attention weight matrix is ​​weighted, summed, and normalized along the attention head dimension to generate the final winning attention weight vector.

[0010] For example, the candidate route concept skeleton is subjected to multi-dimensional simulation and adaptive evaluation, and iterative optimization is performed through reinforcement learning strategy based on the simulation feedback, finally outputting the most robust personalized travel route, specifically: The conceptual skeleton of candidate routes is injected into a virtual twin space, and a multi-dimensional environment model is generated through an environment modeling engine. The multi-dimensional environment model integrates geographical, social and real-time contextual data. Parallel simulation and deduction are performed based on the multi-dimensional environment model to simulate user behavior, external events and resource constraints, and output the deduction trajectory sequence. The predicted trajectory sequence is subjected to a multi-index adaptive evaluation, and evaluation indicators including time cost, user interest matching degree and environmental disturbance robustness are calculated to generate an adaptive score vector. The adaptive score vector is used as a reward signal and input into a policy gradient-based reinforcement learning agent to generate a policy update signal through the agent policy network. The node sequence and weights of the candidate route conceptual skeleton are adjusted using the strategy update signal to generate an optimized version of the candidate route; Repeat the simulation, adaptive evaluation, and strategy update steps until the adaptive score vector converges or the maximum number of iterations is reached, and then output the most robust personalized travel route.

[0011] For example, the adaptive score vector is used as a reward signal input to a policy gradient-based reinforcement learning agent, and a policy update signal is generated through the agent policy network, specifically as follows: The adaptive score vector is decomposed into multi-objective dimensions and normalized and weighted to generate standardized reward components; The standardized reward component is fused with the action probability distribution currently output by the policy network to construct an advantage function estimate; Based on the advantage function, the original policy gradient of the policy loss function with respect to the network parameters is calculated; the original policy gradient is subjected to gradient stability analysis to evaluate the consistency of its magnitude and direction, and a gradient stability index is generated. Based on the gradient stability index, an adaptive pruning threshold is applied to prune the magnitude and correct the direction of the original policy gradient, resulting in a normalized policy gradient vector. Based on the normalized policy gradient vector and the current network parameter state, a policy update signal is output.

[0012] A second aspect of this invention discloses a deep learning-based multimodal personalized travel route generation system, comprising: Multimodal alignment module: Performs parallel preprocessing and semantic alignment on multimodal tourism data streams from different data sources, and extracts primary entity features with consistent spatiotemporal annotations; Hypergraph Engine: Based on the aforementioned primary entity features, a tourism knowledge hypergraph is constructed and incrementally updated, wherein nodes represent aligned multimodal entities, and their hyperedges characterize higher-order interaction relationships between entities through event perception. Preference perception module: Utilizing the aforementioned tourism knowledge hypergraph, a spatiotemporally perceptive graph neural network is used to extract users' implicit temporal preferences and cross-entity collaborative signals, generating a predictive interest drift model; Intent decoder: Deeply fuses the interest drift model with real-time perceived contextual signals and performs intent analysis. By introducing an attention competition mechanism, it decodes and generates a set of candidate route concept skeletons that reflect the user's potential exploration intent. Simulation and deduction module: Performs multi-dimensional simulation and deduction and adaptive evaluation on the candidate route concept skeleton, and iteratively optimizes it through reinforcement learning strategy based on the deduction feedback, and finally outputs the most robust personalized travel route.

[0013] This invention addresses the technical deficiencies in the prior art and offers the following advantages: It performs parallel preprocessing and semantic alignment on multimodal tourism data streams from different data sources, extracts primary entity features with consistent spatiotemporal annotations, constructs and incrementally updates a tourism knowledge hypergraph, and utilizes this hypergraph to extract users' implicit temporal preferences and cross-entity collaborative signals through a spatiotemporally aware graph neural network, generating an interest drift model. The interest drift model is then deeply fused with real-time perceived contextual signals for intent analysis. By introducing an attention competition mechanism, candidate route concept skeletons are decoded and generated. Multi-dimensional simulation and adaptive evaluation are performed on the candidate route concept skeletons, and iterative optimization is achieved through reinforcement learning strategies based on the simulation feedback, ultimately outputting the most robust personalized tourism route. This invention can generate highly personalized and environmentally adaptable robust tourism routes, thereby improving recommendation accuracy and user experience. Attached Figure Description

[0014] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other embodiments can be obtained from these drawings without creative effort.

[0015] Figure 1 Here is a flowchart of the method for generating personalized multimodal tourism routes; Figure 2 This is a system block diagram of the multimodal personalized tourism route generation system. Detailed Implementation

[0016] To better understand the above-mentioned objectives, features, and advantages of the present invention, the present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments. It should be noted that, unless otherwise specified, the embodiments and features described in these embodiments can be combined with each other.

[0017] Many specific details are set forth in the following description in order to provide a full understanding of the invention. However, the invention may also be practiced in other ways different from those described herein, and therefore the scope of protection of the invention is not limited to the specific embodiments disclosed below.

[0018] like Figure 1 As shown, the first aspect of this invention discloses a method for personalized generation of multimodal travel routes based on deep learning, comprising the following steps: S1. Perform parallel preprocessing and semantic alignment on multimodal tourism data streams from different data sources to extract primary entity features with consistent spatiotemporal annotations; S2. Based on the primary entity features, construct and incrementally update the tourism knowledge hypergraph, where nodes represent aligned multimodal entities, and their hyperedges characterize the higher-order interaction relationships between entities through event perception. S3. Using the aforementioned tourism knowledge hypergraph, extract users' implicit temporal preferences and cross-entity collaborative signals through a spatiotemporally aware graph neural network to generate a predictive interest drift model. S4. Deeply fuse the interest drift model with the real-time perceived contextual signals and perform intent analysis. By introducing an attention competition mechanism, decode and generate a set of candidate route concept skeletons that reflect the user's potential exploration intent. S5. Perform multi-dimensional simulation and adaptive evaluation on the candidate route concept skeleton, and iteratively optimize it through reinforcement learning strategy based on the simulation feedback, and finally output the most robust personalized travel route.

[0019] For example, parallel preprocessing and semantic alignment are performed on multimodal tourism data streams from different data sources to extract primary entity features with consistent spatiotemporal annotations, specifically: Receive multimodal tourism data streams from different data sources, including text, images, and sensor data; It should be further explained that the text data includes travel blogs and comments posted by users on social media platforms, attraction descriptions on the official websites of scenic spots, text information in travel guides, and Q&A interactions between users; the image data includes travel photos shared by users, real-time images captured by surveillance cameras in scenic spots, satellite remote sensing geographic images, and official promotional images of scenic spots; the sensor data includes trajectory points generated by the GPS module of users' smartphones, physiological indicators such as heart rate and steps recorded by smart wearable devices, temperature, humidity and weather conditions collected by environmental sensors, and real-time information such as crowd density and queuing time sent by IoT devices in scenic spots.

[0020] Synchronous and independent processing of various types of data: text data is segmented and named entity recognition is performed to extract text entities, image data is convolved to generate visual entities, and sensor data is parsed to obtain location entities; Semantic alignment analysis based on a shared tourism ontology is performed on the text entities, visual entities, and location entities, and semantically consistent fused entities are generated through concept mapping and relational reasoning. It should be further explained that a predefined structured "shared tourism ontology" is used. This ontology is organized in the form of a knowledge graph and includes a hierarchical structure of concepts in the tourism domain (e.g., "attractions > natural landscapes > mountains"), entity attributes (e.g., "opening hours", "ticket prices"), and relation sets (e.g., "located in", "belongs to", "similar to"). Then, the extracted text entities (e.g., "Eiffel Tower"), visual entities (e.g., CNN feature vectors labeled "tower"), and location entities (e.g., GPS coordinates) are mapped to the shared semantic space of the ontology through an embedding layer. The text entities are mapped by calculating similarity with ontology concepts using models such as BERT. The visual entities are matched with visual concept nodes in the ontology using a pre-trained visual model. The location entities are converted into address text through a reverse geocoding service before mapping. Next, based on the relational axioms (such as transitivity and symmetry) and rules (such as "if A' is located in 'B' and B' belongs to 'C', then A' is located in 'C'") defined in the ontology, relational reasoning is performed on entities co-occurring in similar spatiotemporal contexts to identify whether they represent the same real-world object (e.g., different modal data all point to "Eiffel Tower") or have other semantic associations (e.g., "Louvre" and "Paris" have a "located" relationship). Finally, a globally unique ID is assigned to multiple entities representing the same object, and their attributes from all modalities are aggregated to generate a "fused entity" containing multimodal evidence and rich semantic attributes.

[0021] Extract timestamps and geographic coordinates from the raw data stream, and use these to attach a unified time stamp and spatial stamp to each fused entity to form a spatiotemporal anchor entity; Based on the spatiotemporal anchor entities, primary entity features with consistent spatiotemporal annotations are generated through a feature encoding layer.

[0022] It should be noted that the heterogeneous original information (including text description, visual features, latitude and longitude coordinates, and timestamps) contained in each spatiotemporal anchor entity is mapped and fused into a unified, high-dimensional numerical vector representation, i.e., the primary entity feature. Specifically, firstly, an independent encoding path is established for the information of each modality: deep semantic embedding vectors are extracted for text information through a pre-trained language model (such as BERT), high-level abstract feature vectors are extracted for visual information through a convolutional neural network, and spatiotemporal information is converted from absolute timestamps and geographic coordinates into high-dimensional spatiotemporal encoded vectors with periodicity and relativity through a sine function time encoder and a spatial grid encoder, respectively. Subsequently, these modal feature vectors from different paths are concatenated into a multimodal joint vector and input into a shared multilayer perceptron for nonlinear transformation and dimensionality reduction. The multilayer perceptron learns through training how to most effectively weigh and fuse the contributions of different modalities. Finally, a dense vector with fixed dimensions and combining semantic, visual, and strong spatiotemporal context information is output, which is the primary entity feature with consistent spatiotemporal annotation.

[0023] For example, based on the primary entity features, a tourism knowledge hypergraph is constructed and incrementally updated, wherein nodes represent aligned multimodal entities, and their hyperedges characterize higher-order interaction relationships between entities through event awareness, specifically: Each primary entity feature is mapped to a high-dimensional vector to form a node embedding that includes multimodal semantics and spatiotemporal annotation; By employing a sliding time window and geographic grid partitioning, spatiotemporal clustering of nodes is performed to generate spatiotemporal co-occurrence event clusters centered on events; It should be further explained that, using the Transformer model, each primary entity feature is taken as input and, through forward propagation, further mapped into a higher-dimensional, more representative dense vector. This vector simultaneously encodes the multimodal semantics of textual meaning and visual content, as well as its additional spatiotemporal annotation information, thereby generating the final node embedding. Next, the system employs a sliding time window mechanism to initially divide all nodes by timestamp. Within each time window, a geographic grid partitioning method is used to further subdivide the nodes into spatial regions. For the node embedding set within each specific time window and geographic grid unit, the DBSCAN clustering algorithm is applied. Semantic-spatiotemporal similarity is measured by calculating the Euclidean distance between node embeddings in the feature space, thus clustering nodes under this dual constraint (i.e., temporal co-occurrence, spatial proximity, and semantic feature similarity) into groups. Finally, each identified cluster is defined as a spatiotemporally co-occurring event cluster, representing a potential local hotspot event or group activity.

[0024] All nodes within each spatiotemporal co-occurrence event cluster are modeled for relationships using a multi-head hypergraph attention mechanism. The affinity between nodes is calculated, and event-aware hyperedges are generated. These event-aware hyperedges are attached with attribute vectors representing the event type and intensity. All event-aware hyperedges are combined into a sparse hypergraph adjacency tensor, and the node-relationship joint representation is learned through a hypergraph convolutional network using the sparse hypergraph adjacency tensor. It is worth mentioning that a sparse hypergraph adjacency tensor H is constructed, with dimensions [number of nodes, number of hyperedges]. The element H[i,e] in the tensor represents the association strength between node i and hyperedge e. This strength value comes from the node-hyperedge affinity calculated during the previous hyperedge generation process, rather than a simple 0 or 1 binary association, thus forming a weighted sparse tensor. This tensor H is then used as input to a hypergraph convolutional network (HGCN), and joint representations are learned by executing a "node-hyperedge-node" message-passing paradigm: In the first stage, the network uses tensor H to connect all nodes to hyperedges. The node features of the same hyperedge e are aggregated (e.g., by attention-weighted summation) to generate the semantic embedding of the hyperedge e, which encodes the inherent pattern of the event. In the second stage, the network uses the transpose of the same tensor H to backpropagate the semantic embedding information of each hyperedge e and aggregate it to each node i contained therein. Through multiple stacked convolutional operations of this kind, the final representation of each node not only integrates its own multimodal and spatiotemporal features, but also more efficiently absorbs the high-order contextual semantic relations of all events (hyperedges) to which it belongs, thereby learning the node-relationship joint representation.

[0025] When new data flows in, the new nodes are aligned with existing graph nodes, and local hyperedge recomputation is triggered. The affected event-aware hyperedges and their attribute vectors are updated using the temporal gating fusion unit, thereby realizing the dynamic updating and semantic maintenance of the tourism knowledge hypergraph.

[0026] In summary, this invention effectively integrates multimodal tourism data and uses a hypergraph structure to characterize high-order semantics and spatiotemporal interaction relationships between entities, thereby achieving deep perception and dynamic modeling of complex events and group activities in tourism scenarios, and improving the accuracy and richness of route generation in understanding user preferences and contextual environment.

[0027] For example, utilizing the aforementioned tourism knowledge hypergraph, a predictive interest drift model is generated by extracting users' implicit temporal preferences and cross-entity collaborative signals through a spatiotemporally aware graph neural network, specifically as follows: The time encoding layer in the spatiotemporally aware graph neural network encodes the node timestamps to generate time-enhanced node features, and at the same time, the spatial encoding layer calculates spatial attention weights based on geographic coordinates to output spatially aware node embeddings. The temporally enhanced node features and spatially aware node embeddings are fused to form a spatiotemporal context node representation, which is then input into a multi-layer hypergraph convolution module for message propagation. By aggregating the association relationships of nodes within hyperedges, node-level temporal pattern features are extracted. Based on the node-level temporal pattern features, a temporal attention mechanism is used to dynamically weight the user's historical interaction sequence to generate a user's implicit temporal preference vector. The spatiotemporal context node representation is then used for cross-entity collaborative analysis. By calculating the semantic and spatiotemporal correlation between entities, a cross-entity collaborative signal matrix is ​​output. The implicit temporal preference vector of users is concatenated with the cross-entity collaborative signal matrix and then input into a predictive recurrent unit network with a gating mechanism to model the temporal evolution of user interests and generate an interest drift model that represents the future changes in user preferences.

[0028] In specific implementation, firstly, the temporal encoding layer in the spatiotemporally aware graph neural network uses a sinusoidal positional encoding function to transform the timestamp attached to each node in the tourism knowledge hypergraph, generating a high-dimensional temporal encoding vector that includes absolute time and relative periodicity. This vector is added to the original node features to form time-enhanced node features. Simultaneously, the spatial encoding layer calculates the spatial relative distance between different nodes based on the geographic coordinates carried by the nodes, and uses a softmax function to normalize and generate spatial attention weights, thereby weighted aggregation of node features and outputting a spatially aware node embedding that incorporates geographic proximity information.

[0029] Secondly, the obtained temporally enhanced node features and spatially aware node embeddings are concatenated into vectors, and then nonlinearly fused and dimensionality reduced through a fully connected layer to form a unified spatiotemporal context node representation, which is then input into a multi-layer hypergraph convolutional module. This module utilizes a pre-constructed hypergraph adjacency structure defined by event-aware hyperedges for message propagation: in each convolutional layer, node features are first aggregated within the same hyperedge to generate a hyperedge-level semantic representation, which is then diffused back to all nodes connected to that hyperedge, updating their states. Through multi-layer stacking, the final representation of each node (i.e., node-level temporal pattern features) includes the high-order interaction context and temporal dependency patterns of its multiple events (hyperedges).

[0030] Subsequently, based on the extracted node-level temporal pattern features, the focus is on the target user's diachronic interaction sequence (such as the sequence of scenic spot entities visited in chronological order). A temporal attention mechanism is used to dynamically weight and sum this sequence, generating a latent temporal preference vector that represents the user's historical preferences and highlights their recent key interests. In parallel, cross-entity collaborative analysis is performed on the spatiotemporal context node representations of all nodes. By calculating the similarity between any two entities in their semantic feature vectors (such as category label embeddings from ontology mapping) and spatiotemporal features (such as timestamp differences and geographical distance), a symmetric cross-entity collaborative signal matrix is ​​constructed to quantify the potential substitutability or complementarity between entities.

[0031] Finally, the user's implicit temporal preference vector and the cross-entity collaborative signal matrix are expanded and concatenated into tensors to form a comprehensive input tensor, which is then input into a predictive recurrent unit network with a gating mechanism (such as a GRU or LSTM unit). This network takes the current comprehensive input tensor and its own hidden state as input, and controls the retention and forgetting of historical information through its internal update and reset gates. It gradually models the state transition and evolution of user interests from historical, current, to future states. The final output hidden state represents the trend and direction of future user preference changes, thus generating a predictable interest drift model.

[0032] For example, the interest drift model is deeply fused with real-time perceived contextual signals and intent analysis is performed. By introducing an attention competition mechanism, a set of candidate route concept skeletons reflecting the user's potential exploration intent is decoded and generated, specifically: The user's future preference change representation output by the interest drifting model is concatenated with the real-time perceived context signal to form an initial fusion intent tensor; the influence weights of different context dimensions on the user's historical behavior and real-time state are calculated in parallel by multiple independent attention heads to generate a multi-faceted attention weight matrix. The real-time perceived contextual signals include real-time weather coding vectors, current geographic location embeddings, and instantaneous crowd density indices.

[0033] It should be noted that the user's future preference change representation output by the interest drift model is concatenated with the real-time perceived contextual signals (both converted into numerical vectors of the same dimension) to form a comprehensive initial fusion intent tensor, which serves as the base input for subsequent deep intent analysis. Subsequently, a multi-head attention layer is used, with multiple independent attention heads (e.g., 8 heads) working in parallel. Each attention head focuses on a specific contextual dimension (such as "weather influence," "spatiotemporal accessibility," or "social popularity"), calculating its influence weight for the user's historical behavior sequence and real-time state. Finally, the output weights of all heads are concatenated to generate a multi-faceted attention weight matrix.

[0034] Based on the aforementioned multi-faceted attention weight matrix, a gating competition mechanism is used to perform saliency competition and sparsification screening on the weights output by different attention heads, generating the winning attention weight vector. The initial fusion intent tensor is weighted and pooled using the winning attention weight vector to output dynamic fusion features; It should be noted that the weight vector and the intent tensor are weighted and summed in the corresponding dimensions to form a feature vector, which reflects the user's comprehensive intent under the modulation of the key context signal that wins the competition.

[0035] The dynamic fusion features are subjected to multi-scale intent analysis. The dependencies between potential intent units are mined through a self-attention layer, and a set of candidate route concept skeletons that reflect the user's multi-dimensional potential exploration intent are generated through feedforward network transformation and decoding.

[0036] It should be noted that after the dynamically fused features are input into the multi-scale intent analysis module, the dependencies between potential intent units within the feature are first mined through a self-attention layer (for example, it is found that the intent to "seek indoor activities" is highly correlated with the context of "current rainstorm"), outputting a refined intent representation with an internally related structure. Subsequently, this representation undergoes nonlinear transformation and decoding via a feedforward neural network, outputting a sequence or a set. For example, a set of latent vectors representing different possible exploration directions is generated, and each latent vector constitutes a candidate route concept skeleton, reflecting the user's potential exploration intent in multiple dimensions (such as "cultural and artistic exploration", "food and leisure", "rain shelter emergency plan"), providing structured and diversified initial plans for subsequent simulation and inference, thereby improving the real-time decision-making capability and user satisfaction of the personalized recommendation system.

[0037] For example, based on the aforementioned multi-faceted attention weight matrix, a gating competition mechanism is used to perform saliency competition and sparsity filtering on the weights output by different attention heads to generate the winning attention weight direction, specifically: Based on the aforementioned multi-faceted attention weight matrix, a gated generative network is used to calculate the saliency score of each attention head, forming an attention head saliency score vector. It should be further explained that the weight submatrix corresponding to each attention head in the multi-faceted attention weight matrix is ​​globally averaged and pooled to compress it into a scalar feature value, thereby generating a preliminary summary statistic for each head. Then, these scalars are concatenated in head order to form an initial feature vector, which is then input into a lightweight gated generation network. The gated generation network consists of a fully connected layer (which can be followed by a ReLU activation function) and a linear layer with the same output dimension as the number of attention heads. Finally, the linear layer outputs an unnormalized saliency score for each attention head, and all these scores together constitute the attention head saliency score vector.

[0038] The attention head saliency score vector is weighted and fused with the original multi-faceted attention weight matrix to generate a weighted attention weight matrix. The weighted attention weight matrix is ​​subjected to a temperature parameter-based sparsification transformation to calculate the competitive threshold of each weight unit and generate a sparsification mask. It should be further explained that each weight value in the weighted attention matrix is ​​scaled using a preset temperature parameter (usually set between 0.1 and 1.0), and the entire scaled matrix is ​​then globally normalized to transform it into a probability distribution. The global mean μ and standard deviation σ of this probability distribution matrix are then calculated and combined with an adjustable significance coefficient α (usually set between 0.5 and 3.0) to obtain the competitive threshold θ (i.e., θ = μ + α·σ). Finally, each normalized weight value in the matrix is ​​compared element-wise with the competitive threshold to generate a binary sparse mask matrix, where positions greater than the competitive threshold are set to 1 (retained), and positions less than or equal to the competitive threshold are set to 0 (masked).

[0039] The sparse attention weight matrix is ​​obtained by element-wise filtering and resetting the weighted attention weight matrix using the sparse mask. It should be further explained that the sparse mask matrix and the weighted attention weight matrix are multiplied element-wise (Hadamard product) so that the weight values ​​corresponding to the positions with a value of 0 in the mask are reset to 0, while the positions with a value of 1 in the mask retain their original weight values. In addition, to ensure numerical stability, the mask is inverted (i.e., the positions that were originally 1 become 0, and the positions that were originally 0 become 1) and then multiplied element-wise with a preset constant (such as multiplying by 10 to the power of -9) to generate a compensation matrix. This compensation matrix is ​​then added element-wise to the filtered weight matrix to obtain the sparse attention weight matrix.

[0040] The sparse attention weight matrix is ​​weighted, summed, and normalized along the attention head dimension to generate the final winning attention weight vector.

[0041] For example, the candidate route concept skeleton is subjected to multi-dimensional simulation and adaptive evaluation, and iterative optimization is performed through reinforcement learning strategy based on the simulation feedback, finally outputting the most robust personalized travel route, specifically: The conceptual skeleton of candidate routes is injected into a virtual twin space, and a multi-dimensional environment model is generated through an environment modeling engine. The multi-dimensional environment model integrates geographical, social and real-time contextual data. Parallel simulation and deduction are performed based on the multi-dimensional environment model to simulate user behavior, external events and resource constraints, and output the deduction trajectory sequence. The predicted trajectory sequence is subjected to a multi-index adaptive evaluation, and evaluation indicators including time cost, user interest matching degree and environmental disturbance robustness are calculated to generate an adaptive score vector. The adaptive score vector is used as a reward signal and input into a policy gradient-based reinforcement learning agent to generate a policy update signal through the agent policy network. The node sequence and weights of the candidate route conceptual skeleton are adjusted using the strategy update signal to generate an optimized version of the candidate route; Repeat the simulation, adaptive evaluation, and strategy update steps until the adaptive score vector converges or the maximum number of iterations is reached, and then output the most robust personalized travel route.

[0042] In practice, candidate route concept skeletons (i.e., a preliminary sequence of attractions or intent frameworks) are injected into a pre-constructed virtual twin space. This space is driven by an environment modeling engine, which integrates in real time geographic data from multiple sources (such as high-precision maps, terrain elevation, and real-time traffic conditions), social data (such as real-time visitor density at attractions and social media sentiment), and broader real-time contextual data (such as weather forecasts, public transportation timetables, and emergencies). Based on this data, it generates a multi-dimensional environment model that integrates physical and social rules. Based on this multi-dimensional environment model, the system initiates parallel simulation (e.g., starting an independent simulation instance for each candidate route concept skeleton). In each simulation instance, the system simulates the user's behavior along the route (such as walking speed and mode of transportation selection), random or rule-based triggering of external events (such as attractions closing due to overcrowding, sudden rainfall, and traffic congestion), and the imposition of resource constraints (such as total time budget and monetary budget). The simulation engine progresses step by step over time, recording the complete sequence of user state changes in the virtual environment and outputting a sequence of deduced trajectories including time, location, user state, and encountered events.

[0043] Then, a multi-metric adaptive evaluation is performed on each simulated trajectory sequence. The evaluation module calculates a series of quantifiable metrics, such as: time cost (total time, waiting time at each stage), user interest matching degree (calculated by vector similarity between the features of visited attractions and the preferences predicted by the interest drift model), and environmental perturbation robustness (such as the route's ability to maintain the integrity of its core experience to introduced random perturbation events, which can be measured by comparing the overlap between the planned and actual completed attractions). The calculated results of each metric are then normalized and weighted to generate a comprehensive adaptive score vector, which serves as the performance evaluation of the candidate route in this simulation. This adaptive score vector is used as a reward signal and input into a policy gradient-based reinforcement learning agent (such as using the PPO or REINFORCE algorithm). The agent's policy network takes the representation of the candidate route's conceptual skeleton (such as sequence encoding) as input, and its output is defined as the "action" to adjust the route (such as adding, deleting, or replacing an attraction, or adjusting the stay time weight). The reward signal is used to calculate the policy gradient. The policy update signal is generated through backpropagation of the agent policy network. The policy update signal indicates how the parameters of the policy network should be adjusted to obtain a higher expected reward (i.e. a better fitness score).

[0044] Furthermore, the strategy for generating candidate routes is directly adjusted using the strategy update signal. Specifically, after updating the strategy network parameters according to the gradient direction, the node sequence and weight adjustment scheme of the original candidate route conceptual skeleton are regenerated or sampled (e.g., replacing low-rated attractions with high-rated alternative attractions recommended in the collaborative signal matrix, or increasing the stay time weight of popular attractions), thereby generating an optimized candidate route version.

[0045] The system repeatedly executes the simulation, adaptive evaluation, and strategy update steps described above. Each iteration tests the candidate routes in a virtual environment and improves them based on feedback, until the adaptive score vectors of all candidate routes no longer significantly improve (convergence) or the preset maximum number of iterations is reached. At this point, the candidate with the highest overall score is selected from all candidate versions generated in the final iteration and output as the most robust personalized travel route to the user, thus providing a high-quality travel route that not only meets user preferences but also possesses high environmental adaptability and robustness.

[0046] For example, the adaptive score vector is used as a reward signal input to a policy gradient-based reinforcement learning agent, and a policy update signal is generated through the agent policy network, specifically as follows: The adaptive score vector is decomposed into multi-objective dimensions and normalized and weighted to generate standardized reward components; The standardized reward component is fused with the action probability distribution currently output by the policy network to construct an advantage function estimate; It should be noted that the log probability of the action actually taken by the agent in the simulation environment is calculated using the action probability distribution currently output by the policy network. This log probability is then multiplied by a tensor with the standardized reward component to obtain the unadjusted raw reward estimate. Next, a trainable state value function baseline network is introduced, whose input is the current state and whose output is a scalar estimate of the state value. Finally, the raw reward estimate is subtracted from the state value baseline, and the difference is the advantage function estimate.

[0047] Based on the advantage function, the original policy gradient of the policy loss function with respect to the network parameters is calculated; the original policy gradient is subjected to gradient stability analysis to evaluate the consistency of its magnitude and direction, and a gradient stability index is generated. It should be noted that, based on the action probability distribution under the policy network parameters, the log probability of the executed actions is calculated. Then, this log probability is multiplied by the advantage function estimate obtained in the previous step, and the negative of the product is taken to form the core loss term of the policy gradient algorithm. Finally, using an automatic differentiation framework (such as PyTorch or TensorFlow), the loss term is backpropagated to find its derivative with respect to the policy network parameters. The resulting gradient tensor is the original policy gradient, and its direction points towards the optimization direction that maximizes the expected reward.

[0048] Next, the L2 norm (module) of the current original policy gradient is calculated and compared with a dynamic range threshold based on the moving average of recent historical gradient norms to calculate its relative volatility. Simultaneously, the cosine similarity between the current gradient direction and the historical average gradient direction is calculated to assess directional consistency. The module volatility and direction cosine similarity are weighted and fused, and a comprehensive scalar value between 0 and 1 is generated through a sigmoid function mapping. This value is the gradient stability index, where a value closer to 1 indicates a more stable gradient.

[0049] Based on the gradient stability index, an adaptive pruning threshold is applied to prune the magnitude and correct the direction of the original policy gradient, generating a normalized policy gradient vector. The adaptive pruning threshold is determined by linearly mapping the gradient stability index calculated in real time to a preset minimum and maximum pruning threshold range, such as a preset minimum pruning threshold of 0.1 and a maximum pruning threshold of 10.0. Based on the normalized policy gradient vector and the current network parameter state, a policy update signal is output.

[0050] In this embodiment, the multimodal personalized travel route generation method further includes: Calculate the cosine similarity between the user's real-time trajectory and the recommended route at each spatiotemporal anchor point, and then perform a weighted summation of the cosine similarities at each spatiotemporal anchor point to obtain the route consistency index. When the route consistency index is lower than a preset threshold, it is determined that the spatiotemporal trajectory dispersion exceeds the limit. When the phenomenon of excessive spatiotemporal trajectory dispersion is triggered, the implicit temporal preference vector of the user and the real-time multimodal context signal are fused to construct an inference structure represented by a causal graph. The state of the nodes in the graph is initialized by the spatiotemporal context node representation, and the edge weights are quantized by the cross-entity collaborative signal matrix calculated in real time. Counterfactual reasoning is used to intervene in the edge relationships in the causal graph to generate a set of counterfactual scenarios and calculate their deviation from the expected target to obtain the significance ranking of the impact factors; and the top k impact factors are mapped to an interpretable attribution rule set. The attribution rule is input as a hard constraint into the policy optimization network. By iteratively solving the constraint reinforcement learning problem, a set of minimal and interpretable intervention action sequences is output, which dynamically corrects the spatiotemporal attributes of subsequent tourist route nodes.

[0051] It's important to note that in existing travel recommendation systems, when users deviate from the personalized route recommended by the system due to personal interests, unforeseen circumstances, or environmental factors during their actual visit, the system often lacks an effective real-time perception and response mechanism. Traditional methods typically employ a simple replanning strategy, completely discarding the original route and regenerating a new one based on the user's current location. This ignores the potential intent information contained in the user's previous deviation, potentially leading to the repeated recommendation of nodes that the user is not interested in. Furthermore, the regenerated route may deviate significantly from the user's initial overall travel intentions, disrupting the route's coherence and the overall experience.

[0052] Therefore, this embodiment continuously collects the user's GPS coordinates and timestamps through embedded sensors and a positioning module to form the user's real-time trajectory. The system compares this trajectory with the currently recommended personalized travel route. Specifically, it calculates the direction vectors of the two trajectories at preset spatiotemporal anchor points (i.e., semantically meaningful coordinate points such as key attractions or rest areas in the route), and calculates the cosine similarity between these vectors to quantify the consistency of the user's behavior at each key point. Subsequently, the system performs a weighted summation of the above similarities based on the importance of each anchor point (such as the interest matching weight of the attraction), and finally obtains a comprehensive route consistency index. The system presets an empirical threshold (e.g., 0.8). When the index is lower than this threshold, it is automatically determined that the spatiotemporal trajectory dispersion exceeds the limit, indicating that the user's behavior has deviated significantly, triggering the route intervention process.

[0053] After intervention is triggered, the core task of the system is to diagnose the root causes of deviations from the data. To this end, the system deeply integrates previously calculated implicit temporal preference vectors of users (representing the long-term, potential evolutionary trends of user interests) with current real-time multimodal context signals (such as real-time weather, surrounding crowd density, traffic conditions, etc.) to construct an inference structure represented by a causal graph. In this graph, nodes represent various potential factors influencing user decisions (such as "the crowding level of attraction A" or "the user's immediate interest in topic B"), and their states are initialized by the latest spatiotemporal context node representation. The edges connecting these nodes represent causal relationships or strong correlations between factors, and their weights are quantified by a cross-entity collaborative signal matrix calculated in real time (e.g., quantifying the correlation strength between "rain" and "user abandoning outdoor attractions").

[0054] Next, the system uses counterfactual reasoning to simulate intervention in this causal model. For example, it poses counterfactual questions such as, "If attraction C hadn't been so crowded, would the user have still deviated?" By systematically simulating changes in different influencing factors, a set of counterfactual scenarios is generated, and the deviation between the user's trajectory and the intended goal (original recommended route intent) in each scenario is calculated. By comparing these deviations, the system identifies which changes in factors have the greatest impact on the outcome, thus obtaining a ranking of the significance of influencing factors. The top k key factors in the ranking are then mapped to an understandable set of attribution rules (e.g., "Main cause of deviation: 75% probability of originating from 'sudden rainfall,' causing the user to skip outdoor attraction E").

[0055] Finally, these mined attribution rules are transformed into hard constraints (e.g., "subsequent routes must prioritize indoor attractions" or "must avoid areas currently experiencing rainfall") and input into the policy optimization network. This network transforms a constraint reinforcement learning problem into an optimization problem, aiming to find the sequence of intervention actions that minimizes changes to the spatiotemporal attributes of subsequent recommended routes (such as node order, dwell time, and access time windows) while satisfying all attribution rule constraints. Through iterative solving, a set of minimal and interpretable intervention schemes is ultimately output, thereby dynamically modifying subsequent travel routes. This ensures that while respecting the new intentions revealed by users' real-time behavior, consistency with the overall goals of the original plan is maintained to the greatest extent possible.

[0056] In this embodiment, the multimodal personalized travel route generation method further includes: Extract multimodal entities and their semantic relationships related to historical events or myths from the tourism knowledge hypergraph to construct an original narrative element set; The original narrative elements are input into the generator of the generative adversarial network and mapped to the latent semantic space through a multilayer perceptron to form a narrative element embedding vector. At the same time, the discriminator performs adversarial training on the generated embedding based on the distribution of real historical narrative data, and outputs an adversarial cultural feature vector with high fidelity. The adversarial cultural feature vector is concatenated with the spatiotemporal anchor point entities in the tourism knowledge hypergraph, and its mapping relationship with geographic coordinates is analyzed by a spatiotemporal encoder to generate a set of spatiotemporal folded anchor points that have both semantic connotation and latitude and longitude coordinates. Based on the set of spatiotemporal folding anchor points, a storyline coherence graph network is constructed, in which nodes represent anchor points and edge weights are jointly determined by narrative semantic similarity and temporal order. This network is then jointly topologically mapped with the physical route network, and the narrative and spatial features are analyzed through a dual-graph convolutional network to form a dual topological constraint matrix. The dual topological constraint matrix is ​​input into the route optimization module, and the optimal path is calculated iteratively through dual constraint satisfaction analysis, so that the generated route simultaneously satisfies the narrative coherence based on the anchor point sequence and the actual geographical accessibility.

[0057] It should be noted that entities and semantic relationships related to history and mythology are extracted from the constructed tourism knowledge hypergraph to form a raw set of narrative elements, providing raw materials for narrative construction. Generative adversarial networks (GANs) are used to adversarially train these elements. The generator maps the elements to the latent semantic space, while the discriminator ensures that the generated cultural feature vectors conform to the data distribution of real historical narratives, thus outputting high-fidelity adversarial cultural feature vectors to address the issues of subjectivity and inefficiency in manually constructed narrative threads. Then, by concatenating the cultural feature vectors with spatiotemporal anchor entities in the knowledge hypergraph and analyzing their mapping relationship with geographical coordinates via a spatiotemporal encoder, a set of spatiotemporally folded anchor points containing both semantic connotations and latitude / longitude information is generated. Based on this set of spatiotemporally folded anchor points, a storyline coherence graph network is constructed with narrative semantic similarity and temporal order as edge weights, and this network is jointly topologically mapped with the physical route network. A dual-graph convolutional network is used to analyze the dual features of narrative and space, generating a dual topological constraint matrix, thereby unifying the abstract cultural narrative logic and the specific spatial path structure within a single mathematical framework for analysis. Finally, the dual topological constraint matrix is ​​input into the route optimization module, and iterative calculations are performed through dual constraint satisfaction analysis to ultimately output an optimal path that is not only physically reachable but also coherent in terms of narrative. This process deeply integrates cultural narrative into the route generation process, thereby enhancing the cultural immersion of the recommended routes and improving the overall integrity and logic of the user experience.

[0058] like Figure 2As shown, the second aspect of this invention discloses a multimodal personalized travel route generation system based on deep learning, comprising: Multimodal alignment module: Performs parallel preprocessing and semantic alignment on multimodal tourism data streams from different data sources, and extracts primary entity features with consistent spatiotemporal annotations; Hypergraph Engine: Based on the aforementioned primary entity features, a tourism knowledge hypergraph is constructed and incrementally updated, wherein nodes represent aligned multimodal entities, and their hyperedges characterize higher-order interaction relationships between entities through event perception. Preference perception module: Utilizing the aforementioned tourism knowledge hypergraph, a spatiotemporally perceptive graph neural network is used to extract users' implicit temporal preferences and cross-entity collaborative signals, generating a predictive interest drift model; Intent decoder: Deeply fuses the interest drift model with real-time perceived contextual signals and performs intent analysis. By introducing an attention competition mechanism, it decodes and generates a set of candidate route concept skeletons that reflect the user's potential exploration intent. Simulation and deduction module: Performs multi-dimensional simulation and deduction and adaptive evaluation on the candidate route concept skeleton, and iteratively optimizes it through reinforcement learning strategy based on the deduction feedback, and finally outputs the most robust personalized travel route.

[0059] In summary, this invention effectively solves the problems of insufficient data utilization, difficulty in capturing dynamic preferences, and weak environmental adaptability in traditional recommendation methods by multimodal data fusion and semantic alignment, tourism knowledge hypergraph construction, spatiotemporal perception preference mining, and reinforcement learning optimization. It can generate highly personalized and robust tourism routes that adapt to environmental changes, thereby improving recommendation accuracy and user experience.

[0060] In the several embodiments provided in this application, it should be understood that the disclosed devices and methods can be implemented in other ways. The device embodiments described above are merely illustrative. For example, the division of units is only a logical functional division, and in actual implementation, there may be other division methods, such as: multiple units or components can be combined, or integrated into another system, or some features can be ignored or not executed. In addition, the coupling, direct coupling, or communication connection between the various components shown or discussed can be through some interfaces, and the indirect coupling or communication connection between devices or units can be electrical, mechanical, or other forms.

[0061] The units described above as separate components may or may not be physically separate. The components shown as units may or may not be physical units. They may be located in one place or distributed across multiple network units. Some or all of the units may be selected to achieve the purpose of this embodiment according to actual needs.

[0062] In addition, in the various embodiments of the present invention, each functional unit can be integrated into one processing unit, or each unit can be a separate unit, or two or more units can be integrated into one unit; the integrated unit can be implemented in hardware or in the form of hardware plus software functional units.

[0063] Those skilled in the art will understand that all or part of the steps of the above method embodiments can be implemented by hardware related to program instructions. The aforementioned program can be stored in a computer-readable storage medium. When the program is executed, it performs the steps of the above method embodiments. The aforementioned storage medium includes various media capable of storing program code, such as mobile storage devices, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0064] Alternatively, if the integrated units of this invention are implemented as software functional modules and sold or used as independent products, they can also be stored in a computer-readable storage medium. Based on this understanding, the technical solutions of the embodiments of this invention, or the parts that contribute to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the methods of the various embodiments of this invention. The aforementioned storage medium includes various media capable of storing program code, such as mobile storage devices, ROM, RAM, magnetic disks, or optical disks.

[0065] The above are merely specific embodiments of the present invention, but the scope of protection of the present invention is not limited thereto. Any changes or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in the present invention should be included within the scope of protection of the present invention.

Claims

1. A deep learning-based multi-modal tourism route individualization generation method, characterized in that, The method comprises the following steps: Parallel preprocessing and semantic alignment of multi-modal tourism data streams from different data sources to extract primary entity features with consistent spatiotemporal annotations; Based on the primary entity features, a tourism knowledge supergraph is constructed and incrementally updated, wherein the nodes represent aligned multi-modal entities, and the superedges depict high-order interaction relationships between entities through event perception; Using the tourism knowledge supergraph, a user's implicit temporal preference and cross-entity collaborative signal are extracted through a spatiotemporal-aware graph neural network to generate a predictive interest drift model; The interest drift model is deeply fused with real-time perceived context signals and intent analysis, and through the introduction of an attention competition mechanism, a set of candidate route concept skeletons reflecting the user's potential exploration intent is decoded and generated; The candidate route concept skeletons are simulated and evaluated in multiple dimensions, and based on the feedback from the simulation, the strategy is iteratively optimized through reinforcement learning to finally output a robustly optimal personalized tourism route.

2. The deep learning-based multi-modal tourism route personalization generation method according to claim 1, characterized in that, Parallel preprocessing and semantic alignment of multi-modal tourism data streams from different data sources to extract primary entity features with consistent spatiotemporal annotations, specifically: Receive multi-modal tourism data streams from different data sources, including text, image, and sensor data; Synchronously and independently process each type of data: perform word segmentation and named entity recognition on text data to extract text entities, perform convolution on image data to generate visual entities, and analyze sensor data to obtain location entities; Perform semantic alignment analysis based on shared tourism ontology on the text entities, visual entities, and location entities to produce fused entities with consistent semantics through concept mapping and relationship reasoning; Extract timestamps and geographic coordinates from the original data stream and attach uniform time labels and space labels to each fused entity to form spatiotemporal anchor entities; Based on the spatiotemporal anchor entities, generate primary entity features with consistent spatiotemporal annotations through a feature encoding layer. 3.The deep learning based multi-modal tourism route personalization generation method of claim 1, wherein, Based on the primary entity features, a tourism knowledge supergraph is constructed and incrementally updated, wherein the nodes represent aligned multi-modal entities, and the superedges depict high-order interaction relationships between entities through event perception, specifically: Map each primary entity feature to a high-dimensional vector to form node embeddings containing multi-modal semantics and spatiotemporal annotations; Use a sliding time window and geographic grid division to perform spatiotemporal clustering on the nodes to generate event-centric spatiotemporal co-occurrence clusters; Model the relationships between all nodes within each spatiotemporal co-occurrence cluster through a multi-head supergraph attention mechanism to calculate the affinity between nodes and generate event-aware superedges, which have attribute vectors representing event types and intensities; Combine all event-aware superedges into a sparse supergraph adjacency tensor, and use the sparse supergraph adjacency tensor to learn node-relation joint representations through a supergraph convolutional network; When new data flows in, align the new nodes with the existing graph nodes and trigger local superedge recalculation, update the affected event-aware superedges and their attribute vectors using a temporal gating fusion unit to achieve dynamic updating and semantic maintenance of the tourism knowledge supergraph.

4. The deep learning-based multi-modal tourism route personalization generation method according to claim 1, characterized in that, Using the aforementioned tourism knowledge hypergraph, a predictive interest drift model is generated by extracting users' implicit temporal preferences and cross-entity collaboration signals through a spatiotemporally aware graph neural network. Specifically: The time encoding layer in the spatiotemporally aware graph neural network encodes the node timestamps to generate time-enhanced node features, and at the same time, the spatial encoding layer calculates spatial attention weights based on geographic coordinates to output spatially aware node embeddings. The temporally enhanced node features and spatially aware node embeddings are fused to form a spatiotemporal context node representation, which is then input into a multi-layer hypergraph convolution module for message propagation. By aggregating the association relationships of nodes within hyperedges, node-level temporal pattern features are extracted. Based on the node-level temporal pattern features, a temporal attention mechanism is used to dynamically weight the user's historical interaction sequence to generate a user's implicit temporal preference vector. The spatiotemporal context node representation is then used for cross-entity collaborative analysis. By calculating the semantic and spatiotemporal correlation between entities, a cross-entity collaborative signal matrix is ​​output. The implicit temporal preference vector of users is concatenated with the cross-entity collaborative signal matrix and then input into a predictive recurrent unit network with a gating mechanism to model the temporal evolution of user interests and generate an interest drift model that represents the future changes in user preferences. 5.The deep learning based multi-modal tourism route personalization generation method according to claim 1, characterized in that, The interest drift model is deeply fused with real-time perceived contextual signals and intent analysis is performed. By introducing an attention competition mechanism, a set of candidate route concept skeletons reflecting the user's potential exploration intent is decoded and generated, specifically: The user's future preference change representation output by the interest drifting model is concatenated with the real-time perceived context signal to form an initial fusion intent tensor; the influence weights of different context dimensions on the user's historical behavior and real-time state are calculated in parallel by multiple independent attention heads to generate a multi-faceted attention weight matrix. Based on the aforementioned multi-faceted attention weight matrix, a gating competition mechanism is used to perform saliency competition and sparsification screening on the weights output by different attention heads, generating the winning attention weight vector. The initial fusion intent tensor is weighted and pooled using the winning attention weight vector to output dynamic fusion features; The dynamic fusion features are subjected to multi-scale intent analysis. The dependencies between potential intent units are mined through a self-attention layer, and a set of candidate route concept skeletons that reflect the user's multi-dimensional potential exploration intent are generated through feedforward network transformation and decoding. 6.The deep learning based multi-modal tourism route personalization generation method according to claim 5, characterized in that, Based on the aforementioned multi-faceted attention weight matrix, a gating competition mechanism is employed to perform significant competition and sparsity filtering on the weights output by different attention heads, generating the winning attention weight direction, specifically as follows: Based on the aforementioned multi-faceted attention weight matrix, a gated generative network is used to calculate the saliency score of each attention head, forming an attention head saliency score vector. The attention head saliency score vector is weighted and fused with the original multi-faceted attention weight matrix to generate a weighted attention weight matrix. The weighted attention weight matrix is ​​subjected to a temperature parameter-based sparsification transformation to calculate the competitive threshold of each weight unit and generate a sparsification mask. The sparse attention weight matrix is obtained by element-level screening and resetting of the weighted attention weight matrix using the sparsification mask; The sparse attention weight matrix is weighted summed and normalized along the attention head dimension to generate a final winning attention weight vector. 7.The deep learning based multi-modal tourism route personalization generation method according to claim 1, characterized in that, The candidate route concept skeleton is subjected to multi-dimensional simulation deduction and adaptability evaluation, and based on the deduction feedback, an optimal personalized travel route is iteratively optimized through reinforcement learning strategy, specifically: The candidate route concept skeleton is injected into a virtual twin space, and a multi-dimensional environment model is generated through an environment modeling engine, which integrates geographic, social, and real-time context data; Parallel simulation deduction is performed based on the multi-dimensional environment model to simulate user behavior, external events, and resource constraints, and a deduction trajectory sequence is output; Multi-index adaptability evaluation is performed on the deduction trajectory sequence to calculate evaluation indexes including time cost, user interest matching degree, and environmental disturbance robustness, and an adaptability score vector is generated; The adaptability score vector is input as a reward signal into a policy gradient-based reinforcement learning agent, and a policy update signal is generated through the agent policy network; The node sequence and weight of the candidate route concept skeleton are adjusted using the policy update signal to generate an optimized version of the candidate route; The simulation deduction, adaptability evaluation, and policy update steps are repeated until the adaptability score vector converges or the maximum number of iterations is reached, and an optimal personalized travel route is output. 8.The deep learning-based multi-modal tourism route personalization generation method of claim 7, wherein, The adaptability score vector is input as a reward signal into a policy gradient-based reinforcement learning agent, and a policy update signal is generated through the agent policy network, specifically: The adaptability score vector is subjected to multi-objective dimension decomposition and normalized weighting to generate a standardized reward component; The standardized reward component and the action probability distribution output by the policy network are fused to construct an advantage function estimate; Based on the advantage function estimate, the original policy gradient of the policy loss function with respect to the network parameters is calculated; the gradient stability of the original policy gradient is analyzed to evaluate its modulus and direction consistency, and a gradient stability index is generated; Based on the gradient stability index, an adaptive clipping threshold is applied to clip the amplitude and correct the direction of the original policy gradient, generating a normalized policy gradient vector; According to the normalized policy gradient vector and the current network parameter state, a policy update signal is output.

9. A deep learning-based multi-modal tourism route personalized generation system, characterized in that, It includes: Multi-modal alignment module: parallel pre-processing and semantic alignment of multi-modal travel data streams from different data sources to extract primary entity features with consistent spatio-temporal annotations; Hypergraph engine: based on the primary entity features, a tourism knowledge hypergraph is constructed and incrementally updated, where nodes represent aligned multi-modal entities, and hyperedges depict high-order interaction relationships between entities through event perception; Preference perception module: using the tourism knowledge hypergraph, the implicit temporal preference and cross-entity collaborative signal of users are extracted through a spatio-temporal perception graph neural network to generate a predictive interest drift model; An intention decoder: deeply fusing the interest drift model with real-time perceived context signals and analyzing the intention, generating a set of candidate route concept skeletons reflecting the user's potential exploration intention by introducing an attention competition mechanism; A simulation deduction module: multi-dimensional simulation deduction and adaptability evaluation are performed on the candidate route concept skeletons, and the robustness of the personalized travel route is finally optimized based on the iterative optimization of reinforcement learning strategy feedback.

Images