Multi-modal reinforcement learning dynamic travel itinerary planning system and method

Abstract

The invention discloses a multi-mode reinforcement learning dynamic travel itinerary planning system and method, and belongs to the technical field of artificial intelligence. The system comprises a multi-modal data input layer, a cognitive decision-making layer, a feedback optimization layer and a dynamic journey planning output layer. The core of the method is as follows: a cognitive decision-making layer cooperatively adopts a near-end strategy optimization PPO algorithm and a deep Q network DQN algorithm to respectively process continuous and discrete actions in a travel; the feedback optimization layer performs closed-loop optimization on the algorithm through a multi-dimensional reward function according to a travel execution result and user feedback; and the dynamic journey planning output layer dynamically adjusts the journey scheme based on the optimized decision and the real-time environment data. According to the method, the problems of static planning and single decision in the prior art are solved, and the individuation, flexibility and user satisfaction of route planning are remarkably improved through algorithm collaboration and dynamic response.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of artificial intelligence, and in particular to a dynamic travel planning system and method based on reinforcement learning. BACKGROUND

[0002] With the booming development of modern tourism, tourists have an increasing demand for personalized dynamic travel planning that can adapt to real-time changes. Traditional travel planning methods, such as early rule-based systems or later systems based on historical data for recommendations, have gradually shown their limitations in the face of complex and changing tourism scenarios and diverse user needs.

[0003] For example, rule-based systems are difficult to cope with changes in attraction information caused by seasonal and weather changes, and cannot meet the personalized preferences of users. Although the planning method based on the recommendation system can recommend popular attractions, it often lacks consideration of real-time dynamic factors such as the current actual flow of the scenic spot and temporary traffic control, resulting in a lack of timeliness and dynamic adaptability of the planning scheme.

[0004] In order to improve the intelligent level of travel planning, the existing technology begins to try to introduce reinforcement learning algorithm. For example, some schemes use deep Q network (DQN) algorithm to deal with discrete decision-making problems in tourism planning (such as choosing which attraction), and DQN algorithm has certain advantages in high-dimensional state space decision-making, but it is weak in dealing with continuous action and state (such as accurate allocation of travel time). At the same time, its experience replay mechanism is easily affected by data sparsity in the tourism scene, resulting in biased value estimation.

[0005] In addition, some schemes also apply proximal policy optimization (PPO) algorithm to resource scheduling and path planning scenarios. PPO algorithm has good stability and sample efficiency when dealing with continuous action space problems, but it has limited ability in deep modeling and integrating multi-modal tourism data (such as combining text introduction, picture information and user evaluation of attractions), making it difficult to fully understand the user's deep preferences.

[0006] In addition, the existing multi-modal framework mostly stays at the level of surface feature splicing of data from different sources, and fails to achieve algorithm-level collaborative optimization, often ignoring the spatio-temporal correlation between data. More importantly, the existing technology generally relies on static environment models, lacks real-time interface with external data sources (such as scenic area management system API, real-time traffic sensors), and cannot form a feedback loop. When unexpected situations occur during travel execution (such as scenic area flow exceeding limit, traffic congestion), the system cannot automatically trigger alternative solutions, and the user needs to manually restart the planning, which greatly reduces the flexibility and feasibility of the travel.

[0007] Therefore, how to design a tourism itinerary planning system and method capable of deeply integrating multi-modal data, collaboratively processing complex discrete and continuous decisions, and dynamically responding to real-time environmental changes is a technical problem to be solved in the field. SUMMARY

[0008] Therefore, the present application aims to at least partially solve at least one of the problems existing in the prior art, such as limited decision-making ability of a single algorithm, insufficient depth of multi-modal data fusion, and lack of real-time response mechanism to dynamic environment.

[0009] The present application provides a multi-modal reinforcement learning dynamic tourism itinerary planning system, characterized by comprising: a multi-modal data input layer configured to obtain multi-modal data including user-side data, scene-side data and dynamic constraint data, and to preprocess and feature extract the multi-modal data to construct a unified feature vector for representing the current decision state; a cognitive decision layer configured to receive the unified feature vector and obtain a historical itinerary state sequence and corresponding actions, dynamic constraint conditions, and to collaboratively decide through a proximal policy optimization (PPO) algorithm and a deep Q network (DQN) algorithm to generate a decision action set, wherein the PPO algorithm processes a continuous action space including time allocation and budget optimization, and the DQN algorithm processes a discrete action space including attraction selection and transportation mode selection; a feedback optimization layer configured to receive itinerary execution result data and user interaction feedback data reflecting the decision action set, calculate a comprehensive reward through a multi-dimensional reward function, and update the policy of the PPO algorithm and the Q network of the DQN algorithm in the cognitive decision layer based on the comprehensive reward; and a dynamic itinerary planning output layer configured to generate and dynamically adjust a user itinerary plan based on the optimized decision action set generated by the updated cognitive decision layer in combination with real-time acquired environmental data.

[0010] Optionally, the multi-modal data input layer comprises a data collection unit for synchronously obtaining text data, image / video data, and spatio-temporal sequence data through multiple channels to constitute the multi-modal data, and a feature extraction and fusion unit for respectively performing feature extraction and standardization processing on the multi-modal data, and dynamically allocating weights of different modal data according to the importance of real-time data through a Transformer-based attention model to obtain the unified feature vector by weighted summation.

[0011] Optionally, the cognitive decision layer is characterized in that the policy network of the PPO algorithm adopts an LSTM+fully connected layer structure for constructing, for capturing time sequence features in state space data and outputting action probability distribution; and the value network of the DQN algorithm adopts CNN or Transformer for constructing, for estimating Q values of state-action pairs.

[0012] Optionally, the cognitive decision layer is further configured to: combine the unified feature vector, historical trip state and dynamic constraint conditions into a comprehensive state space representation; input the comprehensive state space representation into a policy network of the PPO algorithm and a value network of the DQN algorithm respectively to generate continuous action decisions and discrete action decisions respectively, and integrate them into the decision action set.

[0013] Optionally, the cognitive decision layer is further configured to: use the discrete action decisions output by the DQN algorithm as constraint conditions or input features for the PPO algorithm to optimize continuous actions.

[0014] Optionally, the feedback optimization layer is further configured to: output the optimized policy parameters and the comprehensive reward to the cognitive decision layer as reinforcement learning signals to form a decision optimization closed loop.

[0015] Optionally, the multi-dimensional reward function in the feedback optimization layer is configured to: calculate a first reward of a user experience dimension based on user satisfaction scores and attraction matching degrees; calculate a second reward of a system efficiency dimension based on time utilization or budget utilization; calculate a third reward of an environmental adaptation dimension based on real-time traffic costs or crowd congestion avoidance degrees; and form the comprehensive reward by fusing the first reward, the second reward and the third reward through dynamic weights.

[0016] Optionally, the dynamic trip planning output layer is further configured to: determine the user trip plan based on the discrete action decisions and the continuous action decisions in the decision action set; and when detecting that a real-time environmental change exceeds a preset threshold, trigger local adjustment or global re-planning of the user trip plan, and generate at least one alternative plan containing alternative attractions.

[0017] Another aspect of the present application also provides a multi-modal reinforcement learning dynamic tourism trip planning method, which includes multi-modal data processing, collaborative decision making, feedback optimization and dynamic planning steps corresponding to the functions of each level, which are executed by any of the aforementioned system schemes.

[0018] Specifically, the present application provides an integrated intelligent tourism trip planning solution of "perception-decision-learning-planning", deeply collaborates PPO and DQN algorithms, and constructs a dynamic optimization closed loop based on multi-modal data input and real-time environmental feedback. Compared with the prior art, the present application can more accurately understand user individualized preferences, efficiently handle complex discrete and continuous decisions, and make real-time and intelligent adaptive adjustments according to unexpected situations in the trip, significantly improving the accuracy, flexibility and overall experience of the tourism trip planning.

[0019] The above description is only a summary of the technical solutions of the present application. In order to enable one skilled in the art to better understand the technical means of the present application, the contents of the specification can be implemented, and in order to enable the above and other purposes, features and advantages of the present application to be more apparent and easy to understand, the specific embodiments of the present application are described below. BRIEF DESCRIPTION OF DRAWINGS

[0020] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the drawings needed in the embodiments or the prior art description will be briefly introduced. Obviously, the drawings in the following description are embodiments of the present application, and those skilled in the art can obtain other drawings without creative labor on the basis of these drawings.

[0021] In order to more clearly illustrate the technical content of the present application, the embodiments of the present application will be described in detail below with reference to the drawings.

[0022] Figure 1 The functional module structure block diagram of the multi-modal reinforcement learning dynamic travel itinerary planning system provided by the embodiments of the present application is schematically shown; Figure 2 The core flowchart of the multi-modal reinforcement learning dynamic travel itinerary planning method provided by the embodiments of the present application is schematically shown; Figure 3 The example schematic diagram of the dynamic re-planning process in a specific scenario provided by the embodiments of the present application is schematically shown. DETAILED DESCRIPTION

[0023] The embodiments of the present disclosure will be described in detail below with reference to the drawings.

[0024] The embodiments of the present disclosure will be described in detail below with reference to the drawings.

[0025] It is noted that various aspects of the embodiments described below are within the scope of the claims. It should be apparent that the aspects described herein can be embodied in a wide variety of forms and that any specific structure and / or function described herein is merely illustrative. Based on the teachings herein one skilled in the art should appreciate that an aspect described herein can be implemented independently of any other aspects and that an aspect described herein can be implemented both as any claim and as an aspect independent of a related aspect.

[0026] It is also noted that the illustrative figures that are provided in the following embodiments merely show the basic idea of the present disclosure, and only the components related to the present disclosure are shown in the figures, not the number, shape and size of the components as in actual implementation, and the shape, number and proportion of each component in actual implementation can be arbitrarily changed, and the layout of the components can also be more complex.

[0027] In addition, in the following description, specific details are provided in order to facilitate thorough understanding of examples. However, one skilled in the art will understand that the aspects can be practiced without these specific details.

[0028] Referring to Figure 1 , the figure schematically shows the top-level functional module structure block diagram of the multi-modal reinforcement learning dynamic travel itinerary planning system provided by the embodiment. In a preferred embodiment, the system mainly includes: a multi-modal data input layer 10, a cognitive decision layer 20, a feedback optimization layer 30, and a dynamic itinerary planning output layer 40.

[0029] The core function of the multi-modal data input layer 10 is to provide high-quality, standardized data for subsequent intelligent decision-making. Specifically, this layer is configured to obtain multi-modal data containing user-side data, scene-side data, and dynamic constraint data, and to preprocess and extract features from these heterogeneous data, ultimately constructing a unified feature vector representing the current decision state. In a specific embodiment, this layer can include a data collection unit and a feature extraction and fusion unit. The data collection unit synchronously obtains text data (such as user reviews, attraction introductions), image / video data, and spatio-temporal sequence data (such as geographic location, weather forecast) through various channels (such as tourism information platform API, official website of scenic spot, real-time monitoring system, and user interaction interface). The feature extraction and fusion unit then processes these data, for example, performing natural language processing (NLP) on text data to extract semantic features, and performing computer vision (CV) processing on image data to extract visual features. Subsequently, through advanced fusion strategies, such as using a Transformer-based attention model, the weights of different modal data are dynamically assigned according to the importance of real-time data, and the modal features are weighted and summed to obtain the final unified feature vector (such as [user preference vector + scenic visual feature vector + real-time environment vector]) for the cognitive decision-making layer 20's policy learning.

[0030] The core of the cognitive decision-making layer 20 is to cooperatively use two reinforcement learning algorithms to efficiently handle complex and compound decision-making problems in tourism planning. Please refer to Figure 3The figure shows the working principle inside the cognitive decision layer 20. This layer is configured to receive the unified feature vector from the input layer 10, and make collaborative decisions by combining the historical trip state sequence and corresponding actions, dynamic constraint conditions, through the proximal policy optimization (PPO) algorithm and the deep Q network (DQN) algorithm, to generate a set of decision actions. Among them, the historical trip state sequence is like the visited scenic spots, time consumption, and the corresponding actions are like route selection, time allocation decisions; dynamic constraint conditions include but are not limited to real-time input constraint parameters (such as scenic spot closure caused by sudden weather, traffic delay duration). Specifically, the DQN algorithm uses its advantage in discrete action space decision-making to handle tasks such as scenic spot selection, transportation mode selection, etc., to generate a strategy probability distribution. In a preferred embodiment, the value network of the DQN algorithm can be built with a convolutional neural network (CNN) or a Transformer architecture, which is used to efficiently estimate the Q value of the state-action pair and perform state value estimation of discrete actions. At the same time, the PPO algorithm uses its stability and efficiency in continuous action space optimization to handle tasks such as time allocation, budget optimization, etc. Its strategy network can adopt the structure of long short-term memory network (LSTM) and fully connected layer (FC) to effectively capture the time sequence dependency in trip planning. The collaborative working mechanism of the two is: the discrete action decision output by the DQN algorithm can be used as a constraint condition or input feature for the PPO algorithm to optimize continuous actions more accurately. In addition, in order to ensure that the system can find potentially better trip options, the layer can also balance the use of known optimal actions and the exploration of unknown actions through an ε-greedy strategy.

[0031] The feedback optimization layer 30 is configured to receive trip execution result data (such as actual trip trajectory, cost) and user interaction feedback data (such as rating, review) reflecting the set of decision actions, and calculate a comprehensive reward through a multi-dimensional reward function. In a preferred embodiment, the multi-dimensional reward function can include: a user experience dimension reward calculated based on user satisfaction score and scenic spot matching degree; a system efficiency dimension reward calculated based on time utilization rate or budget utilization rate; and an environmental adaptation dimension reward calculated based on real-time traffic cost or crowd congestion avoidance degree. The calculated comprehensive reward will be used to update the strategy of the PPO algorithm and the Q network of the DQN algorithm in the cognitive decision layer 20. At the same time, the optimized strategy parameters and improved reward function can be used as reinforcement learning signals and output again to the cognitive decision layer 20, thereby forming a closed loop of continuous learning and continuous optimization.

[0032] The dynamic trip planning output layer 40 is the human-computer interaction channel between the system and the user, responsible for presenting the results of intelligent decision-making in a user-friendly manner and performing dynamic adjustments. This layer is configured to generate and dynamically adjust the user's trip plan based on the optimized decision action set generated by the updated cognitive decision-making layer 20, combined with real-time environmental data (such as weather changes, traffic congestion). Please refer to Figure 3 , which shows an example of dynamic re-planning. When the system detects that the real-time environmental change (such as the weather forecast changing to "heavy rain") exceeds the preset threshold, it will trigger local or global re-planning of the user's trip plan and generate at least one alternative plan containing alternative scenic spots (such as replacing the outdoor "Jade Dragon Snow Mountain" with the indoor "Museum"), which is pushed to the user.

[0033] In a specific embodiment, the multi-modal data input layer 10 can include a data collection unit and a feature extraction and fusion unit.

[0034] The data collection unit is configured to synchronously collect text data, image / video data, spatio-temporal sequence data, etc. to form the multi-modal data through multiple channels. Specifically, the user-side data can be obtained through user registration information, historical trip records, real-time interaction interfaces, etc., and can include the user's historical trip preferences, interest tags (such as "natural scenery" and "cultural heritage"), real-time location information, demographic characteristics (such as age and gender), and real-time interaction instructions; The scene-side data can be obtained from at least one of the official website of the scenic spot, the tourism information platform, and the multimedia database, and can include multimedia information (such as images, videos, and 3D models) of the scenic spot, text descriptions (such as historical background and tourist reviews), spatio-temporal attributes (such as opening hours and geographic coordinates), and real-time environmental data (such as weather, traffic flow, and scenic flow density); The dynamic constraint data can be obtained through real-time monitoring systems and user feedback, and can include temporary user needs and real-time resource data (such as hotel vacancy rates and transportation ticket availability).

[0035] The feature extraction and fusion unit is configured to perform deep processing on the collected heterogeneous data. Specifically, the unit can perform natural language processing (NLP) on text data to extract keywords and convert them into semantic vectors; perform computer vision processing on image / video data using convolutional neural networks (CNNs such as ResNet models) to extract visual features; and perform standardization processing on spatio-temporal sequence data. In a preferred embodiment, the unit adopts an early fusion strategy, converts the extracted features of various types into multi-dimensional vectors of uniform dimensions (e.g., converts 768-dimensional text semantic vectors output by the BERT model and 2048-dimensional image features extracted by the ResNet model into a uniform 512-dimensional space) through a fully connected layer or an embedding layer, and performs weighted summation to obtain a final unified feature vector. The weights of different modal data features are not fixed but are dynamically allocated based on the importance of real-time data (such as weather and crowd) in the current decision-making scenario through a Transformer-based attention model. In addition, the unit can also construct dynamic time series features based on a sliding time window (e.g., window size 30 minutes, step 10 minutes) and capture the evolution law of traffic congestion through a long short-term memory network (LSTM); at the same time, it models the spatial correlation features between scenic spots (such as the visiting order preference of adjacent scenic spots) through a graph neural network (GNN), thereby achieving more intelligent and context-aware feature fusion.

[0036] Through the all-around data collection by the data collection unit and the deep and dynamic processing by the feature extraction and fusion unit, the multi-modal data input layer 10 of the present application collectively constructs a highly intelligent data perception front end. The front end not only converts heterogeneous, multi-source raw data into standardized feature information understandable by AI models, but also gives these feature information rich context awareness through attention mechanisms and time / spatial modeling, thereby providing high-quality, high-dimensional decision-making basis for subsequent cognitive decision-making layer 20 to make accurate and dynamic planning.

[0037] In a specific embodiment, in order to efficiently handle the complex temporal dependencies and high-dimensional state features in the tourism planning scenario, the present application designs specific neural network architectures for PPO and DQN algorithms.

[0038] Specifically, the policy network of the PPO algorithm, i.e., the Actor network responsible for outputting continuous action decisions, preferably adopts a structure combining long short-term memory networks (LSTMs) and fully connected layers (FCs). It should be noted that the LSTM layer is adopted because it is particularly suitable for processing sequence data and can effectively capture the time sequence features in the input state space data. For example, by analyzing the historical trip state sequence, this structure can learn the dynamic preference changes and timing dependencies of the user in trip planning. The fully connected layer receives the features processed by the LSTM layer, performs further nonlinear transformation and feature extraction, and finally outputs a probability distribution representing continuous actions (such as time allocation and budget proportion).

[0039] At the same time, the value network of the DQN algorithm, i.e., the network responsible for evaluating the value of discrete actions, is preferably constructed using a convolutional neural network (CNN) or a Transformer architecture. This is because, when the input state features include multi-dimensional spatial data such as images, CNN can efficiently extract local spatial features; and when the state features exhibit complex global correlations, the Transformer architecture can effectively capture these correlations through its self-attention mechanism. The core task of this value network is to receive the comprehensive state space representation and accurately estimate the Q value of each state-action pair corresponding to each selectable discrete action in that state.

[0040] Through the above targeted network structure design, the cognitive decision layer 20 of the present application applies the most suitable AI tool to the most suitable task. The application of LSTM ensures that the system can fully consider historical information when making continuous planning (such as time allocation); and the application of CNN or Transformer ensures that the system can capture key local information from complex multi-modal data when making discrete choices. This structural optimization is the key technical basis for the system to achieve efficient and accurate collaborative decision-making, significantly improving the professionalism and effectiveness of the algorithm in processing specific tourism scenario data.

[0041] In a preferred embodiment, the cognitive decision layer 20 ensures that the PPO algorithm and the DQN algorithm can make their respective professional decisions on the basis of unified information through a structured processing flow, and ultimately form a collaborative and consistent output. This flow specifically includes the following steps: First, perform comprehensive state space construction. The cognitive decision layer 20 receives the unified feature vector from the multi-modal data input layer 10 and integrates it with dynamically acquired historical trip states, dynamic constraint conditions (such as the closure of a scenic spot due to sudden weather), and other information. Through this step, the system constructs a comprehensive and integrated state space representation of all static and dynamic information related to the current decision, providing a unified and complete decision basis for subsequent algorithmic decisions.

[0042] Subsequently, the flow enters the parallel decision-making and integration phase. On the one hand, the aforementioned integrated comprehensive state space representation is input into the policy network of the PPO algorithm. The policy network (such as LSTM + fully connected layer structure) analyzes the state representation and generates a probability distribution for each action in the continuous action space. The system then selects a specific continuous action decision based on this probability distribution using methods such as probability sampling, for example, determining the specific stay time at a scenic spot as "3.5 hours". On the other hand, the same integrated state space representation is input into the value network of the DQN algorithm. The value network (such as CNN or Transformer architecture) evaluates the state representation and outputs the Q value corresponding to each selectable action in the discrete action space. The system then selects an optimal discrete action decision based on these Q values using the epsilon-greedy policy, for example, determining the transportation method as "high-speed rail".

[0043] Finally, the cognitive decision-making layer 20 integrates the continuous action decision generated by the PPO algorithm and the optimal discrete action decision generated by the DQN algorithm to form a complete decision action set containing multi-dimensional instructions and outputs it for use by the subsequent dynamic travel planning output layer 40.

[0044] Through the above structured workflow, the cognitive decision-making layer 20 of the present application ensures that its two core algorithms, PPO and DQN, always work on the same, non-informational decision basis. This mechanism of first constructing a unified state, then parallel processing, and finally integrating output effectively avoids decision conflicts that may be caused by information asymmetry, ensuring that the final generated travel plan is highly coordinated and logically consistent in macro selection (such as going to which scenic spot) and micro parameters (such as waiting for how long), which is the key to achieving high-quality, integrated intelligent planning.

[0045] In a preferred embodiment, the core function of the feedback optimization layer 30 is not only to calculate the reward, but also to build a decision optimization closed loop that enables the system to continuously evolve.

[0046] Specifically, after completing the calculation of the comprehensive reward and the optimization of the model parameters, the layer is configured to build the calculated comprehensive reward and the optimized model parameters into a reinforcement learning signal and output it back to the cognitive decision-making layer 20. These two key pieces of information are: Optimized policy parameters: that is, the more optimal PPO policy network parameters and DQN value network parameters updated after this round of learning.

[0047] The calculated comprehensive reward in this round: this reward value itself is also part of the reinforcement learning signal, which can be used by the cognitive decision layer to make more complex value assessments or adjustments.

[0048] It should be noted that this reinforcement learning signal is the key to driving the entire system to achieve "learning". When the cognitive decision layer 20 receives this signal, it replaces its original, old model parameters with the "optimized strategy parameters" in it. Through this closed-loop feedback and updating action, the cognitive decision layer has a more "experienced" model as the starting point for its decision-making when planning the next trip.

[0049] Through the design of the above decision optimization closed loop, the present application ensures that the system is not a static, one-time planning tool, but a dynamic intelligent agent that can learn from each user interaction and trip execution result, continuously iterate and optimize itself. This ability to continuously learn is the fundamental guarantee for the system to adapt to changes in user preferences, cope with new tourism scenarios, and maintain high planning quality at all times.

[0050] In a preferred embodiment, the multi-dimensional reward function in the feedback optimization layer 30 comprehensively evaluates the trip decision from at least three dimensions: the user experience dimension, which aims to quantify whether the trip meets the user's individualized preferences and subjective feelings; the system efficiency dimension, which aims to evaluate the efficiency of the trip plan in terms of objective resources such as time and budget; and the environmental adaptation dimension, which aims to measure the adaptability and avoidance ability of the trip plan to real-time dynamic environmental changes.

[0051] To elaborate on the calculation details of the user experience dimension, it is particularly important to note that a key component of this dimension reward is the experience reward for quantifying user implicit feedback, which is preferably implemented through an exponential function model. The model calculates the impact of the deviation between the actual travel time and the planned time as follows: where, represents the time reward, and represents the reward score converted from the matching degree of actual time and planned time. is the baseline reward coefficient, which determines the maximum possible value of the reward (when the time deviation is 0, = ). is the time sensitivity coefficient (positive number), which controls the speed of reward decay as the time deviation increases. The larger the coefficient, the more sensitive the system is to time deviation. represents the actual completion time (e.g., the end time of a scenic spot tour, the arrival time of transportation). Scheduled time (pre-planned time point). Absolute deviation of actual time from scheduled time, taking absolute value to ensure non-negativity (regardless of advance or delay, deviation is calculated as a positive value).

[0052] Then, we integrate the reward calculation of each dimension to fully demonstrate the mathematical construction of the entire reward system.

[0053] First, calculate the first reward of the user experience dimension. This reward aims to quantify whether the itinerary meets the user's personalized preferences and subjective feelings. In a specific embodiment, the reward value can be calculated by the following formula: Where, represents the first reward, and the total reward value after integrating the multi-dimensional expected score, used to guide the policy optimization of the reinforcement learning agent; respectively represent the weight coefficients of each expected score; Score reward, usually from user satisfaction ratings (such as 1-5 star conversion to numerical value) or objective indicators (such as attraction ratings, service evaluations); represents the time reward, quantifying the reward / punishment of time deviation through an exponential function model Preference reward, indicating the degree to which the itinerary meets the user's personalized preferences (such as whether it contains the user's preferred types of attractions, dining styles, etc.).

[0054] Second, calculate the second reward of the system efficiency dimension. This reward aims to evaluate the efficiency of itinerary planning in terms of objective resources such as time and budget. In a specific embodiment, the reward value can be calculated by the following formula: Second reward, total reward value after integrating multi-dimensional efficiency indicators, used to evaluate the overall effectiveness of the decision scheme and guide policy optimization; respectively represent the weight coefficients of each efficiency indicator; Time utilization reward, an indicator that quantifies the efficiency of resource (such as time) usage, usually related to the deviation between planned time and actual time spent; represents the budget utilization reward, an indicator that measures the efficiency of cost control, usually related to the matching degree between budget consumption and actual expenditure (for example: positive reward when expenditure is lower than budget and meets demand); represents the task completion rate reward, indicating the degree of completion of the target task (such as the rate of checking in at attractions in itinerary planning, the completion of milestones in projects).

[0055] ​Third, the third reward of the environmental adaptation dimension is calculated. The reward aims to measure the adaptability and avoidance ability of the trip scheme to real-time dynamic environmental changes. In a specific embodiment, the reward value can be calculated by the following formula: wherein, represents the third reward, and the total reward value after integrating the traffic cost and the peak avoidance demand, used to evaluate the adaptability of the scheme in the dynamic scenario and guide the policy optimization; respectively represent the adaptive weight coefficients of each environmental dimension; the traffic cost reward, which quantifies the index of travel cost efficiency, is usually related to the cost, time consumption or energy consumption (the lower the cost, the higher the reward); the peak avoidance reward, which measures the degree of avoiding crowded areas or time periods (the lower the crowd density or the shorter the waiting time, the higher the reward).

[0056] Finally, the first reward, the second reward and the third reward are fused through dynamic weights to form the final comprehensive reward . For example, the system can automatically increase the weight of a certain dimension according to the feedback mode of the user, so as to realize deeper personalized adaptation.

[0057] Through the above precise multi-dimensional reward function design based on specific formulas, the feedback optimization layer 30 of the application can build a highly accurate, comprehensive and personalized evaluation system. The system not only provides clear and effective learning signals for reinforcement learning, but also guides the system to learn a truly high-quality dynamic trip planning strategy that meets both user subjective preferences and objective efficiency and environmental changes.

[0058] Further, after calculating the comprehensive reward, the feedback optimization layer 30 updates the PPO and DQN network parameters in the cognitive decision layer 20 using the comprehensive reward. This core step provides solid technical support for updating the policy of the PPO algorithm and the Q network of the DQN algorithm in the cognitive decision layer. The specific updating process is as follows: For the policy update of the PPO algorithm, the generalized advantage estimation (GAE) is preferably used to calculate the advantage function, and the policy network parameters are updated through the clipping objective function specific to PPO. For the value function update of the DQN algorithm, the Q network parameters are updated by minimizing the TD error.

[0059] Through the above complete closed-loop process from reward calculation to model updating, the feedback optimization layer 30 of the application can ensure that the system effectively learns from each trip planning and execution, and quantifies it as an accurate mathematical signal that can guide the iteration of the algorithm model, thereby realizing the continuous and stable evolution of the system's decision-making ability.

[0060] In a preferred embodiment, to ensure that the system not only follows the known optimal strategy, but also continuously discovers new, possibly better itinerary options, the cognitive decision-making layer 20 also introduces a mechanism of exploration-exploitation balance.

[0061] Specifically, the mechanism is preferably implemented through an ε-greedy strategy, especially when the DQN algorithm selects discrete actions. Under this strategy, the system will randomly select an action for exploration with a preset small probability ε (for example, ε = 0.1) when making decisions, rather than selecting the action with the highest current Q value. While under a larger probability of 1-ε, the system will select the action with the highest current Q value for exploitation. The existence of this exploration mechanism enables the system to avoid falling into a local optimal solution (for example, avoiding recommending only a few popular attractions forever due to historical data bias). Through active exploration, the system has the opportunity to discover new attractions, restaurants or activities that are unpopular but highly rated by users, thereby greatly enriching the diversity of planning options and bringing unexpected surprises to users.

[0062] In a preferred embodiment, the dynamic itinerary planning output layer 40 is the human-computer interaction interface between the system and the user, which can be manifested as an interactive interface, responsible for converting the abstract decisions output by the cognitive decision-making layer 20 into specific, feasible and user-friendly itinerary plans, and performing the core dynamic adjustment function.

[0063] Specifically, this layer is configured to first perform the generation and integration of itinerary plans. It receives the set of decision actions from the cognitive decision-making layer 20 and integrates the discrete actions (such as attraction selection, transportation mode) determined by the DQN algorithm with the continuous actions (such as stay time allocation, budget proportion) determined by the PPO algorithm to form a structured, complete itinerary plan containing time window constraints, thereby determining the initial user itinerary plan.

[0064] The core function of this layer is its dynamic adjustment mechanism. The dynamic itinerary planning output layer 40 continuously receives real-time feature streams (such as the latest traffic status, weather changes, attraction flow density, etc.) from the multi-modal data input layer 10. When detecting that the real-time environmental changes exceed a preset threshold (for example, traffic delay exceeds 15 minutes, or attraction flow density exceeds 80% of the carrying capacity), the system will automatically trigger adjustment of the user's itinerary plan.

[0065] The adjustment can be a local adjustment, i.e., only the affected segment of the itinerary is re-planned, for example, only the afternoon itinerary is re-planned without changing the morning schedule. The adjustment can also be a global re-planning, i.e., when the environmental changes have a significant impact on the entire subsequent itinerary, the entire itinerary is re-optimized.

[0066] During the re-planning process, the system generates at least one alternative solution containing alternative attractions. For example, when a heavy rain warning is monitored, the system automatically recommends indoor alternative attractions that match the user's interest tags. To better cope with uncertainties, this layer can also adopt a flexible time allocation strategy to reserve a floating time block for high-risk activities.

[0067] Finally, this layer also processes the itinerary data into a user-friendly multi-modal presentation format, such as generating a text summary with key point highlights, a timeline, a cost distribution chart, etc. visualizations, and supporting user operations such as drag-and-drop adjustments, voice interactions, etc. through an interactive interface.

[0068] Through the above series of precise operations such as itinerary generation, conflict detection, dynamic adjustment, and friendly presentation, the dynamic itinerary planning output layer 40 of the present application ensures that the final output to the user is not only an AI-based intelligent planning result, but also a reliable and flexible dynamic itinerary solution that can be effectively executed in the real world and can handle unexpected situations with ease.

[0069] Another aspect of the present application also provides a multi-modal reinforcement learning dynamic travel itinerary planning method. Please refer to Figure 2 The figure schematically shows the core flowchart of the method embodiment of the present application, which is preferably executed by the aforementioned system and corresponds to each functional module, specifically including the following steps: S10, start.

[0070] S20, acquire and process multi-modal data.

[0071] In this S20 step, the system acquires heterogeneous multi-modal data from multiple dimensions such as users, scenes, and real-time environments through its multi-modal data input layer 10, and processes these data in depth through natural language processing, computer vision, etc. to construct a unified feature vector that can fully and accurately represent the current decision state.

[0072] S30, cognitive decision-making layer collaborative decision-making.

[0073] Based on the unified feature vector constructed in the previous step, the system collaboratively makes decisions through its cognitive decision-making layer 20 using the proximal policy optimization PPO algorithm and the deep Q network DQN algorithm. Specifically, the DQN algorithm is responsible for handling discrete actions in itinerary planning, such as attraction selection; the PPO algorithm is responsible for handling continuous actions, such as time allocation. It should be noted that the collaboration of the two is preferably achieved through a hierarchical mechanism: that is, the discrete action decisions output by the DQN algorithm are used as constraint conditions or input features to allow the PPO algorithm to optimize continuous actions more accurately. After completing the decision, the system generates a decision action set containing discrete and continuous actions.

[0074] S40, generating / dynamically adjusting the user itinerary scheme.

[0075] The system generates or dynamically adjusts the user itinerary scheme based on the decision action set generated in the previous step and in combination with the real-time acquired environmental data through the dynamic itinerary planning output layer 40.

[0076] S50, receiving execution results and user feedback.

[0077] During the execution of the itinerary, the method enters the feedback and optimization step. The system continuously receives feedback data reflecting the execution results of the decisions and user feedback.

[0078] S60, feedback optimization layer calculates rewards and updates the model.

[0079] The feedback optimization layer 30 of the system quantitatively evaluates these feedbacks through a multi-dimensional reward function to calculate the comprehensive reward, and optimizes and updates the strategy of the PPO algorithm and the Q network of the DQN algorithm in the cognitive decision layer 20 based on the comprehensive reward. The multi-dimensional reward function preferably calculates the comprehensive reward based on at least one of user satisfaction score, system efficiency and environmental adaptability to guide the optimization and update. This "feedback-optimization-update" process, together with the aforementioned decision step, forms a reinforcement learning closed loop that enables the system to continuously learn and evolve.

[0080] S70, determining whether the itinerary is over.

[0081] If the entire itinerary has not ended (e.g., the day before a multi-day tour ends), the process will execute the "No" branch and return to the collaborative decision step S20 to start planning the next stage of the itinerary; if the itinerary has ended completely, the "Yes" branch is executed and step S80 is executed.

[0082] S80, end.

[0083] Through the above series of precise and collaborative method steps, the present application constructs a complete method process from intelligent perception, collaborative decision, dynamic planning to closed-loop learning. The method not only solves the limitations of existing technologies in dealing with complex tourism planning problems, but also ensures that the generated itinerary scheme can maintain high personalization, timeliness and reliability in the long term through its continuous self-optimization capability.

[0084] In order to more clearly show the dynamic adjustment capability of the system described in the present application in actual application, the following will be combined Figure 3 , a specific dynamic re-planning scene instance is described in detail.

[0085] In a typical application scenario, it is assumed that a user's core preference is "natural scenery", but also hopes "itinerary comfortable".

[0086] Step 1: Initial planning. The system first makes an initial plan based on the user's preferences and attraction data. As shown in Figure 3 , the DQN algorithm of the cognitive decision-making layer 20 selects "Jade Dragon Snow Mountain" as the core tour target (discrete decision) according to the user's high-weighted preference for "natural scenery"; at the same time, the PPO algorithm allocates it a 5-hour tour time and the corresponding transportation budget (continuous decision).

[0087] Step 2: Dynamic perception. Before the trip starts, the multi-modal data input layer 10 continuously obtains real-time environmental data from external data sources (such as meteorological agencies) through its API interface, such as real-time weather data through the weather API. As shown in Figure 3 , the system obtains the next day's weather forecast as "heavy rain" the night before the trip. This new dynamic information is immediately processed and used to update the current decision state vector, with the "weather" feature dimension being assigned a negative weight, determining that the risk of outdoor activities is high.

[0088] Step 3: Intelligent re-planning. The updated decision state vector is passed to the cognitive decision-making layer 20 and triggers the re-planning mechanism. As shown in Figure 3 , due to the negative environmental feature of "rainy day", the Q value of the originally high-Q outdoor activity "touring Jade Dragon Snow Mountain" drops significantly. At this time, the DQN algorithm re-evaluates all available actions and finds that the Q value of the indoor attraction "Lijiang Ancient City Museum" becomes the highest, so it outputs a new discrete decision. In turn, the PPO algorithm receives this new attraction decision as a constraint condition and re-optimizes the continuous action, adjusting the tour time to 2.5 hours and re-planning the budget for the city's transportation.

[0089] Step 4: Plan output. Finally, the dynamic trip planning output layer 40 integrates the results of the re-planning into a clear alternative plan that includes the adjustment reason and pushes it to the user through the user interaction interface. As shown in Figure 3 , the plan clearly indicates that the new attraction is "Lijiang Ancient City Museum" and the new tour time is 2.5 hours, and explains the adjustment reason "indoor attraction, avoiding rain risk", and finally pushes the alternative plan to the user.

[0090] In another specific embodiment, taking "five-day four-night Yunnan tour" as a scenario, the operation process and algorithm details of each layer of the multi-modal reinforcement learning dynamic tourism trip planning system are elaborated in detail.

[0091] I. Multi-modal data input layer (A) Input data User preference: Fill in the questionnaire "natural scenery (80%), ethnic culture (70%), special food (60%)", budget 5000 yuan, daily walking no more than 15000 steps, refuse high-intensity hiking.

[0092] Attraction information includes Jade Dragon Snow Mountain (natural scenery, ticket 100 yuan, 5-hour tour, including cable car and Blue Moon Valley), Lijiang Ancient City (ethnic culture, free, 3-hour tour, including Wood House and Sifang Street), Dali Erhai (cycling / boat, partially charged, surrounding Bai villages), etc. with pictures, opening hours, and tourist review text.

[0093] Real-time data includes: weather forecast information, i.e. rain in Lijiang the next day; scenic area flow information, i.e. Lijiang Ancient City tourist volume reaches 80% of the carrying capacity; traffic condition information, i.e. high-speed rail from Kunming to Dali has limited tickets, which needs to be purchased in advance.

[0094] (II) Processing flow First, data cleaning and feature extraction, for text data: use NLP technology to extract key words of attraction description (such as "Jade Dragon Snow Mountain - glacier" and "Lijiang Ancient City - Naxi architecture"), and quantify user preference as a vector; for image data: extract visual features of attraction pictures through CNN (such as snow texture of Jade Dragon Snow Mountain and lake color of Erhai); for structured data: normalize opening hours, ticket prices, etc.

[0095] Then, multi-modal fusion: concatenate user preference vector, attraction text-image features, and real-time data (weather code, flow ratio) into a unified feature matrix, and pass it to the cognitive decision layer.

[0096] II. Cognitive decision layer 1. The flow of DQN algorithm generating discrete action decisions is as follows: The core task is to select attractions, transportation methods, and activity types, etc.

[0097] The state is defined as the current location, remaining budget, time, user preference, and real-time environment (such as "the next morning in Lijiang, budget remaining 4000 yuan, rainy day"). The action space is defined as {visit Jade Dragon Snow Mountain, visit Lijiang Ancient City, go to Dali Erhai, take high-speed rail, take a private car, etc.}.

[0098] Input state features through deep neural network (CNN+LSTM), output Q value of each action (such as "visit Lijiang Ancient City: Q=12, Jade Dragon Snow Mountain: Q=8"). Q value represents the expected cumulative reward after executing the action (combined with historical data and current state).

[0099] Use the ε-greedy strategy to randomly select actions with an ε probability (explore new options, such as trying a less popular scenic spot) and select the action with the highest Q value with a 1-ε probability (use known optimal solutions, such as choosing the popular Lijiang Ancient City).

[0100] Store historical decisions (state s, action a, reward r, next state s') to avoid data correlation; periodically synchronize Q network parameters, calculate target Q values, and reduce training fluctuations; use mean square error (MSE) minimization to update Q network parameters.

[0101] Example: The next day, due to rain, DQN compares "indoor tour of Lijiang Ancient City (Q=12)" with "adventure to Jade Dragon Snow Mountain (Q=8)", and chooses the former, storing the decision in the experience pool for subsequent optimization.

[0102] 2. PPO algorithm generates continuous action decision-making process as follows: The core task is to optimize the time spent at scenic spots, budget allocation, and travel intensity, among other continuous variables.

[0103] Input state features into the policy network to output action probability distribution, such as the probability of "staying in Lijiang Ancient City for 3.5 hours"; evaluate the long-term value of the current state through the value network, such as the expected satisfaction of "touring Lijiang Ancient City for 3.5 hours".

[0104] Example: If a user is satisfied with a 3-hour stay in Lijiang Ancient City with a satisfaction score of 4, while the average stay time corresponds to a satisfaction score of 3.5, it indicates that this decision is better than the average.

[0105] If the system attempts to extend the stay in Lijiang Ancient City from 3 hours to 4 hours, but historical data shows that users are prone to fatigue (advantage function is negative), the clip mechanism will prevent excessive adjustment. Through repeated sampling, calculating the advantage function, and updating the network, continuous decisions are gradually optimized, such as finding that the optimal stay time for natural scenery spots is 4-6 hours.

[0106] 3. Dual algorithm coordination mechanism First, determine the discrete action (such as "go to Erhai on the third day") using the DQN algorithm, and then optimize the continuous parameters (such as "ride 3 hours in Erhai, allocate 15% of the budget for rental and dining") using the PPO algorithm.

[0107] It should be noted that both DQN and PPO algorithms evaluate decisions based on the same reward function (such as user satisfaction and budget compliance); the discrete decision actions of DQN (such as "choose Lijiang Ancient City on rainy days") and the continuous parameter optimization of PPO algorithm (such as "stay in the ancient city for 3.5 hours") work together to perfect the strategy.

[0108] Three, feedback optimization layer 1. Calculate each reward according to the user's actual itinerary (e.g. 3 hours of cycling on Erhai Lake on the third day, spending 300 yuan, scoring 4 points) and weighted sum to get the final reward.

[0109] 2. Strategy optimization process Store state, action, and final reward into experience pool for cognitive decision layer to update network.

[0110] If a certain decision (e.g. "choose a niche attraction") consistently receives high rewards, increase the Q value of that action in DQN; if PPO's dwell time decision often leads to reduced rewards (e.g. the user ends the tour early), optimize the dwell time prediction model.

[0111] Four, dynamic itinerary planning output layer Integrate the decision action set (attractions, time, budget) output by the cognitive decision layer into a daily itinerary, such as: 09:00-12:00 cycling on Erhai Lake (rental fee 50 yuan) on the third day, 12:30-14:00 lunch at Xizhou Ancient Town (spending 100 yuan).

[0112] Conflict detection and adjustment: Check real-time data (e.g. Erhai cruise is suspended due to wind), trigger DQN to reselect alternative attractions (e.g. change to Shuanglang Ancient Town); PPO reallocates time and budget (e.g. reduce cycling time, increase ancient town tour).

[0113] Preferably, generate a photo and text guide, map navigation route, and budget breakdown, and push them to the user's APP.

[0114] Complete example summary When a user who prefers "natural scenery" but wants a "comfortable itinerary" plans the next day's itinerary, the DQN algorithm of the cognitive decision layer selects "Jade Dragon Snow Mountain" as the core tour target (discrete decision) based on user preferences and attraction data, and the PPO algorithm allocates 5 hours of tour time and corresponding transportation budget (continuous decision).

[0115] The night before the trip, the system obtains the next day's weather forecast as "heavy rain" through the multi-modal data input layer and updates this information to the decision state. The cognitive decision layer receives the new state and re-evaluates all available actions. Due to the negative environmental feature of "rainy day", the Q value of the outdoor activity "touring Jade Dragon Snow Mountain", which originally had a high Q value, drops significantly. At this time, the DQN algorithm finds that the Q value of the indoor attraction "Lijiang Ancient City Museum" becomes the highest, so it outputs a new discrete decision. Then the PPO algorithm receives the new attraction decision and optimizes the continuous action, adjusting the tour time to 2.5 hours and re-planning the budget for city transportation.

[0116] The user accepted the alternative and gave a "very good" text evaluation and a 4-star rating after the tour. The feedback optimization layer received this positive feedback, calculated a higher overall reward, and updated the network parameters of PPO and DQN in the cognitive decision-making layer based on this reward. This successful adjustment strengthened the weight of the strategy "rainy day + preference for natural scenery -> recommend high-quality indoor cultural landscape" in the model, and the system became more intelligent.

[0117] Through the above examples, the application vividly demonstrates how it transforms a static, one-time planning task into a dynamic, intelligent, and self-adjusting adaptive process that can adapt to real-time environmental changes. This process not only embodies the precise coordination of PPO and DQN algorithms, but also significantly improves the feasibility of the travel plan in the real world and the actual experience of the user through a complete "perception-decision-output" closed loop, perfectly solving the technical pain points of the existing technology that the planning scheme is out of touch with the actual situation.

[0118] The multi-modal reinforcement learning dynamic tourism itinerary planning system and method provided by the embodiments of the present application realizes dynamic and accurate planning of personalized itineraries through an integrated and closed-loop self-optimizing intelligent system. First, the system fundamentally solves the contradiction between discrete selection (such as attraction selection) and continuous optimization (such as time allocation) that traditional single algorithms cannot balance by constructing a collaborative decision-making mechanism based on PPO and DQN algorithms in the cognitive decision-making layer. This approach perfectly solves the problem of complex decision-making in the tourism planning scenario, shortens the decision-making time of itinerary planning by more than 30%, and improves the accuracy by about 40%. Second, the system creatively uses a multi-modal deep fusion strategy based on the Transformer attention model to accurately understand the user's real intentions through context perception of heterogeneous data such as text, images, and space-time. This system perfectly solves the problem of superficial data utilization in existing technologies that cannot truly understand user preferences, making the user satisfaction score of the planned itinerary 25% higher than traditional recommendation methods. More importantly, the system creatively applies the feedback optimization closed loop of reinforcement learning to the field of tourism planning, enabling the system to learn from experience and continuously evolve through a fine multi-dimensional reward function based on each trip execution and user feedback. By real-time access to dynamic data such as tourist flow, traffic, and weather and responding to them, the system solves the fundamental defect of traditional schemes that cannot adapt to environmental changes, shortens the response time of dynamic itinerary adjustment to minutes, and greatly enhances the reliability of the itinerary.

[0119] Through the synergistic effect of the above-mentioned key technical links, the application overcomes the limitations of traditional planning tools, builds a new intelligent planning paradigm that can adapt to users, content and real-time environment, and significantly improves the intelligent level and comprehensive user experience of trip planning in dynamic and complex scenarios.

[0120] It should be understood that, although each step in the flowchart of the accompanying drawings is shown in sequence according to the direction of the arrow, these steps are not necessarily executed in sequence according to the direction of the arrow. Unless otherwise specified herein, the execution of these steps is not strictly limited in sequence, and they can be executed in other sequences. Moreover, at least part of the steps in the flowchart of the accompanying drawings can include multiple sub-steps or multiple stages, which are not necessarily executed at the same time, but can be executed at different times, and the execution sequence is not necessarily sequential, but can be alternately or alternately executed with at least part of other steps or sub-steps or stages of other steps.

[0121] As referred to herein, "one embodiment," "an embodiment," or "one or more embodiments," means that a particular feature, structure, or characteristic described in connection with the embodiment is included in at least one embodiment of the application. The appearances of the phrase "in one embodiment" in various places in the specification are not necessarily all referring to the same embodiment.

[0122] In the specification provided herein, a large number of specific details are described. However, it can be understood that the embodiments of the application can be practiced without these specific details. In some instances, well-known methods, structures and techniques are not shown in detail in order not to obscure the understanding of the present specification.

[0123] In the claims, any reference signs placed between parentheses shall not be construed as limiting the claim. The word "comprising" does not exclude the presence of elements or steps not listed in a claim. The word "a" or "an" preceding an element does not exclude the presence of a plurality of such elements. The application can be implemented by means of both hardware and software, and any combination thereof. In a unit claim, several devices can be listed with a comma. The use of the term "first", "second", "third", etc. does not limit the number for these devices. These terms can be construed as names.

[0124] Finally, it should be noted that the above examples are only used to illustrate the technical solutions of the present application, and are not intended to limit the same; although the present application has been described in detail with reference to the foregoing examples, those of ordinary skill in the art should understand that they can still modify the technical solutions recorded in the foregoing examples, or make equivalent replacements for some of the technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. A multimodal reinforcement learning dynamic tourism itinerary planning system, characterized in that, include: The multimodal data input layer is configured to acquire multimodal data containing user-side data, scene-side data, and dynamic constraint data, and to preprocess and extract features from the multimodal data to construct a unified feature vector representing the current decision state. The cognitive decision layer is configured to receive the unified feature vector and obtain the historical travel state sequence and corresponding actions and dynamic constraints. It performs collaborative decision-making through the Proximal Policy Optimization (PPO) algorithm and the Deep Q Network (DQN) algorithm to generate a set of decision actions. The PPO algorithm processes the continuous action space including time allocation and budget optimization, and the DQN algorithm processes the discrete action space including attraction selection and transportation mode selection. The feedback optimization layer is configured to receive process execution result data and user interaction feedback data reflecting the set of decision actions, calculate a comprehensive reward through a multi-dimensional reward function, and update the strategy of the PPO algorithm and the Q network of the DQN algorithm in the cognitive decision layer based on the comprehensive reward. The dynamic trip planning output layer is configured to combine real-time acquired environmental data with an optimized set of decision actions generated by the updated cognitive decision layer to generate and dynamically adjust user trip plans.

2. The system according to claim 1, characterized in that, The multimodal data input layer includes: The data collection unit is used to simultaneously acquire text data, image / video data, and spatiotemporal sequence data through multiple channels to form the multimodal data; The feature extraction and fusion unit is used to extract and standardize features from text data, image / video data, and spatiotemporal sequence data in the multimodal data, respectively. It adopts an early fusion strategy, and transforms the features extracted from text data, features extracted from image / video data, and standardized spatiotemporal sequence data into multidimensional vectors through an embedding layer, and then performs weighted summation to obtain the unified feature vector. The weights of different modal data are dynamically allocated according to the importance of real-time data through a Transformer-based attention model.

3. The system according to claim 1, characterized in that: The policy network of the PPO algorithm is constructed using an LSTM+fully connected layer structure to capture time-series features in the state-space data and output the action probability distribution. The value network of the DQN algorithm is constructed using CNN or Transformer to estimate the Q-value of state-action pairs.

4. The system according to claim 1, characterized in that, The cognitive decision-making layer is also configured as follows: The unified feature vector, historical travel state, and dynamic constraints are combined and integrated into a comprehensive state-space representation; The integrated state space representation is input into the policy network of the PPO algorithm and the value network of the DQN algorithm to generate continuous action decisions and discrete action decisions, respectively, and then integrated into the decision action set.

5. The system according to claim 4, characterized in that, The cognitive decision-making layer is also configured as follows: The discrete action decisions output by the DQN algorithm are used as constraints or input features for the PPO algorithm to optimize continuous actions.

6. The system according to any one of claims 1 to 5, characterized in that, The feedback optimization layer is also configured to: The optimized strategy parameters and the comprehensive reward are used as reinforcement learning signals and output to the cognitive decision-making layer to form a closed loop for decision optimization.

7. The system according to any one of claims 1 to 5, characterized in that, The multi-dimensional reward function in the feedback optimization layer is configured to perform the following operations: The first reward is calculated based on user satisfaction ratings and attraction matching accuracy, representing the user experience dimension. The second reward is calculated based on the system efficiency dimension, which is determined by time utilization or budget utilization. The third reward is calculated based on real-time traffic costs or crowd congestion avoidance. The first reward, the second reward, and the third reward are dynamically weighted and combined to form the comprehensive reward.

8. The system according to any one of claims 1 to 5, characterized in that, The dynamic travel planning output layer is also configured as follows: The user's itinerary plan is determined based on the discrete action decisions and continuous action decisions in the set of decision actions; When real-time environmental changes exceed a preset threshold, a partial adjustment or global replanning of the user's itinerary is triggered, and at least one alternative plan containing alternative attractions is generated.

9. A multimodal reinforcement learning dynamic tourism itinerary planning method, characterized in that, include: Acquire multimodal data containing user-side data, scene-side data, and dynamic constraint data, and preprocess and extract features from the multimodal data to construct a unified feature vector to characterize the current decision state; The system receives the unified feature vector and dynamic constraints, and obtains the historical travel state sequence and corresponding actions. It then uses a proximal strategy optimization PPO algorithm to process continuous actions and a deep Q-network DQN algorithm to process discrete actions, generating a set of decision actions. Receive trip execution result data and user interaction feedback data reflecting the set of decision actions, calculate the comprehensive reward through a multi-dimensional reward function, and update the strategy of the PPO algorithm and the Q network of the DQN algorithm based on the comprehensive reward; By combining real-time acquired environmental data and based on the optimized decision action set generated by the updated PPO algorithm and DQN algorithm, user travel plans are generated and dynamically adjusted.

10. The method according to claim 9, characterized in that, In the step of coordinating the use of the PPO algorithm and the DQN algorithm: The discrete action decisions output by the DQN algorithm are used as constraints or input features for the PPO algorithm to optimize continuous actions.

Images