Intelligent tourism personalized service recommendation method based on reinforcement learning
By constructing user feature fusion and latent variable modeling through reinforcement learning methods, and combining them with adaptive policy optimization, the problems of user interest evolution and environmental adaptation in smart cultural tourism personalized recommendation are solved, and efficient personalized recommendation is achieved.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- FOCALCREST LTD
- Filing Date
- 2026-03-05
- Publication Date
- 2026-06-05
AI Technical Summary
Existing smart tourism personalized recommendation methods lack the ability to model the evolution of user interests in dynamic and non-stationary user behavior environments. They also have limited context adaptation capabilities, fixed strategy scheduling mechanisms, and low feedback utilization efficiency, making it difficult to achieve accurate personalized service recommendations.
We employ a reinforcement learning-based approach to construct a reinforcement learning model that integrates user feature fusion modeling, contextual multi-armed gambling machine mechanism, and latent variable fusion. Combined with an adaptive policy optimization mechanism, we dynamically model user interests and update recommendation policies, and introduce a multimodal environment context awareness and feedback-driven behavior update mechanism.
It enables dynamic modeling of user interests and personalized recommendations, improves the context adaptability and accuracy of recommendation systems, solves the shortcomings of traditional methods in adapting to changes in user interests and the environment, and improves recommendation response capabilities and strategy optimization efficiency.
Smart Images
Figure CN122153161A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of artificial intelligence technology, and in particular to a method for recommending personalized smart cultural tourism services based on reinforcement learning. Background Technology
[0002] With the rapid development of the smart tourism industry, users' demand for personalized and intelligent service experiences is constantly growing. Personalized recommendation technology, as an important means to improve user satisfaction and platform operational efficiency, is gradually becoming a key component of smart tourism service platforms. However, user behavior in smart tourism scenarios exhibits significant contextual dependence, interest drift, and interaction complexity, leading to multiple challenges for traditional recommendation algorithms in practical applications.
[0003] In existing technologies, mainstream personalized recommendation methods mainly rely on collaborative filtering, content-based recommendation, or deep learning models. These methods are effective in handling static preference modeling and preliminary recommendation tasks, but they still have significant shortcomings in dynamic and non-stationary user behavior environments, primarily manifested in the following ways: 1. Lack of modeling ability for the evolution of user interests: Traditional methods usually treat user interests as static vectors, which cannot effectively capture the user's preference migration behavior as time and context change.
[0004] 2. Limited context adaptation capability: Most methods fail to fully integrate the user's current environmental factors, such as geographical location, time period, holiday activities, etc., resulting in the recommended content being out of touch with actual needs.
[0005] 3. Fixed strategy scheduling mechanism: Existing methods usually adopt static recommendation strategies, which lack a dynamic balancing mechanism between exploration and utilization, making it difficult to continuously optimize the recommendation effect in long-term interactions.
[0006] 4. Inefficient use of feedback: User feedback data is not fully utilized in traditional recommendation systems, making it difficult to achieve dynamic updates and personalized evolution of strategies based on feedback.
[0007] Therefore, how to provide a smart cultural tourism personalized service recommendation method based on reinforcement learning is a problem that urgently needs to be solved by those skilled in the art. Summary of the Invention
[0008] One objective of this invention is to propose a smart cultural tourism personalized service recommendation method based on reinforcement learning. This invention employs user feature fusion modeling, contextual multi-armed gambling machine mechanism, reinforcement learning model with fused latent variables, and adaptive policy optimization mechanism to achieve dynamic modeling of user interests, generation of recommendation policy distribution, and feedback-driven policy update. It has the advantages of strong context adaptability, high personalization accuracy, and strong recommendation response capability.
[0009] A method for recommending personalized smart tourism services based on reinforcement learning according to an embodiment of the present invention includes the following steps: S1. Obtain the user's historical behavior data and current environmental context information in the smart cultural tourism service platform, extract the user's long-term interest feature vector and short-term behavior feature vector, and construct the user feature fusion vector; S2. Based on the user feature fusion vector, construct an improved contextual multi-armed gambling machine model. Use the user's long-term interest feature vector and the user's short-term behavior feature vector as context input, calculate the initial reward estimate of the candidate recommendation content, and generate the recommendation strategy distribution. S3. Construct a reinforcement learning model that integrates latent variable modeling mechanism, define the state space of user interest evolution, take historical behavior data, current environmental context information and recommendation strategy distribution as input, and initialize user potential preference state variables and state transition function; S4. Input the current environmental context information, historical behavior data, user potential preference state variables, recommendation strategy distribution and initial reward estimate into the reinforcement learning model, learn the state transition function and action selection strategy through the policy optimization algorithm, and output a personalized recommendation strategy. S5. Perform recommendation actions on candidate content according to the personalized recommendation strategy, generate the current recommendation results and push them to the user; S6. Collect user feedback data on the current recommendation results, update the user's short-term behavioral feature vector and user's potential preference state variables, and adjust the state transition function and action selection strategy. S7. Introduce an adaptive exploration and utilization balance mechanism to dynamically adjust the ratio of exploration probability to utilization probability.
[0010] Optionally, S1 specifically includes: S11. Collect historical behavioral data of users on the smart cultural tourism service platform. The historical behavioral data includes browsing records, click records, collection records, comment content, dwell time and tour path data, and timestamp them to construct a time series behavior matrix. S12. Collect current environmental context information, including the user's current location, time information, weather conditions, holiday type, current activity information, and characteristics of the user's terminal device, and convert them into a structured context vector representation. S13. Based on historical behavior data, calculate the user's long-term interest feature vector, and use a weighted sliding window aggregation model to model the user's historical preferences and define the long-term interest feature vector. S14. Based on environmental context information and user behavior data over a recent period, extract short-term user behavior feature vectors, generate short-term interest expression vectors using the recent behavior window mechanism, and embed contextual semantic factors with weights. S15. Perform vector-level fusion of long-term interest feature vectors and short-term behavior feature vectors to construct user feature fusion vectors.
[0011] Optionally, S2 specifically includes: S21. Receive the user feature fusion vector, and use the user's long-term interest feature vector and the user's short-term behavior feature vector as context input variables to construct the candidate recommendation content context representation matrix. S22. Based on the context representation matrix of candidate recommendation content, an improved contextual multi-armed gambling machine model is established. The improved contextual multi-armed gambling machine model takes each candidate recommendation content as an arm of the gambling machine model, and combines the user's long-term interest feature vector and the user's short-term behavior feature vector to form a context variable, which serves as the basis for calculating the reward of each arm. S23. For each arm in the improved contextual multi-armed gambling machine model, estimate the expected reward of the arm using the Bayesian inference method, and calculate the initial reward estimate of the candidate recommendation content. S24. Construct a recommendation strategy probability distribution based on the initial reward estimates of all candidate recommended content, assign a recommendation probability to each candidate recommended content using the Thompson sampling strategy, and output the recommendation strategy probability distribution.
[0012] Optionally, the improved contextual multi-armed gambling machine model specifically includes: Obtain the user's long-term interest feature vector and the user's short-term behavior feature vector, and concatenate the user's long-term interest feature vector and the user's short-term behavior feature vector to form a context input vector, which is used to represent the current user's state information; When constructing the user's long-term interest feature vector, a user interest drift perception mechanism is introduced. A time decay function is constructed based on the user's historical behavior data, and the historical behavior vector is weighted to obtain the user's long-term interest feature vector after time weighting. Input the context input vector into the improved contextual multi-armed gambling machine model. Based on the joint representation of the user's long-term interest feature vector and the user's short-term behavior feature vector, and combined with historical behavior data, estimate the initial reward value of each candidate recommendation content in the current context. Based on the initial reward estimates of all candidate recommendations, select the candidate recommendation with the highest initial reward estimate as the recommendation action to be executed; After executing the selected candidate recommendations, user feedback is received, and the improved contextual multi-armed gambling machine model is updated based on the feedback reward value and the context input vector.
[0013] Optionally, S3 specifically includes: S31. Obtain user historical behavior data, current environmental context information and recommendation strategy probability distribution, and combine them to form a state input data sequence; S32. Based on the state input data sequence, construct a reinforcement learning model that integrates latent variable modeling mechanism. The reinforcement learning model includes five elements: state space, action space, reward function, policy function, and state transition function. S33. Define the state space of user interest evolution and model the potential user preference state variables as a set of continuous latent variables. S34. Initialize the user's potential preference state variables. By performing sequence modeling on the user's historical behavior data, the initial state variables are generated by sampling from the prior distribution using variational inference methods. S35. Construct a state transition function to describe the dynamic changes of user potential preference state variables during multiple rounds of recommendation interaction; S36. Input the initial state variables, the state input data sequence, and the recommendation policy probability distribution into the reinforcement learning model to complete the model initialization. Optionally, S4 specifically includes: S41. Encode the current environmental context information into an environmental state vector, encode the user's historical behavior data into a behavior sequence feature vector, represent the user's potential preference state variables as a continuous hidden state vector, and encode the recommendation strategy probability distribution and the initial reward estimate into a strategy feature vector and a reward feature vector, respectively, and concatenate them to form a joint state input vector. S42. Map the joint state input vector to the state space of the reinforcement learning model to construct the current state representation; S43. In the reinforcement learning model, a policy function is constructed based on the current state representation to calculate the action sampling probability distribution of the candidate recommendation content in the current state; S44. Introduce a latent variable-guided policy reparameterization mechanism, which uses the distribution information of the user's potential preference state variables as a gradient path modulation signal and embeds it into the parameter update process of the policy function. S45. The policy function is iteratively trained using a policy optimization algorithm. The optimal recommended action is sampled based on the action distribution output by the policy function to generate a personalized recommendation policy.
[0014] Optionally, S5 specifically includes: S51. Receive the personalized recommendation strategy output by the reinforcement learning model, and establish a mapping between recommendation actions and content indexes from the candidate recommendation content set; S52. Based on the action sampling probability distribution given in the recommendation strategy, execute the recommendation action and determine the target set of recommended content; S53. Based on the user's current environment context information, perform context-driven content fusion processing on each recommended content in the target recommended content set, and fuse user location information, current time period and scene tags to generate context-enhanced recommended content; S54. Structure the context-enhanced recommendation content to generate a recommendation push content structure that includes title, images and text, tags and trigger source information; S55. Determine the push channel based on the user terminal configuration, and call the recommendation push service module to push the recommended content structure to the user device; S56. Bind and record the pushed recommended content with the execution of the recommended action, and write it into the user behavior log.
[0015] Optionally, S6 specifically includes: S61. Collect user feedback data on the current recommendation results. The feedback data includes click behavior, dwell time, exit behavior, interactive actions and evaluation content. Different types of feedback are uniformly encoded to generate a set of feedback vectors. S62. An adaptive feedback weight adjustment mechanism is introduced, which calculates the updated weight of each type of feedback in the current scenario based on the user's current environment context information and the set of feedback vectors, and forms a weighted feedback input. S63. Update the user's short-term behavior feature vector according to the weighted feedback input, and use a sliding time window mechanism to encode and fuse the latest behavior to generate the updated user short-term behavior feature vector. S64. Perform preference state estimation operation on the updated user short-term behavior feature vector and the current environmental context information to update the user's potential preference state variable; S65. Based on the updated user potential preference state variables, adjust the parameters of the state transition function and action selection policy function in the reinforcement learning model to complete the dynamic update of the policy.
[0016] Optionally, S7 specifically includes: S71. Construct an adaptive exploration and utilization balance function, and set exploration probability variables and utilization probability variables, wherein the exploration probability variables are control factors that change dynamically with time and user feedback; S72. Obtain feedback data from multiple rounds of user recommendation interactions on the smart cultural tourism service platform, generate a sequence of changes in user behavior responses, and calculate feedback response fluctuation indicators. S73. Based on the feedback response fluctuation index and the recommendation interaction time step information, the exploration probability variable is dynamically adjusted using an exponential decay model to generate the exploration probability at the current moment. S74. Based on the current exploration probability and utilization probability, construct a fusion recommendation strategy, combine the user interest-guided recommendation strategy function and the random recommendation strategy function, and generate a hybrid action selection strategy of exploration and utilization. S75. During the recommendation action execution phase, the final recommended content is determined by joint sampling from the candidate recommendation content set based on the constructed exploration and utilization hybrid action selection strategy. S76. Record the user feedback results corresponding to the current exploration probability and the selected recommended content, and periodically evaluate the stability of the exploration and utilization balance function under the current user preference state, and update the exploration probability variable adjustment parameters as appropriate.
[0017] The beneficial effects of this invention are: (1) This invention constructs a user feature fusion mechanism that integrates long-term user interest feature vectors and short-term behavior feature vectors, and introduces an improved contextual multi-armed gambling machine model for initial reward estimation. This effectively improves the content relevance and user fit of the personalized recommendation strategy, and avoids the limitations of traditional recommendation systems in modeling single interests.
[0018] (2) This invention constructs a reinforcement learning model that integrates latent variable modeling mechanism, defines the user interest evolution state space and introduces the latent variable state transition modeling method, realizes the modeling and updating of the dynamic evolution of user interests, and effectively overcomes the defect that existing recommendation methods cannot accurately model the changes of user preferences over time.
[0019] (3) This invention introduces an adaptive exploration and utilization balance mechanism, which dynamically adjusts the exploration probability and utilization probability of the recommendation strategy based on user feedback behavior, thereby achieving a balanced optimization of personalized strategies and avoiding the local optimum problem caused by fixed strategies in traditional reinforcement learning methods.
[0020] (4) This invention constructs a closed-loop recommendation feedback system through multimodal environment context awareness, strategy-guided recommendation execution mechanism and feedback-driven behavior update mechanism, which improves the response capability of personalized recommendation system to environmental changes and user feedback, and achieves accurate recommendation effect under the high dynamic and highly personalized needs of cultural and tourism scenarios. Attached Figure Description
[0021] The accompanying drawings are provided to further illustrate the invention and form part of the specification. They are used in conjunction with embodiments of the invention to explain the invention and do not constitute a limitation thereof. In the drawings: Figure 1 This is a flowchart of a smart cultural tourism personalized service recommendation method based on reinforcement learning proposed in this invention; Figure 2 This is a schematic diagram of the structure of an improved contextual multi-armed gambling machine model proposed in this invention; Figure 3 This is a flowchart illustrating the structure and training process of a reinforcement learning model that incorporates latent variable mechanisms, as proposed in this invention. Detailed Implementation
[0022] The present invention will now be described in further detail with reference to the accompanying drawings. These drawings are simplified schematic diagrams, illustrating only the basic structure of the invention, and therefore only show the components relevant to the invention.
[0023] refer to Figure 1-3 A smart tourism personalized service recommendation method based on reinforcement learning includes the following steps: S1. Obtain the user's historical behavior data and current environmental context information in the smart cultural tourism service platform, extract the user's long-term interest feature vector and short-term behavior feature vector, and construct the user feature fusion vector; S2. Based on the user feature fusion vector, construct an improved contextual multi-armed gambling machine model. Use the user's long-term interest feature vector and the user's short-term behavior feature vector as context input, calculate the initial reward estimate of the candidate recommendation content, and generate the recommendation strategy distribution. S3. Construct a reinforcement learning model that integrates latent variable modeling mechanism, define the state space of user interest evolution, take historical behavior data, current environmental context information and recommendation strategy distribution as input, and initialize user potential preference state variables and state transition function; S4. Input the current environmental context information, historical behavior data, user potential preference state variables, recommendation strategy distribution and initial reward estimate into the reinforcement learning model, learn the state transition function and action selection strategy through the policy optimization algorithm, and output a personalized recommendation strategy. S5. Perform recommendation actions on candidate content according to the personalized recommendation strategy, generate the current recommendation results and push them to the user; S6. Collect user feedback data on the current recommendation results, update the user's short-term behavioral feature vector and user's potential preference state variables, and adjust the state transition function and action selection strategy. S7. Introduce an adaptive exploration and utilization balance mechanism to dynamically adjust the ratio of exploration probability to utilization probability.
[0024] In this embodiment, S1 specifically includes: S11. Collect historical behavioral data of users on the smart cultural tourism service platform. The historical behavioral data includes browsing records, click records, collection records, comment content, dwell time and tour path data, and timestamp them to construct a time series behavior matrix. S12. Collect current environmental context information, including the user's current location, time information, weather conditions, holiday type, current activity information, and characteristics of the user's terminal device, and convert them into a structured context vector representation. S13. Based on historical behavior data, calculate the user's long-term interest feature vector, and use a weighted sliding window aggregation model to model the user's historical preferences and define the long-term interest feature vector. S14. Based on environmental context information and user behavior data over a recent period, extract short-term user behavior feature vectors, generate short-term interest expression vectors using the recent behavior window mechanism, and embed contextual semantic factors with weights. S15. Perform vector-level fusion of long-term interest feature vectors and short-term behavior feature vectors to construct user feature fusion vectors.
[0025] This implementation method collects multi-dimensional historical behavioral data and current environmental context information of users on a smart cultural tourism service platform, constructs a time-series behavioral matrix and a structured context vector, extracts long-term interest feature vectors and short-term behavioral feature vectors of users respectively, and uses a weighted sliding window and contextual semantic embedding mechanism to model historical preferences and current behavioral expressions respectively, and fuses them at the vector level to form a comprehensive user feature vector that reflects the stability of user behavior and real-time interest changes. This provides a more accurate, dynamic and semantically related user profile for the subsequent generation of personalized recommendation strategies, significantly improving the recommendation system's ability to capture user interests and matching accuracy.
[0026] In this embodiment, S2 specifically includes: S21. Receive the user feature fusion vector, and use the user's long-term interest feature vector and the user's short-term behavior feature vector as context input variables to construct the candidate recommendation content context representation matrix. S22. Based on the context representation matrix of candidate recommendation content, an improved contextual multi-armed gambling machine model is established. The improved contextual multi-armed gambling machine model takes each candidate recommendation content as an arm of the gambling machine model, and combines the user's long-term interest feature vector and the user's short-term behavior feature vector to form a context variable, which serves as the basis for calculating the reward of each arm. S23. For each arm in the improved contextual multi-armed gambling machine model, estimate the expected reward of the arm using the Bayesian inference method, and calculate the initial reward estimate of the candidate recommendation content. S24. Construct a recommendation strategy probability distribution based on the initial reward estimates of all candidate recommended content, assign a recommendation probability to each candidate recommended content using the Thompson sampling strategy, and output the recommendation strategy probability distribution.
[0027] This implementation constructs a contextual representation matrix for candidate recommendation content by inputting both the user's long-term interest feature vector and short-term behavior feature vector. Based on this, an improved contextual multi-armed gambling machine model is established, mapping each candidate recommendation content to one arm of the model. Bayesian inference is used to estimate the expected reward of each arm, and a recommendation strategy probability distribution is constructed using a Thompson sampling strategy to dynamically allocate recommendation probabilities. This scheme balances the stability of user interests with the sensitivity of immediate behavior, combining probabilistic modeling and strategy sampling methods to effectively balance exploration and exploitation when generating recommendation strategies. This helps improve the relevance of recommended content and user click-through rates, enhancing the personalized recommendation effect in smart tourism scenarios.
[0028] In this embodiment, the improved contextual multi-armed gambling machine model specifically includes: Obtain the user's long-term interest feature vector and the user's short-term behavior feature vector, and concatenate the user's long-term interest feature vector and the user's short-term behavior feature vector to form a context input vector, which is used to represent the current user's state information; When constructing the user's long-term interest feature vector, a user interest drift perception mechanism is introduced. A time decay function is constructed based on the user's historical behavior data, and the historical behavior vector is weighted to obtain the user's long-term interest feature vector after time weighting. Input the context input vector into the improved contextual multi-armed gambling machine model. Based on the joint representation of the user's long-term interest feature vector and the user's short-term behavior feature vector, and combined with historical behavior data, estimate the initial reward value of each candidate recommendation content in the current context. Based on the initial reward estimates of all candidate recommendations, select the candidate recommendation with the highest initial reward estimate as the recommendation action to be executed; After executing the selected candidate recommendations, user feedback is received, and the improved contextual multi-armed gambling machine model is updated based on the feedback reward value and the context input vector.
[0029] This implementation introduces a user interest drift perception mechanism, incorporating a time decay function when constructing the user's long-term interest feature vector. This weighted modeling of historical behavior data enhances the model's ability to depict dynamic changes in user interests. Simultaneously, the user's long-term interest feature vector is concatenated with the user's short-term behavior feature vector to form a contextual input vector, which is then fed into the improved contextual multi-armed gambling machine model. This model uses historical behavior data to estimate the initial reward for candidate recommended content and selects the optimal recommendation action accordingly. After executing the recommendation action, the model receives user feedback and updates its parameters, achieving adaptive optimization of the recommendation strategy and effectively improving the relevance and timeliness of the recommendation results.
[0030] In this embodiment, S3 specifically includes: S31. Obtain user historical behavior data, current environmental context information and recommendation strategy probability distribution, and combine them to form a state input data sequence; S32. Based on the state input data sequence, construct a reinforcement learning model that integrates latent variable modeling mechanism. The reinforcement learning model includes five elements: state space, action space, reward function, policy function, and state transition function. S33. Define the state space of user interest evolution and model the potential user preference state variables as a set of continuous latent variables. S34. Initialize the user's potential preference state variables. By performing sequence modeling on the user's historical behavior data, the initial state variables are generated by sampling from the prior distribution using variational inference methods. S35. Construct a state transition function to describe the dynamic changes of user potential preference state variables during multiple rounds of recommendation interaction; S36. Input the initial state variables, the state input data sequence, and the recommendation policy probability distribution into the reinforcement learning model to complete the model initialization.
[0031] This implementation combines historical user behavior data, current environmental context information, and recommendation strategy probability distribution to form a state input data sequence. It then initializes the user's latent preference state variables using variational inference methods, constructing a reinforcement learning model that integrates latent variable modeling mechanisms. This enables dynamic modeling of the user's interest evolution process. The reinforcement learning model comprises five elements: state space, action space, reward function, policy function, and state transition function. It captures the implicit trends in user preferences and continuously updates the user's latent preference state through the state transition function. This allows for a more accurate response to changes in user interests during recommendation strategy generation, thereby enhancing the personalization and dynamic adaptability of the recommendation results.
[0032] In this embodiment, S4 specifically includes: S41. Encode the current environmental context information into an environmental state vector, encode the user's historical behavior data into a behavior sequence feature vector, represent the user's potential preference state variables as a continuous hidden state vector, and encode the recommendation strategy probability distribution and the initial reward estimate into a strategy feature vector and a reward feature vector, respectively, and concatenate them to form a joint state input vector. S42. Map the joint state input vector to the state space of the reinforcement learning model to construct the current state representation; S43. In the reinforcement learning model, a policy function is constructed based on the current state representation to calculate the action sampling probability distribution of the candidate recommendation content in the current state; S44. Introduce a latent variable-guided policy reparameterization mechanism, which uses the distribution information of the user's potential preference state variables as a gradient path modulation signal and embeds it into the parameter update process of the policy function. S45. The policy function is iteratively trained using a policy optimization algorithm. The optimal recommended action is sampled based on the action distribution output by the policy function to generate a personalized recommendation policy.
[0033] In this embodiment, S5 specifically includes: S51. Receive the personalized recommendation strategy output by the reinforcement learning model, and establish a mapping between recommendation actions and content indexes from the candidate recommendation content set; S52. Based on the action sampling probability distribution given in the recommendation strategy, execute the recommendation action and determine the target set of recommended content; S53. Based on the user's current environment context information, perform context-driven content fusion processing on each recommended content in the target recommended content set, and fuse user location information, current time period and scene tags to generate context-enhanced recommended content; S54. Structure the context-enhanced recommendation content to generate a recommendation push content structure that includes title, images and text, tags and trigger source information; S55. Determine the push channel based on the user terminal configuration, and call the recommendation push service module to push the recommended content structure to the user device; S56. Bind and record the pushed recommended content with the execution of the recommended action, and write it into the user behavior log.
[0034] This implementation encodes the current environmental context information, user historical behavior data, user potential preference state variables, recommendation strategy probability distribution, and initial reward estimate into multi-dimensional vectors, concatenates them to form a joint state input vector, maps it to the state space of the reinforcement learning model, constructs the current state representation, and further introduces a latent variable-guided policy reparameterization mechanism to embed the distribution characteristics of user preference state variables during the policy function update process, thereby achieving effective control and optimization of the policy function. Combined with policy optimization algorithms for training and action sampling, a personalized recommendation strategy that dynamically adapts to changes in user interests is finally generated, improving recommendation accuracy and user response rate, and realizing intelligent policy modeling and refined recommendation control in cultural tourism recommendation services.
[0035] In this embodiment, S6 specifically includes: S61. Collect user feedback data on the current recommendation results. The feedback data includes click behavior, dwell time, exit behavior, interactive actions and evaluation content. Different types of feedback are uniformly encoded to generate a set of feedback vectors. S62. An adaptive feedback weight adjustment mechanism is introduced, which calculates the updated weight of each type of feedback in the current scenario based on the user's current environment context information and the set of feedback vectors, and forms a weighted feedback input. S63. Update the user's short-term behavior feature vector according to the weighted feedback input, and use a sliding time window mechanism to encode and fuse the latest behavior to generate the updated user short-term behavior feature vector. S64. Perform preference state estimation operation on the updated user short-term behavior feature vector and the current environmental context information to update the user's potential preference state variable; S65. Based on the updated user potential preference state variables, adjust the parameters of the state transition function and action selection policy function in the reinforcement learning model to complete the dynamic update of the policy.
[0036] This implementation collects multidimensional feedback data from users regarding the current recommendation results and introduces an adaptive feedback weight adjustment mechanism. It dynamically adjusts the influence of feedback types based on the user's current environmental context, thereby generating weighted feedback inputs and updating the user's short-term behavioral feature vector in real time. Combining the updated short-term behavioral feature vector with the current environmental context information, it performs preference state estimation, dynamically corrects the user's potential preference state variables, and adjusts the parameters of the state transition function and action selection strategy function in the reinforcement learning model based on this. This achieves continuous adaptive optimization of the recommendation strategy under different contextual environments. This approach effectively improves the system's response speed to changes in user interests and the accuracy of the recommendation strategy, realizing highly sensitive behavioral feedback modeling and strategy optimization updates in the personalized recommendation process.
[0037] In this embodiment, S7 specifically includes: S71. Construct an adaptive exploration and utilization balance function, and set exploration probability variables and utilization probability variables, wherein the exploration probability variables are control factors that change dynamically with time and user feedback; S72. Obtain feedback data from multiple rounds of user recommendation interactions on the smart cultural tourism service platform, generate a sequence of changes in user behavior responses, and calculate feedback response fluctuation indicators. S73. Based on the feedback response fluctuation index and the recommendation interaction time step information, the exploration probability variable is dynamically adjusted using an exponential decay model to generate the exploration probability at the current moment. S74. Based on the current exploration probability and utilization probability, construct a fusion recommendation strategy, combine the user interest-guided recommendation strategy function and the random recommendation strategy function, and generate a hybrid action selection strategy of exploration and utilization. S75. During the recommendation action execution phase, the final recommended content is determined by joint sampling from the candidate recommendation content set based on the constructed exploration and utilization hybrid action selection strategy. S76. Record the user feedback results corresponding to the current exploration probability and the selected recommended content, and periodically evaluate the stability of the exploration and utilization balance function under the current user preference state, and update the exploration probability variable adjustment parameters as appropriate.
[0038] This implementation constructs an adaptive exploration-utilization balance function, dynamically sets exploration and utilization probability variables, and combines this with the feedback response sequence of users in multiple rounds of recommendation interactions on the smart cultural tourism service platform. An exponential decay model is used to adjust the exploration probability variable in real time, achieving flexible control of the exploration-utilization ratio under different user states and feedback conditions. During the recommendation action execution phase, a user interest guidance strategy and a random recommendation strategy are integrated to generate a hybrid action selection strategy for joint sampling of candidate recommendation content, thereby improving the diversity and matching degree of the recommended content. By recording the feedback corresponding to exploration behavior and periodically evaluating the stability of the strategy, this implementation effectively solves the problem of recommendation stagnation or user cold start caused by a fixed exploration rate in traditional strategies, significantly enhancing the adaptability and dynamic optimization capability of the recommendation system.
[0039] Example 1: To verify the feasibility of this invention in practice, it was applied to a smart cultural tourism service platform of a national 5A-level scenic area. This scenic area receives over 20,000 visitors daily and offers services across multiple dimensions, including attraction guides, restaurant recommendations, route planning, and intangible cultural heritage experiences. Visitor types are diverse, and their needs are highly personalized. Traditional recommendation systems based on collaborative filtering or static rules struggle to accurately capture and respond in real-time to dynamic changes in visitor interests, resulting in low relevance of service recommendations and low user satisfaction.
[0040] In this smart cultural tourism platform, the method of this invention is integrated as a core recommendation module. The system first collects long-term interest characteristics (such as historical browsing and favorites data) and short-term behavioral data (such as current clicks, dwell areas, and search keywords) of tourists through the user registration and behavior tracking module, and generates a user feature fusion vector through vector modeling. The platform further utilizes an improved contextual multi-armed gambling machine model to calculate initial reward estimates for all candidate recommendation content, and combines current environmental context information (such as weather, time of day, and geographical location) with platform content pool characteristics to establish a probability distribution for the fusion recommendation strategy, forming a basic recommendation action sequence.
[0041] Subsequently, the platform invokes a reinforcement learning model that integrates latent variable modeling mechanisms. It introduces state space modeling to represent users' potential preference state variables and dynamically learns the user behavior evolution path through policy functions and state transition functions to generate personalized recommendation strategies. During the recommendation execution phase, the system integrates content context, location information, and scene tags to achieve structured content packaging and push it through the visitor's terminal. Visitor feedback (clicks, ratings, dwell time) is collected in real time to dynamically adjust the user's short-term feature vector and state transition function, and periodically updates the balance parameters between exploration and utilization.
[0042] To verify the effectiveness of this method in real-world scenarios, the platform conducted a two-month dual-group controlled experiment, selecting 10,000 tourists who were randomly divided into an experimental group (using the method of this invention) and a control group (using a collaborative filtering algorithm) for personalized service recommendations. Experimental metrics included click-through rate, average dwell time, service conversion rate, and tourist satisfaction ratings. The experimental results are shown in the table below: Table 1. Comparison of the performance of different recommendation methods on the smart cultural tourism platform.
[0043] The data shows that after applying the method of this invention, the system has significantly improved in terms of recommendation click-through rate, content relevance, and service conversion rate. In particular, by introducing an explore-utilize balance mechanism, the system can dynamically adjust the recommendation strategy according to changes in user behavior, taking into account both content diversity and interest accuracy, effectively improving user stickiness and platform activity.
[0044] For example, when a tourist searches for "nighttime experiences" on the platform, the system uses short-term behavior modeling and context recognition to recommend two highly popular activities in real time: "West Lake Night Light Show" and "Grand Canal Intangible Cultural Heritage Iron Flower Show." Within an hour of the tourist clicking on these activities, the system also triggers route planning and ticket booking services. The platform records this behavioral chain and feeds it back to the model, automatically increasing the ranking priority of such nighttime cultural activities in the next round of recommendations.
[0045] In summary, the smart cultural tourism personalized service recommendation method proposed in this invention, through a deep integration of reinforcement learning and user potential preference modeling, possesses high dynamic adaptability and predictive ability, significantly improving the personalization and effectiveness of recommendations, and providing practical technical support and service optimization paths for smart cultural tourism platforms.
[0046] The above description is only a preferred embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any equivalent substitutions or modifications made by those skilled in the art within the scope of the technology disclosed in the present invention, based on the technical solution and inventive concept of the present invention, should be covered within the scope of protection of the present invention.
Claims
1. A method for recommending personalized smart cultural tourism services based on reinforcement learning, characterized in that, Includes the following steps: S1. Obtain the user's historical behavior data and current environmental context information in the smart cultural tourism service platform, extract the user's long-term interest feature vector and short-term behavior feature vector, and construct the user feature fusion vector; S2. Based on the user feature fusion vector, construct an improved contextual multi-armed gambling machine model. Use the user's long-term interest feature vector and the user's short-term behavior feature vector as context input, calculate the initial reward estimate of the candidate recommendation content, and generate the recommendation strategy distribution. S3. Construct a reinforcement learning model that integrates latent variable modeling mechanism, define the state space of user interest evolution, take historical behavior data, current environmental context information and recommendation strategy distribution as input, and initialize user potential preference state variables and state transition function; S4. Input the current environmental context information, historical behavior data, user potential preference state variables, recommendation strategy distribution and initial reward estimate into the reinforcement learning model, learn the state transition function and action selection strategy through the policy optimization algorithm, and output a personalized recommendation strategy. S5. Perform recommendation actions on candidate content according to the personalized recommendation strategy, generate the current recommendation results and push them to the user; S6. Collect user feedback data on the current recommendation results, update the user's short-term behavioral feature vector and user's potential preference state variables, and adjust the state transition function and action selection strategy. S7. Introduce an adaptive exploration and utilization balance mechanism to dynamically adjust the ratio of exploration probability to utilization probability.
2. The method for recommending personalized smart cultural tourism services based on reinforcement learning according to claim 1, characterized in that, S1 specifically includes: S11. Collect users' historical behavior data on the smart cultural tourism service platform. The historical behavior data includes browsing history, click history, collection history, comment content, dwell time and tour path data. S12. Collect current environmental context information, including the user's current location, time information, weather conditions, holiday type, current activity information, and characteristics of the user's terminal device; S13. Based on historical behavior data, calculate the user's long-term interest feature vector, and use a weighted sliding window aggregation model to model the user's historical preferences and define the long-term interest feature vector. S14. Based on environmental context information and user behavior data over a recent period, extract short-term user behavior feature vectors, generate short-term interest expression vectors using the recent behavior window mechanism, and embed contextual semantic factors with weights. S15. Perform vector-level fusion of long-term interest feature vectors and short-term behavior feature vectors to construct user feature fusion vectors.
3. The method for recommending personalized smart cultural tourism services based on reinforcement learning according to claim 1, characterized in that, S2 specifically includes: S21. Receive the user feature fusion vector, and use the user's long-term interest feature vector and the user's short-term behavior feature vector as context input variables to construct the candidate recommendation content context representation matrix. S22. Based on the context representation matrix of candidate recommendation content, an improved contextual multi-armed gambling machine model is established. The improved contextual multi-armed gambling machine model takes each candidate recommendation content as an arm of the gambling machine model, and combines the user's long-term interest feature vector and the user's short-term behavior feature vector to form a context variable, which serves as the basis for calculating the reward of each arm. S23. For each arm in the improved contextual multi-armed gambling machine model, estimate the expected reward of the arm using the Bayesian inference method, and calculate the initial reward estimate of the candidate recommendation content. S24. Construct a recommendation strategy probability distribution based on the initial reward estimates of all candidate recommended content, assign a recommendation probability to each candidate recommended content using the Thompson sampling strategy, and output the recommendation strategy probability distribution.
4. The method for recommending personalized smart cultural tourism services based on reinforcement learning according to claim 3, characterized in that, The improved contextual multi-armed gambling machine model specifically includes: Obtain the user's long-term interest feature vector and the user's short-term behavior feature vector, and concatenate the user's long-term interest feature vector and the user's short-term behavior feature vector to form a context input vector, which is used to represent the current user's state information; When constructing the user's long-term interest feature vector, a user interest drift perception mechanism is introduced. A time decay function is constructed based on the user's historical behavior data, and the historical behavior vector is weighted to obtain the user's long-term interest feature vector after time weighting. Input the context input vector into the improved contextual multi-armed gambling machine model. Based on the joint representation of the user's long-term interest feature vector and the user's short-term behavior feature vector, and combined with historical behavior data, estimate the initial reward value of each candidate recommendation content in the current context. Based on the initial reward estimates of all candidate recommendations, select the candidate recommendation with the highest initial reward estimate as the recommendation action to be executed; After executing the selected candidate recommendations, user feedback is received, and the improved contextual multi-armed gambling machine model is updated based on the feedback reward value and the context input vector.
5. The method for recommending personalized smart cultural tourism services based on reinforcement learning according to claim 1, characterized in that, S3 specifically includes: S31. Obtain user historical behavior data, current environmental context information and recommendation strategy probability distribution, and combine them to form a state input data sequence; S32. Based on the state input data sequence, construct a reinforcement learning model that integrates latent variable modeling mechanism. The reinforcement learning model includes five elements: state space, action space, reward function, policy function, and state transition function. S33. Define the state space of user interest evolution and model the potential user preference state variables as a set of continuous latent variables. S34. Initialize the user's potential preference state variables. By performing sequence modeling on the user's historical behavior data, the initial state variables are generated by sampling from the prior distribution using variational inference methods. S35. Construct a state transition function to describe the dynamic changes of user potential preference state variables during multiple rounds of recommendation interaction; S36. Input the initial state variables, the state input data sequence, and the recommendation policy probability distribution into the reinforcement learning model to complete the model initialization.
6. The method for recommending personalized smart cultural tourism services based on reinforcement learning according to claim 1, characterized in that, S4 specifically includes: S41. Encode the current environmental context information into an environmental state vector, encode the user's historical behavior data into a behavior sequence feature vector, represent the user's potential preference state variables as a continuous hidden state vector, and encode the recommendation strategy probability distribution and the initial reward estimate into a strategy feature vector and a reward feature vector, respectively, and concatenate them to form a joint state input vector. S42. Map the joint state input vector to the state space of the reinforcement learning model to construct the current state representation; S43. In the reinforcement learning model, a policy function is constructed based on the current state representation to calculate the action sampling probability distribution of the candidate recommendation content in the current state; S44. Introduce a latent variable-guided policy reparameterization mechanism, which uses the distribution information of the user's potential preference state variables as a gradient path modulation signal and embeds it into the parameter update process of the policy function. S45. The policy function is iteratively trained using a policy optimization algorithm. The optimal recommended action is sampled based on the action distribution output by the policy function to generate a personalized recommendation policy.
7. The method for recommending personalized smart cultural tourism services based on reinforcement learning according to claim 1, characterized in that, S5 specifically includes: S51. Receive the personalized recommendation strategy output by the reinforcement learning model, and establish a mapping between recommendation actions and content indexes from the candidate recommendation content set; S52. Based on the action sampling probability distribution given in the recommendation strategy, execute the recommendation action and determine the target set of recommended content; S53. Based on the user's current environment context information, perform context-driven content fusion processing on each recommended content in the target recommended content set, and fuse user location information, current time period and scene tags to generate context-enhanced recommended content; S54. Structure the context-enhanced recommendation content to generate a recommendation push content structure that includes title, images and text, tags and trigger source information; S55. Determine the push channel based on the user terminal configuration, and call the recommendation push service module to push the recommended content structure to the user device; S56. Bind and record the pushed recommended content with the execution of the recommended action, and write it into the user behavior log.
8. The method for recommending personalized smart cultural tourism services based on reinforcement learning according to claim 1, characterized in that, S6 specifically includes: S61. Collect user feedback data on the current recommendation results. The feedback data includes click behavior, dwell time, exit behavior, interactive actions and evaluation content. Different types of feedback are uniformly encoded to generate a set of feedback vectors. S62. An adaptive feedback weight adjustment mechanism is introduced, which calculates the updated weight of each type of feedback in the current scenario based on the user's current environment context information and the set of feedback vectors, and forms a weighted feedback input. S63. Update the user's short-term behavior feature vector according to the weighted feedback input, and use a sliding time window mechanism to encode and fuse the latest behavior to generate the updated user short-term behavior feature vector. S64. Perform preference state estimation operation on the updated user short-term behavior feature vector and the current environmental context information to update the user's potential preference state variable; S65. Based on the updated user potential preference state variables, adjust the parameters of the state transition function and action selection policy function in the reinforcement learning model to complete the dynamic update of the policy.
9. The method for recommending personalized smart cultural tourism services based on reinforcement learning according to claim 1, characterized in that, Specifically, S7 includes: S71. Construct an adaptive exploration and utilization balance function, and set exploration probability variables and utilization probability variables, wherein the exploration probability variables are control factors that change dynamically with time and user feedback; S72. Obtain feedback data from multiple rounds of user recommendation interactions on the smart cultural tourism service platform, generate a sequence of changes in user behavior responses, and calculate feedback response fluctuation indicators. S73. Based on the feedback response fluctuation index and the recommendation interaction time step information, the exploration probability variable is dynamically adjusted using an exponential decay model to generate the exploration probability at the current moment. S74. Based on the current exploration probability and utilization probability, construct a fusion recommendation strategy, combine the user interest-guided recommendation strategy function and the random recommendation strategy function, and generate a hybrid action selection strategy of exploration and utilization. S75. During the recommendation action execution phase, the final recommendation content is determined by joint sampling from the candidate recommendation content set based on the constructed exploration and utilization hybrid action selection strategy. S76. Record the user feedback results corresponding to the current exploration probability and the selected recommended content, and periodically evaluate the stability of the exploration and utilization balance function under the current user preference state, and update the exploration probability variable adjustment parameters as appropriate.