A digital tourism personalized service recommendation method and system based on reinforcement learning

Abstract

The application discloses a digital cultural tourism personalized service recommendation method and system based on reinforcement learning, relates to the field of digital cultural tourism personalized service recommendation, and first collects four kinds of multi-source data, namely, user static attributes, historical cultural tourism interaction, real-time context and cultural tourism resource attributes, wherein the user static attributes include consumption levels and cultural tourism preference tags, the historical cultural tourism interaction data include browsing, ordering and scoring records, the real-time context data include user positioning, current weather and travel modes, and the cultural tourism resource attributes include categories, prices, real-time passenger flow and opening rules; all the collected data are sequentially subjected to outlier elimination, missing value completion, feature coding and numerical normalization pretreatment, and are spliced to generate feature vectors of unified dimensions. The application can dynamically adjust the weights of various optimization targets according to different scenes such as tourism off-season and temporary cultural tourism activities, and takes into account the core needs of user experience, scenic spot passenger flow control and merchant revenue growth.

Description

Technical Field

[0001] This invention relates to the field of personalized digital cultural tourism service recommendation technology, and in particular to a method and system for personalized digital cultural tourism service recommendation based on reinforcement learning. Background Technology

[0002] Digital tourism is the core direction of the digital transformation of the tourism industry. Currently, various tourism service platforms are widely available in China, providing users with a full range of services, including scenic spot reservations, restaurant and accommodation bookings, and tourism activity inquiries. As users' personalized demands for tourism services continue to increase, tourism operators need to simultaneously address three core needs: ensuring user experience, managing visitor flow effectively, and increasing merchant revenue. This places higher demands on the multi-objective adaptation and dynamic adjustment capabilities of recommendation systems.

[0003] One of the mainstream recommendation technologies in the current cultural tourism sector is collaborative filtering. This type of method collects users' historical cultural tourism interaction data, calculates the similarity between different users and different cultural tourism resources, and recommends cultural tourism resources that are of high interest to users with similar preferences. The solution is relatively easy to implement and can quickly obtain recommendation results that match users' historical preferences when the user sample size is small. Currently, many small and medium-sized cultural tourism platforms use this type of solution. However, this type of solution can only match users' historical preference features and cannot adjust the recommendation strategy by combining dynamic context information such as users' current location, real-time weather, and real-time visitor flow at scenic spots. Furthermore, it does not include scenic spot load balancing and merchant revenue needs in the optimization objectives. During peak tourist seasons, a large number of users are easily recommended to popular scenic spots, leading to scenic spot overload, a decline in user experience, and failing to drive traffic and increase revenue for small and medium-sized merchants.

[0004] Another mainstream approach is a rule-based targeted recommendation method. Operators pre-set recommendation rules for different user groups and time periods. For example, they might recommend family-oriented cultural and tourism resources to families with children, while lowering the priority of popular scenic spots during peak tourist seasons. This approach offers strong operational control and can quickly respond to short-term operational needs. However, this type of solution has limited rule coverage, cannot adapt to the diverse and personalized needs of a massive user base, suffers from severe homogenization in recommendations for similar user groups, and requires manual maintenance for rule updates, resulting in high operational costs. It also cannot dynamically adjust the weights of different optimization objectives to adapt to the operational needs of different scenarios such as peak seasons, off-seasons, and promotional periods. Furthermore, it lacks a pre-emptive compliance verification mechanism, making it prone to recommending resources that do not meet regulatory requirements or exceed users' spending power.

[0005] With the increasing demand for refined operation in the cultural and tourism industry, existing recommendation solutions are no longer able to meet the needs of users, scenic spots, and merchants, and lack the ability to dynamically adapt to multi-scenario operations. There is an urgent need for new personalized recommendation solutions to address these pain points. Summary of the Invention

[0006] This invention proposes a method and system for recommending personalized digital cultural tourism services based on reinforcement learning, in order to solve the problems mentioned in the prior art, such as existing cultural tourism recommendation technologies only optimizing a single objective, poor adaptability, insufficient accuracy, and inability to take into account the needs of users, scenic spots, and merchants.

[0007] To achieve the above objectives, the present invention adopts the following technical solution: a personalized digital cultural tourism service recommendation method based on reinforcement learning, comprising the following steps: The system collects static attribute data, historical cultural and tourism interaction data, real-time context data in the current scenario, and full attribute data from the cultural and tourism resources side from the user center, front-end behavior tracking module, user terminal positioning sensor, cultural and tourism resource management backend, and third-party public service interface of the cultural and tourism service platform. After cleaning, filling in missing values, and removing outliers from all the collected raw data, one-hot encoding is used to encode discrete features, and the maximum-minimum normalization method is used to scale continuous features. The processed features are then concatenated to generate fixed-dimensional user feature vectors, real-time context feature vectors, and cultural tourism resource feature vectors. Construct a multi-objective reward function. The optimization objectives of the multi-objective reward function include at least three categories: user satisfaction, scenic area load balance, and revenue contribution of cultural and tourism merchants. Each of the three objectives corresponds to an independent reward calculation logic. The reward values ​​of all objectives are weighted and summed to obtain the final total reward value. Using user feature vectors, real-time context feature vectors, and cultural and tourism resource feature vectors as input layer parameters of the reinforcement learning network, and taking the maximization of the total reward value of the multi-objective reward function as the optimization guide, a deep deterministic policy gradient network containing an Actor policy network and a Critic evaluation network is adopted as the reinforcement learning recommendation policy network. The recommendation ranking weights of each cultural and tourism resource are output through the Actor network, and the candidate recommendation set of cultural and tourism services matching the current user needs is obtained by filtering from high to low weights. After the candidate recommendation set is delivered to the user, user interaction data on the recommendation results, real-time visitor flow data of each scenic spot, and real-time revenue data of each cultural and tourism merchant are collected synchronously at a fixed feedback collection cycle. The above-mentioned collected feedback data is used as a reward signal and input into the deep deterministic policy gradient network to synchronously update the weight coefficients of the multi-objective reward function and the fully connected layer parameters of the reinforcement learning recommendation policy network.

[0008] Furthermore, the user static attribute data includes the user's age and place of residence filled in during registration, the level of spending power obtained through the real-name authentication interface, and the cultural and tourism preference tags actively set by the user; the historical cultural and tourism interaction data includes the user's historical cultural and tourism content browsing records, order transaction records, rating and evaluation records, stay time in scenic spots records, order cancellation records, and search keyword records collected by the platform. The real-time context data includes the current location collected by the user terminal positioning module, the current system time, the current weather obtained from the third-party meteorological interface, the attributes of the companions actively filled in by the user, and the current mode of travel; the cultural and tourism resource attribute data includes category tags, geographical location tags, price range tags, real-time visitor flow tags, user average rating tags, opening time period tags, suitable population tags, and current event information tags for scenic spots, catering, accommodation, cultural and creative products, and performance resources.

[0009] Furthermore, after generating the feature vectors of cultural and tourism resources, all cultural and tourism resources are input into the compliance verification module for pre-compliance verification. Specifically, this involves connecting to the real-time policy interface of the local cultural and tourism regulatory department to filter out cultural and tourism resources that do not comply with the current regional epidemic prevention and control requirements; connecting to the real-time ticketing management system of each scenic spot to filter out cultural and tourism resources with zero remaining reservation quotas or exceeding the maximum carrying capacity of the scenic spot; and calling the user consumption profile data from the user center to filter out cultural and tourism resources that exceed the user's preset consumption capacity threshold. After all the verification conditions are met, the cultural and tourism resources that meet the requirements are composed of a set of compliant resources to be recommended, and then input into the reinforcement learning recommendation strategy network for recommendation and ranking.

[0010] Furthermore, the three optimization objectives of the multi-objective reward function are each assigned dynamically adjustable weights. These dynamically adjustable weights are automatically adjusted according to the current operational strategy of the cultural tourism operator. When the system determines that the current period is the peak tourist season, it automatically increases the weight corresponding to the scenic area load balancing, with the weight allocation being 40% for user satisfaction, 35% for scenic area load balancing, and 25% for cultural tourism merchant revenue contribution. When the system determines that the current period is the off-season, it automatically increases the weight corresponding to the cultural tourism merchant revenue contribution, with the weight allocation being 40% for user satisfaction, 15% for scenic area load balancing, and 45% for cultural tourism merchant revenue contribution. During off-seasons, the weights of the three objectives are equally distributed. The system also supports operators manually adjusting the weights to adapt to temporarily held cultural tourism festivals, music festivals, and food festivals.

[0011] Furthermore, before the reinforcement learning recommendation strategy network is put into operation, it first calls the historically stored full-domain cultural tourism interaction dataset for offline pre-training. The full-domain cultural tourism interaction dataset contains at least the user interaction data, scenic spot visitor flow data, and merchant revenue data of the past 12 months. The pre-training process adopts an experience playback mechanism. Each time, 128 sets of historical samples are randomly sampled from the experience pool to calculate the reward value and backpropagate to update the network parameters. The pre-training termination condition is that the recommendation accuracy of the validation set increases by less than 0.1% for 10 consecutive rounds. At this time, the pre-training is stopped, the initial network parameters are output, and then the network is put into online operation.

[0012] Furthermore, when collecting feedback data after the candidate recommendation set is deployed, priority is given to collecting user interaction data such as clicks, orders, and ratings of the recommendation results. Different reward weights are set according to the confidence level of the interaction behavior. Among them, positive ratings of 4 stars or above correspond to the highest reward weight, orders are completed, and clicks and views correspond to the lowest reward weight. Users who immediately return after clicking, give negative reviews of 1 star or below, or request order cancellations are given negative rewards. The above weighted reward values ​​are used as real-time rewards and input into the reinforcement learning recommendation strategy network. Every 1000 valid interaction data points trigger a parameter update, realizing online incremental updates of network parameters.

[0013] Furthermore, a personalized digital cultural tourism service recommendation system based on reinforcement learning includes: The data acquisition module is deployed on a cloud server cluster and connects to user-side apps / mini-programs, cultural and tourism resource management systems, and third-party public service interfaces. It is used to collect static attribute data, historical cultural and tourism interaction data, real-time context data in the current scenario, and full attribute data from the cultural and tourism resource side. The feature engineering module, equipped with a standardized feature processing engine, is used to clean, encode, and normalize various types of raw data collected, and then generate user feature vectors, real-time context feature vectors, and cultural and tourism resource feature vectors with fixed dimensions. The multi-objective reward configuration module provides a visual configuration interface for constructing multi-objective reward functions. The optimization objectives of the multi-objective reward function include at least three categories: user satisfaction, scenic area load balancing, and revenue contribution of cultural and tourism merchants. The module supports operators in adjusting the weight coefficients of the three categories of objectives. The reinforcement learning recommendation module is equipped with a GPU-accelerated computing unit and has a built-in pre-trained deep deterministic policy gradient network. It takes user feature vectors, real-time context feature vectors, and cultural and tourism resource feature vectors as inputs, uses a multi-objective reward function as the optimization guide, and outputs a set of candidate recommendations for cultural and tourism services that match the current needs. The parameter iteration module sets up an asynchronous update queue to collect user interaction data, scenic spot visitor flow data, and merchant revenue data after the candidate recommendation set is deployed. The above data is used as feedback input to update the weights of the multi-objective reward function and the parameters of the reinforcement learning recommendation strategy network. The asynchronous queue avoids the parameter update from affecting the normal operation of the online recommendation business.

[0014] Furthermore, it also includes a compliance verification module. The input of the compliance verification module is connected to the output of the feature engineering module, and the output is connected to the input of the reinforcement learning recommendation module. The compliance verification module also interfaces with the policy interface of the local cultural and tourism regulatory department, the ticket management system interface of each scenic spot, and the consumer profile interface of the user center. It verifies the policy compliance, remaining carrying capacity, and price suitability of cultural and tourism resources, filters out cultural and tourism resources that do not meet all verification conditions, and generates a set of compliant resources to be recommended, which is then input into the reinforcement learning recommendation module to reduce the amount of invalid computation in the recommendation network.

[0015] Furthermore, it also includes a pre-training module, which is deployed on an independent offline computing cluster and physically isolated from the computing resources of the online recommendation business. The output end is connected to the reinforcement learning recommendation module, which is used to call the historically stored full-domain cultural tourism interaction dataset to perform offline pre-training of the policy network of the reinforcement learning recommendation module. During the pre-training process, abnormal data such as fraudulent orders and malicious reviews are automatically filtered out. After the pre-training is completed, a parameter version log is automatically generated, which supports the operator to roll back to the historical parameter version. The generated initial network parameters are directly imported into the reinforcement learning recommendation module.

[0016] Furthermore, it also includes a user feedback collection module, which is a data tracking SDK embedded in the user's app / mini-program. The output end is connected to the parameter iteration module, which is used to collect all user interaction behaviors on the recommendation results page in real time, automatically distinguish between positive and negative interactions and configure corresponding reward weights. All collected interaction data is anonymized before being transmitted to the parameter iteration module, which complies with personal information protection requirements. The collected feedback data is also stored in the experience pool for subsequent offline training.

[0017] Compared with existing technologies, the beneficial effects of this invention are: This invention constructs a multi-objective reward function that includes three optimization goals: user satisfaction, scenic area load balancing, and revenue contribution from cultural and tourism merchants. It is equipped with a dynamically adjustable weight mechanism that can automatically adjust the priority of each optimization goal according to different scenarios such as peak tourist season, off-season, and temporary events. While prioritizing user experience, it also takes into account the needs of scenic area visitor flow management and merchant revenue growth. This effectively solves the shortcomings of existing recommendation technologies that can only optimize a single goal and cannot balance the core interests of users, scenic areas, and merchants, and greatly improves the operational adaptability of the recommendation solution.

[0018] This invention employs a deep deterministic policy gradient network as the core of reinforcement learning recommendation. The input features integrate user static preferences, historical interaction features, real-time context features, and full-scale attribute features of cultural and tourism resources. It can combine dynamic information such as the user's current location, real-time weather, and companions to output recommendation results that better meet the user's immediate needs. At the same time, it achieves online incremental updates of model parameters through real-time feedback from user interaction, scenic spot visitor flow, and merchant revenue. This effectively solves the shortcomings of existing technologies, such as severe homogenization of recommendations and inability to adapt to dynamic scenario needs, thereby improving the accuracy of recommendation results and user acceptance.

[0019] This invention incorporates a pre-compliant verification process that automatically integrates with local cultural and tourism management rules, real-time ticketing systems for scenic spots, and user consumption profiles. This proactively filters out invalid cultural and tourism resources that do not meet management requirements, exceed the carrying capacity of scenic spots, or do not match users' spending power. While reducing the amount of invalid computation in the recommendation network and improving the recommendation response speed, it also avoids illegal and invalid recommendations. Furthermore, it eliminates the need for operators to manually maintain a large number of recommendation rules, effectively reducing the operational and maintenance costs of the solution. This addresses the shortcomings of existing manual rule-based solutions, such as high maintenance costs and a tendency to generate invalid recommendations. Attached Figure Description

[0020] Figure 1 This is a flowchart illustrating the overall process of personalized digital cultural tourism service recommendation based on reinforcement learning proposed in this invention. Figure 2 This is a flowchart of the multi-source data acquisition and feature vector generation process of this invention; Figure 3 This is a flowchart of the pre-compliance verification and offline pre-training process of this invention; Figure 4 This is a flowchart illustrating the execution of the reinforcement learning recommendation strategy and dynamic reward adjustment in this invention. Figure 5 This is a flowchart of the parameter iterative evolution and online incremental update process of the present invention. Detailed Implementation

[0021] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0022] Reference Figures 1 to 5 This invention discloses a method for recommending personalized digital cultural and tourism services based on reinforcement learning, including the following specific implementation steps: Step 1: Multi-source data acquisition Four types of data are collected from the user center MySQL database of the cultural tourism service platform, the front-end Flume log tracking channel, the user-end SDK positioning sensor, the cultural tourism resource management system PostgreSQL database, and the third-party public service RESTful API interface. Data transmission uses Kafka message queue to achieve peak smoothing and valley filling, with a peak support for stable collection of 100,000 data points per second. The collected raw data is uniformly stored in the HDFS distributed file system with a storage period of 3 years.

[0023] Step 2: Feature Vector Generation All collected raw data are preprocessed: in the data cleaning stage, the 3σ rule is used to remove outliers of continuous features, missing values ​​of discrete features are filled with the mode of the corresponding field, and missing values ​​of continuous features are filled with the mean of the corresponding field; in the feature encoding stage, one-hot encoding is used to encode discrete features such as user gender and cultural tourism resource category, and the maximum-minimum normalization method is used to scale continuous features such as user spending amount and length of stay in scenic spots to the [0,1] interval; finally, the processed features are concatenated into a fixed-dimensional vector: the user feature vector is 128-dimensional, the real-time context feature vector is 32-dimensional, and the cultural tourism resource feature vector is 64-dimensional. After concatenating the three types of vectors, a 224-dimensional input state vector is obtained.

[0024] Step 3: Construction of Multi-Objective Reward Function The formula for calculating the multi-objective reward function is as follows: ;in To enhance the total reward value of the learning network, For user satisfaction weight, For the load balancing weight of the scenic area, The weights for contributing to the revenue of cultural and tourism merchants are all within the range of [0,1] and satisfy the following conditions: ; The user satisfaction score is calculated as the weight of positive user interactions divided by the highest positive interaction weight, with negative interactions corresponding to negative scores. The formula for calculating the load balancing score for the scenic area is as follows: ,in To recommend the current visitor flow of the corresponding scenic spot, 0.7 represents the maximum carrying capacity of the scenic area, while 0.7 is the optimal load rate threshold commonly used in the cultural and tourism industry. The score is calculated based on the merchant's revenue contribution. ,in This represents the transaction volume generated for the corresponding merchants by this recommendation. This represents the average transaction amount for this merchant during the same period over the past 30 days.

[0025] Step 4: Reinforcement learning recommends output Using a 224-dimensional input state vector as the input to the Deep Deterministic Policy Gradient (DDPG) network, with the optimization goal of maximizing the total reward value, the DDPG network consists of two branches: an Actor policy network and a Critic evaluation network. The Actor network is a 3-layer fully connected structure with hidden layer dimensions of 256 and 128, respectively. The activation function is ReLU, and the output layer uses Sigmoid activation to output the recommendation score of each cultural and tourism resource, with a value range of [0,1]. The top 20 resources are sorted from high to low scores to form a candidate recommendation set for cultural and tourism services. The Critic network is a 3-layer fully connected structure with hidden layer dimensions of 256 and 128. The inputs are the state vector and the action vector output by the Actor, and the output is the Q-value corresponding to the action, which is used to evaluate the quality of the recommended action.

[0026] Step 5: Feedback and Iterative Updates After the candidate recommendation set is delivered to the user, user interaction data, real-time visitor flow data of scenic spots, and real-time revenue data of merchants are collected synchronously at a fixed interval of 1 hour. After calculating the corresponding reward value, the Actor network parameters are updated using the gradient ascent method and the Critic network parameters are updated using the gradient descent method. The parameter update task is placed in the RabbitMQ asynchronous queue for execution, which does not occupy the online recommendation computing resources. The response time of a single online recommendation request is consistently less than 50 milliseconds, and the concurrency supports 10,000 requests per second.

[0027] This invention also discloses specific field rules for various types of collected data: user static attribute data includes the user's age and place of residence filled in during registration, the 1-5 level of consumption capacity obtained from the real-name authentication interface, and cultural tourism preference tags such as natural scenery, history and culture, and family entertainment actively set by the user; historical cultural tourism interaction data includes user browsing history, order transaction records, 1-5 star rating records, stay time in scenic spots, order cancellation records, and search keyword records collected by platform tracking. Real-time context data includes the user terminal's current location collected by GPS, the system's current time, the current weather obtained from a third-party meteorological interface, the user's actively reported attributes of whether they are traveling with elderly or children, and the current mode of travel (driving / public transportation). Cultural and tourism resource attribute data includes category tags, latitude and longitude location tags, price range tags, real-time visitor flow tags, user average rating tags, opening time tags, suitable group tags such as parent-child / elderly / study tour tags, and current activity information tags for scenic spots, restaurants, accommodations, cultural and creative products, and performances. All fields have predefined enumerated values ​​to avoid outliers in feature encoding.

[0028] This invention also discloses the specific process of pre-compliance verification of cultural and tourism resources: after generating feature vectors of cultural and tourism resources, all cultural and tourism resources are input into a compliance verification module developed in Go language for parallel verification. The time taken for a single verification is less than 10 milliseconds. The specific verification logic includes three categories: the first category is to connect to the real-time policy interface of the local cultural and tourism regulatory department, synchronize the control rules once an hour, and filter out cultural and tourism resources that do not meet the current regional epidemic prevention requirements and temporary control requirements. The second type connects to the real-time ticketing management system interface of various scenic spots, retrieves the remaining reservation quota in real time, and filters out cultural and tourism resources with zero remaining quota or those exceeding the maximum carrying capacity of the scenic spot; the third type calls on the consumption profile data of the user center to filter out cultural and tourism resources whose average transaction price exceeds the highest acceptable threshold corresponding to the user's consumption ability level; cultural and tourism resources that meet all the verification conditions form a set of compliant resources to be recommended, and then input into the reinforcement learning recommendation strategy network for sorting, which can filter out more than 30% of invalid resources in advance, and significantly reduce the computational load of the recommendation network.

[0029] This invention also discloses a dynamic adjustment logic for multi-objective reward weights: the weights corresponding to the three optimization objectives of the multi-objective reward function support both automatic and manual adjustment modes. In automatic adjustment mode, the system pre-stores historical passenger flow data from the past three years, dividing time periods by week and holidays. If the historical average passenger flow for the current time period exceeds 150% of the annual average passenger flow, it is determined to be a peak tourist season, and the weights are automatically configured. , , Prioritize ensuring balanced visitor flow in scenic areas; if the historical average visitor flow for the current period is less than 60% of the annual average visitor flow, it is considered the off-season, and weights are automatically allocated. , , Prioritize increasing merchant revenue; during off-peak periods, the weighting is... , , The three types of objectives are optimized in a balanced manner; in the manual adjustment mode, the operator can adjust the weights by dragging and dropping sliders in the visual backend to adapt to the needs of temporary events such as cultural and tourism festivals, music festivals, and food festivals.

[0030] This invention also discloses the offline pre-training logic of the reinforcement learning recommendation policy network: Before the network goes online, it uses the historically stored data of the past 12 months of cultural tourism interaction for pre-training. During the pre-training process, abnormal orders are automatically filtered, such as orders placed by the same user more than 5 times within 1 hour, 1-star negative reviews without content, and orders with a single-day revenue increase of more than 100%. The dataset is divided into training and validation sets in an 8:2 ratio. The experience replay pool size is set to 1 million samples. Each time, 128 sets of samples are randomly sampled from the experience pool to calculate the loss. The optimizer is Adam, and the learning rate is set to 1e-4. The core pseudocode for pre-training is as follows: # DDPG Pre-training Core Logic import torch import numpy as np actor = ActorNet(input_dim=224, output_dim=1) critic = CriticNet(state_dim=224, action_dim=1) target_actor = ActorNet(input_dim=224, output_dim=1) target_critic = CriticNet(state_dim=224, action_dim=1) buffer = ReplayBuffer(capacity=1000000) for epoch in range(max_epoch): for step in range(max_step): s = get_state() # 224-dimensional state vector a = actor(s) + torch.normal(0, 0.1, size=a.shape) # Add exploration noise r, s_next = execute_action(a) # Execute the recommended action to obtain the reward and the next state. buffer.add(s, a, r, s_next) if buffer.size()>= 128: s_batch, a_batch, r_batch, s_next_batch = buffer.sample(128) # Update Critic network target_a = target_actor(s_next_batch) target_q = target_critic(s_next_batch, target_a) y = r_batch + 0.99 * target_q q = critic(s_batch, a_batch) critic_loss = torch.nn.MSELoss()(q, y) critic.optimizer.zero_grad() critic_loss.backward() critic.optimizer.step() # Update Actor Network actor_loss = -torch.mean(critic(s_batch, actor(s_batch))) actor.optimizer.zero_grad() actor_loss.backward() actor.optimizer.step() # Update target network for param, target_param in zip(actor.parameters(), target_actor.parameters()): target_param.data.copy_(0.001 * param.data + 0.999 *target_param.data) for param, target_param in zip(critic.parameters(),target_critic.parameters()): target_param.data.copy_(0.001 * param.data + 0.999 *target_param.data) # Validation set evaluation: Training is stopped if the improvement is less than 0.1% for 10 consecutive rounds. hit_rate = evaluate(valid_dataset, actor) if hit_rate - last_hit_rate<0.001 and epoch>= 10: break last_hit_rate = hit_rate The network parameters output after pre-training are used as initial parameters online. The pre-training process is executed on an independent offline computing cluster and does not consume online business resources.

[0031] This invention also discloses the logic for online incremental updates of network parameters: when collecting feedback data after the candidate recommendation set is deployed, priority is given to collecting user interaction data such as clicks, orders, and ratings on the recommendation results, and different reward weights are set according to the confidence level. A positive review of 4 stars or higher corresponds to a reward weight of 5; a completed order corresponds to a reward weight of 3; a click to view corresponds to a reward weight of 1; a negative review returned within 1 second of a user clicking corresponds to a reward of -0.5; a negative review of 1 star or lower corresponds to a negative reward of -5; and an order cancellation within 24 hours corresponds to a negative reward of -3. An online incremental update is triggered every 1,000 valid interaction data points. During the update, only the parameters of the last two fully connected layers of the network are modified, without modifying the parameters of the underlying feature extraction layers, to ensure the stability of the recommendation effect. The updated parameters are automatically released in a gray-scale manner, only to 10% of user traffic, and are released to the full user base after the effect is verified to meet the target.

[0032] This invention discloses a personalized digital cultural tourism service recommendation system based on reinforcement learning, comprising the following specific implementation modules: The data acquisition module is deployed on three 8-core, 16GB Alibaba Cloud ECS servers, using a Flume+Kafka architecture to achieve real-time acquisition and transmission of multi-source data. It supports integration with user-end apps / mini-programs, cultural tourism resource management systems, and third-party public service interfaces, with a data acquisition latency of less than 500 milliseconds and a peak acquisition capacity of 100,000 data points per second. The feature engineering module is built on the Spark distributed computing framework and equipped with a standardized feature processing rule engine, supporting feature processing of terabytes of data per day. The feature processing rules support visual configuration modification and take effect without redeploying the system. The multi-objective reward configuration module uses Vue + Spring Boot to develop a visual backend, providing drag-and-drop adjustment of weights, configuration of reward rules, and real-time preview of effects. Operators can adjust reward rules without needing to have algorithm knowledge. The reinforcement learning recommendation module is equipped with two Tesla T4 GPUs, with a response time of less than 50 milliseconds for a single recommendation request, supporting high-concurrency access of 10,000 recommendation requests per second, and achieving an availability of 99.99%. The parameter iteration module uses RabbitMQ as an asynchronous message queue. All parameter update tasks are placed in the queue and executed asynchronously to avoid parameter updates consuming the recommended online computing resources. If a task fails to execute, it will automatically retry 3 times. If the retry fails, an alarm will be issued to the operations and maintenance personnel.

[0033] This invention also discloses a specific implementation of the compliance verification module: the input of the compliance verification module is connected to the output of the feature engineering module, and the output is connected to the input of the reinforcement learning recommendation module. The module is developed using Go language, achieving a QPS of 20,000 and a single verification time of less than 10 milliseconds; the module simultaneously interfaces with three types of interfaces: First, it connects to the RESTful API of the local cultural and tourism regulatory department to synchronize the latest cultural and tourism control policies every hour and automatically update the filtering rules. Second, it connects to the ticketing system interface of each scenic spot to pull the remaining reservation volume of each scenic spot in real time with a data latency of less than 1 second. Third, it connects to the consumption profile interface of the user center to obtain the user's maximum acceptable consumption threshold in real time. The three types of verification logic are executed in parallel. Resources that pass the verification are stored in the compliant resource pool. The resource pool is valid for 15 minutes and is automatically refreshed upon expiration.

[0034] This invention also discloses a specific implementation of the pre-training module: the pre-training module is deployed on an independent offline computing cluster, which includes 10 computing nodes with 16 cores and 32G of RAM. The storage adopts the HDFS distributed file system with a storage capacity of 1PB. The output end is connected to the reinforcement learning recommendation module. The pre-training process automatically identifies and filters abnormal data such as fraudulent orders and malicious reviews. The abnormal rules support custom configuration. The pre-trained parameter versions are managed using Git. Each version corresponds to complete training logs and validation set performance metrics, allowing operators to roll back to any historical parameter version with one click, avoiding issues with new versions that could affect online business. The initial network parameters generated after pre-training are automatically imported into the reinforcement learning recommendation module, allowing for deployment without manual configuration.

[0035] This invention also discloses a specific implementation of the user feedback collection module: the user feedback collection module is a lightweight SDK embedded in the user-end APP / mini-program, with a size of less than 100KB, which will not affect the running performance of the user end, and the output end is connected to the parameter iteration module; the SDK collects all user interaction behaviors on the recommendation results page in real time, automatically distinguishes between positive and negative interactions and configures corresponding reward weights, all collected data is anonymized, the user's unique identifier is encrypted using SHA256 hash, and sensitive information such as the user's ID card number and mobile phone number is not collected; the collected feedback data is also stored in the experience pool for subsequent offline pre-training, the data storage period is 3 years, and it is automatically deleted upon expiration.

[0036] Reference Figure 1This diagram macroscopically illustrates the entire lifecycle execution of personalized digital cultural tourism service recommendations based on reinforcement learning. The system first collects multi-source heterogeneous data from both the user and cultural tourism resource sides, transforming it into standardized feature vectors through feature extraction. Then, guided by the construction of a multi-objective reward function containing multiple optimization indicators, the feature vectors are input into a deep deterministic policy gradient network for reinforcement learning decision-making, outputting a candidate recommendation set of cultural tourism services matching the current scenario's needs. Finally, multi-dimensional business operation feedback data is collected after the recommendation set is deployed, driving the closed-loop iteration and adaptive evolution of the reward function weights and the underlying recommendation strategy network parameters.

[0037] Reference Figure 2 This diagram details the data aggregation and feature engineering processes at the system's front end. The system synchronously performs multi-channel data retrieval operations, comprehensively collecting static user attribute data covering dimensions such as user age and spending power, as well as historical interaction behavior data including browsing history, orders, and dwell time. Simultaneously, the system captures real-time contextual information such as the user's current physical location, time, and weather conditions, and obtains attribute tags for all cultural and tourism resources. After all raw data is imported into the feature engineering module, it undergoes cleaning and dimensionality reduction calculations, and is mapped and generated into high-dimensional user feature vectors, real-time context feature vectors, and cultural and tourism resource feature vectors, providing a standard input foundation for subsequent models.

[0038] Reference Figure 3 This diagram focuses on two core pre-implementation safeguards before the recommendation engine executes. The first is the model parameter initialization phase. Before being deployed online, the reinforcement learning recommendation strategy network undergoes offline pre-training using historically stored data on the entire cultural and tourism interaction landscape, thereby constructing base network parameters with initial perceptual capabilities. The second is the resource supply-side security filtering phase. After generating initial cultural and tourism resource feature vectors, the system strictly executes compliance verification logic, automatically removing resource objects that do not meet current regional epidemic prevention and control requirements, exceed the maximum carrying capacity of scenic spots, or do not match users' spending power thresholds. This generates an absolutely compliant set of resources to be recommended, ensuring the security and feasibility of the recommendation results.

[0039] Reference Figure 4This diagram highlights the core recommendation algorithm's operational mechanism and reward adjustment strategy. The multi-objective reward configuration module constructs a multi-objective reward function that comprehensively considers user satisfaction, scenic area load balancing, and revenue contribution from cultural and tourism merchants, and assigns it dynamically adjustable weights. During peak tourist seasons, the system prioritizes increasing load balancing weights to alleviate congestion, while during off-seasons, it emphasizes revenue contribution weights to boost the economy. In the recommendation phase, the reinforcement learning module receives a set of compliant resources and user-related feature vectors. Under the continuous optimization guidance of the aforementioned dynamic multi-objective reward function, it drives a deep deterministic policy gradient network to perform nonlinear calculations and policy optimization, accurately outputting a candidate recommendation set that fits the current spatiotemporal context.

[0040] Reference Figure 5 This diagram illustrates the iterative evolution architecture of a recommendation system with closed-loop feedback and continuous evolution. After the candidate recommendation set is deployed, the feedback collection component tracks the business chain in real time, comprehensively capturing data from multiple terminals. On one hand, the system frequently collects real-time interactive behaviors from the user side, such as clicks, orders, and ratings, transforming them into real-time reward feedback to drive online incremental updates of the policy network parameters, maintaining the model's keen sensitivity to shifts in user interests. On the other hand, the system simultaneously aggregates macro-level statistics on actual visitor flow in scenic spots and actual revenue data from cultural and tourism merchants, integrating macro-level business performance with micro-level user behavior, and collaboratively updating the overall weight allocation matrix of the multi-objective reward function to achieve a robust improvement in global recommendation efficiency.

[0041] The above are merely preferred embodiments of the present invention, but the scope of protection of the present invention is not limited thereto. Any equivalent substitutions or modifications made by those skilled in the art within the scope of the technology disclosed in the present invention, based on the technical solution and inventive concept of the present invention, should be covered within the scope of protection of the present invention.

Claims

1. A method for recommending personalized digital cultural tourism services based on reinforcement learning, characterized in that, Includes the following steps: Collect static attribute data and historical cultural and tourism interaction data from the user side, real-time context data in the current scenario, and full attribute data from the cultural and tourism resource side; Feature extraction is performed on the collected data to generate user feature vectors, real-time context feature vectors, and cultural and tourism resource feature vectors, respectively. Construct a multi-objective reward function, wherein the optimization objectives of the multi-objective reward function include at least three categories: user satisfaction, scenic area load balancing, and revenue contribution of cultural and tourism merchants; It provides user feature vectors, real-time context feature vectors, and cultural and tourism resource feature vectors as inputs, provides a multi-objective reward function as the optimization guide, and adopts a deep deterministic policy gradient network as the reinforcement learning recommendation policy network to output a set of candidate recommendations for cultural and tourism services that match the current needs. Collect user interaction data, scenic spot visitor flow data, and merchant revenue data after the candidate recommendation set is deployed. Use the above data as feedback input to update the weights of the multi-objective reward function and the parameters of the reinforcement learning recommendation strategy network.

2. The method for recommending personalized digital cultural tourism services based on reinforcement learning according to claim 1, characterized in that, The user static attribute data includes user age, region, spending power, and preset cultural and tourism preference tags. The historical cultural and tourism interaction data includes user browsing history, order history, evaluation history, and scenic spot stay duration records. The real-time context data includes user current location, current time, current weather, and companion attributes. The cultural and tourism resource attribute data includes category tags, location tags, price tags, visitor flow tags, and rating tags for scenic spots, catering, accommodation, cultural and creative products, and performance resources.

3. The method for recommending personalized digital cultural tourism services based on reinforcement learning according to claim 2, characterized in that, After generating the feature vectors of cultural and tourism resources, the process also includes a preliminary compliance verification step. Specifically, this involves filtering out cultural and tourism resources that do not meet the current regional epidemic prevention requirements, the maximum carrying capacity of the corresponding scenic spot, or the user's preset consumption capacity threshold. After generating a set of compliant resources to be recommended, the results are input into the reinforcement learning recommendation strategy network.

4. The method for recommending personalized digital cultural tourism services based on reinforcement learning according to claim 3, characterized in that, The three optimization objectives of the multi-objective reward function are respectively set with dynamically adjustable weights. The dynamically adjustable weights are adjusted according to the current operation strategy of the cultural and tourism operator. During the peak tourist season, the weight corresponding to the load balance of the scenic spot is increased, and during the off-season, the weight corresponding to the revenue contribution of cultural and tourism merchants is increased.

5. The method for recommending personalized digital cultural tourism services based on reinforcement learning according to claim 4, characterized in that, Before the reinforcement learning recommendation strategy network is put into online operation, it first uses the historically stored full-domain cultural tourism interaction dataset for offline pre-training to obtain the initial network parameters before being put into online operation.

6. The method for recommending personalized digital cultural tourism services based on reinforcement learning according to claim 5, characterized in that, When collecting feedback data after the candidate recommendation set is deployed, priority is given to collecting user interaction data such as clicks, orders, and ratings on the recommendation results. This type of data is used as an immediate reward input into the reinforcement learning recommendation strategy network to achieve online incremental updates of network parameters.

7. A personalized digital cultural tourism service recommendation system based on reinforcement learning, used to implement the personalized digital cultural tourism service recommendation method based on reinforcement learning as described in any one of claims 1-6, characterized in that, include: The data acquisition module is used to collect static attribute data from the user side, historical cultural and tourism interaction data, real-time context data in the current scenario, and full attribute data from the cultural and tourism resource side. The feature engineering module is used to extract features from various types of collected data, generating user feature vectors, real-time context feature vectors, and cultural and tourism resource feature vectors, respectively. The multi-objective reward configuration module is used to construct a multi-objective reward function. The optimization objectives of the multi-objective reward function include at least three categories: user satisfaction, scenic area load balancing, and revenue contribution of cultural and tourism merchants. The reinforcement learning recommendation module takes user feature vectors, real-time context feature vectors, and cultural and tourism resource feature vectors as inputs, uses a multi-objective reward function as the optimization guide, and adopts a deep deterministic policy gradient network as the recommendation policy network to output a set of candidate recommendations for cultural and tourism services that match the current needs. The parameter iteration module is used to collect user interaction data, scenic spot visitor flow data, and merchant revenue data after the candidate recommendation set is deployed. The above data is used as feedback input to update the weights of the multi-objective reward function and the parameters of the reinforcement learning recommendation strategy network.

8. A personalized digital cultural tourism service recommendation system based on reinforcement learning according to claim 7, characterized in that, It also includes a compliance verification module. The input of the compliance verification module is connected to the output of the feature engineering module, and the output is connected to the input of the reinforcement learning recommendation module. The compliance verification module is used to filter out cultural and tourism resources that do not meet the current regional epidemic prevention requirements, the maximum carrying capacity of the corresponding scenic spot, and the user's preset consumption capacity threshold, and generate a set of compliant resources to be recommended and input it into the reinforcement learning recommendation module.

9. A personalized digital cultural tourism service recommendation system based on reinforcement learning according to claim 8, characterized in that, It also includes a pre-training module, which is connected to the reinforcement learning recommendation module and is used to call the historically stored full-domain cultural tourism interaction dataset to perform offline pre-training on the policy network of the reinforcement learning recommendation module and generate initial network parameters.

10. A personalized digital cultural tourism service recommendation system based on reinforcement learning according to claim 9, characterized in that, It also includes a user feedback collection module, whose output is connected to a parameter iteration module. This module collects user interaction data such as clicks, orders, and ratings related to the recommendation results and transmits it to the parameter iteration module as an input for immediate rewards, enabling online incremental updates of the strategy network.

Images