Large model training method based on cultural tourism scene

Abstract

The invention discloses a large model training method based on a cultural tourism scene. According to the specific implementation scheme, vertical corpora are extracted from a database, an initial data set is constructed, entities and relationships are extracted from the initial data set, and a text tourism knowledge graph is constructed; generating a question and answer pair based on manual annotation and the travel knowledge graph, inputting the question and answer pair into the reward model RM to output a quality score, and realizing initialization training of the reward model RM; improving interaction of the question-answer pairs by utilizing a knowledge verification large model to form new question-answer pairs; scoring the new question and answer pair by using the reward model RM which completes initialization training, filtering low-score samples, and taking high-score samples obtained after filtering as training samples of reinforcement learning; and using the training sample pair of reinforcement learning to finely adjust the reward model RM, and updating the parameters of the reward model RM based on the KL divergence. According to the invention, the professional degree of the model in the text travel scene can be improved; the semantic analysis and multi-round guiding capability of the model on fuzzy input is improved; and new and old knowledge fusion is balanced.

Description

Technical Field

[0001] This application relates to the field of large language model technology, and in particular to a method for training large models based on cultural tourism scenarios. Background Technology

[0002] In recent years, general-purpose large language models (LLMs), represented by the GPT series, Doubao, Qianwen, and DeepSeek, have made groundbreaking progress. These models, with their massive parameters and large-scale pre-training, have demonstrated outstanding capabilities in general natural language processing tasks, and are able to understand and generate fluent and coherent text.

[0003] However, when these models are directly applied to highly specialized vertical scenarios such as cultural tourism, their inherent limitations become apparent, mainly in the following three aspects:

[0004] A lack of expertise in vertical domains: While the general model possesses a broad knowledge base, it lacks sufficient depth and structured modeling for fields such as culture, history, and tourism. This leads to frequent factual errors when the model answers specialized questions, such as incorrectly stating the "excavation date of the Longmen Grottoes" as the Tang Dynasty instead of the correct Northern Wei Dynasty; or failing to provide accurate explanations of classical garden terminology such as "changing scenery with each step" and "borrowed scenery."

[0005] Superficial Understanding of User Intent in Contextualized Scenarios: In cultural and tourism interaction scenarios, user queries are often implicit and context-dependent. Existing models struggle to deeply analyze such complex intents. For example, when a user mentions "Xi'an night view," they cannot effectively link it to specific attractions and activities such as "the immersive performance at the Tang Paradise" or "the Bell Tower nighttime light show." The interaction pattern is passive and rigid, lacking the ability to proactively uncover and clarify users' deeper needs, resulting in a poor user experience.

[0006] Domain-adaptive tuning faces challenges: While traditional domain-adaptive methods (such as full parameter fine-tuning) can inject professional knowledge into the model to compensate for insufficient expertise, they are prone to the "catastrophic forgetting" problem—that is, after learning vertical domain knowledge, the model's original general language comprehension and reasoning abilities decline significantly. This "one-sided" dilemma makes it difficult for the model to effectively integrate and flexibly apply vertical domain knowledge while maintaining its core advantages.

[0007] Therefore, how to break through the bottlenecks of the general large model and develop a cultural tourism vertical industry model that has both profound professional knowledge and a deep understanding of user intent, without losing the original general capabilities, has become a key technical problem that the industry and academia urgently need to solve. Summary of the Invention

[0008] Based on this, this application provides a large-scale model training method based on cultural tourism scenarios to address the problems of insufficient vertical domain expertise, shallow understanding of scenario-based user intent, and difficulty in domain-adaptive optimization in existing large language models.

[0009] This application provides a method for training large-scale models based on cultural tourism scenarios, including:

[0010] Vertical corpora are extracted from the database to construct an initial dataset. Information extraction technology is used to extract entities and relationships from the initial dataset to construct a cultural tourism knowledge graph.

[0011] Question-answer pairs are generated based on manual annotation and cultural tourism knowledge graph. The question-answer pairs are then input into the reward model RM to output quality scores, thereby achieving the initial training of the reward model RM.

[0012] The interaction of the question-and-answer pairs is enhanced by using a knowledge verification model to form new question-and-answer pairs;

[0013] The new question-answer pair is scored using the reward model RM that has completed initial training, low-scoring samples are filtered out, and the high-scoring samples obtained after filtering are used as training samples for reinforcement learning.

[0014] The reward model RM is fine-tuned using reinforcement learning training samples, and its parameters are updated based on KL divergence. The initial training of the reward model RM includes the following operations:

[0015] When training a reward model RM using supervised fine-tuning, it is necessary to select the fine-tuning scenario and process the input data.

[0016] The operations for selecting the scene for fine-tuning include:

[0017] For the reward model RM in the cultural and tourism scenario, scenario knowledge injection is first performed. The scenario knowledge injection specifically includes the selection of pre-trained models and the design of fine-tuning strategies.

[0018] The pre-trained model selected is one with strong Chinese semantic understanding capabilities and supports fine-tuning in vertical domains.

[0019] The design of the fine-tuning strategy includes,

[0020] A low learning rate and multiple iterations are used to train the reward model RM, while gradient accumulation is used to improve the training stability of the data.

[0021] The operations that process the input data include,

[0022] A two-step process of manual annotation and verification using a cultural tourism knowledge graph is used to construct question-answer pairs.

[0023] The reward model RM is trained using a supervised fine-tuning method. The input of the model is question-answer pairs generated by manual annotation and verification by the cultural tourism knowledge graph. The output is a question-answer quality score between 0 and 1. The reward model RM outputs a continuous quality score of 0 to 1. During the training process, the loss function is cross-entropy, and BLEU and ROUGE-L are used as evaluation metrics for model performance.

[0024] It also includes,

[0025] During the pre-training knowledge fine-tuning phase, the knowledge retention rate in the GLUE benchmark test is monitored in real time. When the retention rate is below 90%, the learning rate is automatically adjusted.

[0026] Automatic learning rate adjustment includes the following operations:

[0027] Based on the theory of continuous learning, a dynamic learning rate adjustment strategy is designed to balance the integration of new and old knowledge.

[0028] During fine-tuning, the cosine similarity and parameter gradient norm between the current batch of data and the pre-trained knowledge are calculated in real time.

[0029] ;

[0030] Where, δ t θ represents the degree of difference between new knowledge and the model's inherent old knowledge. t θ represents the inference parameters for the current batch. pretrain λ1 and λ2 represent the average parameters during pre-training, and both λ1 and λ2 represent the weight coefficients.

[0031] The AdaFactor optimizer is used, and the learning rate is dynamically adjusted based on the bias metric. ,

[0032] ;

[0033] in, This represents the current learning rate. γ represents the base learning rate, and γ represents the learning rate adjustment factor.

[0034] When the difference between new knowledge and old knowledge is detected to be greater than τ, the learning rate is automatically reduced; otherwise, the learning rate is increased to accelerate the absorption of new knowledge.

[0035] In the post-processing stage of the reward model RM, the model's ability to understand intent and reason under fuzzy input is improved by fusing hard sample injection and Monte Carlo tree search.

[0036] The RM score of the low-scoring sample is lower than the first score setting value a, and the RM score of the high-scoring sample is greater than or equal to the second score setting value b, and the first score setting value a < the second score setting value b.

[0037] The operations for updating the RM parameters of the reward model based on KL divergence include:

[0038] During the RM parameter update phase of the reward model, KL divergence is used as a dual constraint tool for data distribution constraints and model output constraints;

[0039] Data distribution constraints include calculating the distribution difference between the newly generated question-answer pairs and the original training data to ensure that the new data learned by the reward model RM does not exceed a reasonable range of the domain.

[0040] The model output constraints include measuring the difference in the score distribution of the RM before and after the update for the same batch of data. If the standard deviation of the score distribution exceeds the set value, the KL regularization term will trigger a penalty mechanism.

[0041] The operations for updating the RM parameters of the reward model based on KL divergence also include...

[0042] New question-and-answer pairs are filtered in two stages, which include primary screening and KL verification. Primary screening specifically involves retaining high-scoring samples with RM scores greater than or equal to the second score setting value b. KL verification specifically involves calculating the KL divergence between the filtered samples and the original labeled data, and only retaining samples with divergence values ​​less than the third score setting value c into the update pool. The third score setting value c is less than the first score setting value a.

[0043] The loss function for updating the RM parameters of the reward model consists of three parts: data preference ranking loss (Pairwise Loss), KL divergence (KL Divergence), and cross-entropy loss (CrossEntropy Loss). The total loss of the RM for updating the RM parameters of the reward model is:

[0044] total_loss=α×PairwiseLoss+β×KL_Divergence+γ×CrossEntropyLoss;

[0045] Where α, β, and γ represent the weighting coefficients of data preference ranking loss, KL divergence, and cross-entropy loss, respectively.

[0046] Beneficial effects: This application enables the systematic modeling of cultural expertise and tourism scenario knowledge, enhancing the model's professionalism in cultural and tourism scenarios; improving the model's semantic parsing and multi-round guidance capabilities for fuzzy inputs; balancing the integration of new and old knowledge, and avoiding performance degradation during fine-tuning.

[0047] It should be understood that the description in this section is not intended to identify key or essential features of the embodiments of this application, nor is it intended to limit the scope of this application. Other features of this application will become readily apparent from the following description. Attached Figure Description

[0048] The accompanying drawings are provided for a better understanding of this solution and do not constitute a limitation of this application. Wherein:

[0049] Figure 1 This is based on the flowchart provided in this application. Detailed Implementation

[0050] The following description, in conjunction with the accompanying drawings, illustrates exemplary embodiments of this application, including various details to aid understanding. These should be considered merely exemplary. Therefore, those skilled in the art will recognize that various changes and modifications can be made to the embodiments described herein without departing from the scope and spirit of this application. Similarly, for clarity and brevity, descriptions of well-known functions and structures are omitted in the following description.

[0051] like Figure 1 As shown, this application provides a method for training large-scale models based on cultural tourism scenarios, including:

[0052] S1: Extract vertical corpora from the database to construct an initial dataset. Use information extraction techniques to extract entities and relationships from the initial dataset to construct a cultural tourism knowledge graph. It should be noted that:

[0053] The database includes a literature database, a classical poetry database, a local history database, a travel guide, and a scenic spot database. The vertical corpus includes literary and historical works, general cultural knowledge, classical Chinese poems, scenic spot introductions, and tourist Q&A.

[0054] We collected ancient poems, scenic spot introductions (such as "Geological formation of Wuhua Lake in Jiuzhaigou"), tourist Q&A (such as "Winter opening hours of Tianmen Mountain in Zhangjiajie"), literary and historical works, and general cultural knowledge from literary databases, local chronicle databases, and tourist guide databases to construct an initial dataset (data size ≥ 200GB).

[0055] Information extraction techniques (such as BERT-NER) are used to extract entities (such as "Mogao Grottoes" and "Along the River During the Qingming Festival") and relationships (such as "West Lake-Leifeng Pagoda-Adjacent") from the initial dataset, construct a cultural tourism knowledge graph, and improve the accuracy of the data by verifying the knowledge graph.

[0056] S2: Question-answer pairs are generated based on manual annotation and a cultural tourism knowledge graph. These pairs are then input into the reward model RM to output a quality score, thus achieving the initial training of the reward model RM. It should be noted that:

[0057] The initialization training operations for the reward model RM include,

[0058] When training a reward model RM using supervised fine-tuning, it is necessary to select the fine-tuning scenario and process the input data.

[0059] The operations for selecting the scene for fine-tuning include:

[0060] For the reward model RM in the cultural and tourism scenario, scenario knowledge injection is first performed. The scenario knowledge injection specifically includes the selection of pre-trained models and the design of fine-tuning strategies.

[0061] The pre-trained model selected is one with strong Chinese semantic understanding capabilities and supports fine-tuning in vertical domains.

[0062] The design of the fine-tuning strategy includes,

[0063] A low learning rate and multiple iterations are used to train the reward model RM, while gradient accumulation is used to improve the training stability of the data.

[0064] The operations that process the input data include,

[0065] A two-step process of manual annotation and verification using a cultural tourism knowledge graph is used to construct question-answer pairs.

[0066] The reward model RM is trained using a supervised fine-tuning method. The input of the model is question-answer pairs generated by manual annotation and verification by the cultural tourism knowledge graph. The output is a question-answer quality score between 0 and 1. The reward model RM outputs a continuous quality score of 0 to 1. During the training process, the loss function is cross-entropy, and BLEU and ROUGE-L are used as evaluation metrics for model performance.

[0067] The output of RM is defined as a continuous quality score from 0 to 1, with each score range corresponding to a specific quality level of cultural and tourism Q&A (as shown in Table 1), achieving a precise mapping between "score" and "quality".

[0068] Table 1: Specific quality levels for cultural and tourism Q&A for each scoring segment

[0069] Rating range Quality grade Criteria for Determining Cultural and Tourism Scenarios 0.8-1 excellent The information is accurate, tailored to your needs, and detailed (e.g., including transportation arrangements and booking reminders). 0.5-0.8 qualified The information is accurate and meets the core needs, but lacks detail (e.g., only the ticket price is answered, without mentioning the reservation method). 02.-0.5 Unqualified The information is partially incorrect (e.g., ticket price deviation ≤ 20%) or irrelevant (e.g., the user asked for directions, but only provided a brief description of the attraction). 0-0.2 Range The information is seriously incorrect (e.g., the opening hours are reversed) or completely irrelevant (e.g., the answer to a user's question about cultural and tourism policies is a food recommendation).

[0070] The core reason for choosing cross-entropy as the loss function is:

[0071] The quality score of cultural and tourism Q&A can be transformed into a "multi-classification task": the continuous score from 0 to 1 is discretized into multiple levels, and each level is regarded as a "quality category". Cross-entropy can effectively measure the difference between the category distribution predicted by RM and the actual category distribution labeled by humans.

[0072] The punishment for "incorrect scoring" is more precise: In the cultural and tourism scenario, the consequences of "classifying excellent Q&A as extremely poor" (such as misjudging accurate itinerary planning as incorrect information) are more severe. Cross-entropy, through logarithmic calculation, assigns higher loss values ​​to samples with large deviations between predicted probabilities and true labels, forcing RM to reduce such critical errors.

[0073] The manual annotation operation includes,

[0074] Using real-world data from the cultural and tourism industry as the source of labeled data, we developed labeling standards and assigned standardized personnel based on three dimensions: accuracy, relevance, and richness.

[0075] The construction of the cultural tourism knowledge graph includes,

[0076] With entities, relationships, and attributes as the core, a knowledge graph is constructed covering six major categories of entities (scenic spots, hotels, transportation hubs, cultural and tourism policies, intangible cultural heritage projects, and festival activities), such as "scenic spot (West Lake) - includes (Broken Bridge) - attributes (opening hours: all day, admission: free)";

[0077] The automatic verification process includes,

[0078] Manually annotated question-and-answer pairs are input into the knowledge graph verification system. Through entity matching (e.g., identifying whether "Forbidden City" in the answer corresponds to "Beijing Palace Museum" in the knowledge graph) and attribute comparison (e.g., verifying whether "ticket price" in the answer is consistent with the knowledge graph), question-and-answer pairs with incorrect information are filtered out. For content with doubts in the verification (such as policy interpretation), it is then manually reviewed to ensure that the accuracy of the question-and-answer pairs input into the RM is ≥98%.

[0079] In the post-processing stage of the reward model RM, the model's ability to understand intent and reason under fuzzy input is improved by fusing hard sample injection and Monte Carlo tree search.

[0080] BLEU and ROUGE-L are used as evaluation metrics for RM training performance, and their interpretations are as follows, taking into account the characteristics of cultural and tourism scenarios:

[0081] BLEU (Bilingual Assessment Alternate) is used to measure the RM's judgment on "question-answer consistency"—that is, the degree of lexical overlap between the answers generated by the assessment model and the reference answers (high-quality question-answer pairs annotated by humans). In "basic information questions about attractions" (such as tickets and opening hours), the BLEU value must be ≥0.7 (meaning that the answer contains ≥70% of key information words, such as "Forbidden City", "60 yuan", "book 7 days in advance"). If the BLEU value is <0.5, it means that the RM may not be able to identify low-quality questions and answers with "missing key information".

[0082] ROUGE-L (a variant of ROUGE based on the longest common subsequence) is used to measure the ability of the RM to judge the logical coherence of questions and answers—especially suitable for long text questions and answers in the cultural and tourism scenarios (such as itinerary planning and policy interpretation). It evaluates the semantic structural consistency between the answer and the reference answer through the longest common subsequence (LCS). In the "multi-attraction itinerary planning questions and answers", if the logic of the reference answer is "'hotel → attraction A (morning) → attraction B (afternoon) → transportation connection'", a ROUGE-L value ≥ 0.6 means that the RM can identify low-quality answers with "inverted logic (such as 'attraction B → attraction A')"; if the ROUGE-L value < 0.4, it indicates that the RM's ability to judge the logical coherence of cultural and tourism questions and answers is insufficient.

[0083] Initialization effect verification

[0084] After completing the fine-tuning training, the initial capabilities of the RM were verified through a "blind evaluation on the test set":

[0085] Test set composition: 1000 questions and answers were randomly selected from the cultural and tourism scenarios that were not trained (covering 4 major categories of needs), with excellent / qualified / unqualified / very poor question and answer pairs each accounting for 25%;

[0086] Validation criteria: The Pearson correlation coefficient between the RM's scores on the test set and the manually labeled scores is ≥0.8, and the misclassification rate for "excellent" and "very poor" answers is ≤5%;

[0087] Ultimate goal: To enable RM to initially possess "3 core capabilities"—① to identify errors in cultural and tourism information (such as discrepancies in ticket prices and opening hours); ② to determine the relevance of responses to needs (such as whether they deviate from the user's travel itinerary planning needs); ③ to assess the richness of content (such as whether key details such as reservations and transportation are supplemented), laying the foundation for the "reward signal generation" in the subsequent RLHF stage.

[0088] S3: Utilize a large-scale knowledge verification model to enhance the interaction of the question-answer pairs, forming new question-answer pairs. It should be noted that:

[0089] Based on role-playing mechanisms, two different large language models are used to simulate "tourists" and "tour guides" respectively:

[0090] The "tourist" randomly generates cultural and tourism-related questions (such as "How to plan a three-day trip to Dunhuang?").

[0091] The "guide" generates answers based on knowledge graphs and vertical corpora, forming new question-answer pairs (Qn, An).

[0092] S4: The new question-answer pairs are scored using the reward model RM, which has completed initial training. Low-scoring samples are filtered out, and the high-scoring samples obtained after filtering are used as training samples for reinforcement learning. It should be noted that:

[0093] The RM score of the low-scoring sample is lower than the first score setting value a, and the RM score of the high-scoring sample is greater than or equal to the second score setting value b, and the first score setting value a < the second score setting value b.

[0094] The new question-answer pairs are scored using RM, and low-scoring samples (first score setting value < 0.6) are filtered out, while high-scoring data (score ≥ second score setting value 0.8) are retained as training samples for reinforcement learning (DPO).

[0095] S5: Fine-tune the reward model RM using reinforcement learning training samples and update the RM parameters based on KL divergence. It should be noted that:

[0096] The operations for updating the RM parameters of the reward model based on KL divergence include:

[0097] During the RM parameter update phase of the reward model, KL divergence is used as a dual constraint tool for data distribution constraints and model output constraints;

[0098] Data distribution constraints include calculating the distribution difference between the newly generated question-answer pairs and the original training data to ensure that the new data learned by the reward model RM does not exceed a reasonable range of the domain.

[0099] The model output constraints include measuring the difference in the score distribution of the RM before and after the update for the same batch of data. If the standard deviation of the score distribution exceeds the set value, the KL regularization term will trigger a penalty mechanism.

[0100] The operations for updating the RM parameters of the reward model based on KL divergence also include...

[0101] New question-and-answer pairs are filtered in two stages, which include primary screening and KL verification. Primary screening specifically involves retaining high-scoring samples with RM scores greater than or equal to the second score setting value b. KL verification specifically involves calculating the KL divergence between the filtered samples and the original labeled data, and only retaining samples with divergence values ​​less than the third score setting value c into the update pool. The third score setting value c is less than the first score setting value a.

[0102] The loss function for updating the RM parameters of the reward model consists of three parts: data preference ranking loss (Pairwise Loss), KL divergence (KL Divergence), and cross-entropy loss (CrossEntropy Loss). The total loss of the RM for updating the RM parameters of the reward model is:

[0103] total_loss=α×PairwiseLoss+β×KL_Divergence+γ×CrossEntropyLoss;

[0104] Where α, β, and γ represent the weighting coefficients of data preference ranking loss, KL divergence, and cross-entropy loss, respectively.

[0105] PairwiseLoss: Based on human preference ranking of new data (e.g., answer A is better than answer B), it maximizes the score difference of high-quality question-answer pairs.

[0106] KL_Divergence: Constrains the difference in score distribution between the current RM and historical versions on the same batch of data, expressed as KL(P_new||P_old), where P_new is the output distribution of the model during the update, and P_old is the stable distribution before the update.

[0107] Weighting coefficients: α=0.7 (to ensure scoring accuracy), β=0.2 (to control update magnitude), γ=0.1 (to maintain basic classification ability). In cultural and tourism scenarios, the β value can be increased to 0.3 to enhance stability.

[0108] Phased optimization strategy

[0109] Phase 1 (Adaptation Period): Using a low learning rate (1e-5) and high KL weights (β=0.3), the model initially learns the new data patterns while maintaining compatibility with the old standards;

[0110] Phase 2 (convergence period): Reduce the KL weight to 0.1 and increase the learning rate to 3e-5 to accelerate the model's absorption of high-quality new features, such as updating the scoring criteria for emerging demands in cultural and tourism scenarios, like the "smart scenic spot reservation process".

[0111] Enhanced adaptation to cultural and tourism scenarios

[0112] ① Physical-level KL monitoring

[0113] A dedicated entity distribution database for the cultural tourism industry is constructed (including six major categories of entities such as scenic spots, policies, and facilities). The KL divergence between the probability of entities appearing in new data and the database distribution is calculated. When the KL divergence of questions and answers related to "intangible cultural heritage experience projects" increases abnormally, domain expert review is automatically triggered to prevent misjudgments by the Resource Manager (RM) regarding emerging cultural tourism formats.

[0114] ② Dynamic threshold adjustment

[0115] For rigid information (such as opening hours and ticket prices), a strict KL threshold (<0.2) is used to ensure the stability of the evaluation criteria; for flexible needs (such as itinerary planning suggestions), a lenient threshold (<0.4) is used to allow RM to learn richer scoring dimensions (such as the rationality of personalized recommendations).

[0116] ③ Mode collapse protection

[0117] The entropy value of the RM output distribution is monitored in real time. When the entropy value decreases for three consecutive rounds and the KL divergence is greater than 0.5, the training set is automatically reconstructed by introducing high-quality historical data to prevent the model from being over-optimized to a single scoring mode (such as focusing only on the accuracy of information while ignoring the user experience).

[0118] Verification system for update effect

[0119] After completing the parameter update, the RM performance was verified using three metrics: consistency, stability, and cycle gain.

[0120] Consistency verification: The correlation coefficient between the new RM and the manually labeled Pearson correlation coefficient must be ≥0.85;

[0121] Stability verification: The KL divergence of the scores on the same test set compared to previous versions must be <0.2;

[0122] Cyclic gain: Use the updated RM to guide the fine-tuning of the policy model, and the quality rate of newly generated data (≥0.8) needs to be improved by ≥15%.

[0123] Through this mechanism, KL divergence serves both as a "distribution alarm" to ensure data quality and as a "braking system" to control the evolution of RM, ultimately achieving the synergistic evolution of evaluation criteria and strategy capabilities, and building a sustainable model optimization closed loop in vertical fields such as culture and tourism.

[0124] It also includes,

[0125] During the pre-training knowledge fine-tuning phase, the knowledge retention rate in the GLUE benchmark test is monitored in real time. When the retention rate is below 90%, the learning rate is automatically adjusted.

[0126] Automatic learning rate adjustment includes the following operations:

[0127] Based on the theory of continuous learning, a dynamic learning rate adjustment strategy is designed to balance the integration of new and old knowledge.

[0128] During fine-tuning, the cosine similarity and parameter gradient norm between the current batch of data and the pre-trained knowledge are calculated in real time.

[0129] ;

[0130] Where, δ t θ represents the degree of difference between new knowledge and the model's inherent old knowledge. t θ represents the inference parameters for the current batch. pretrain λ1 and λ2 represent the average parameters during pre-training, and both λ1 and λ2 represent the weight coefficients.

[0131] The AdaFactor optimizer is used, and the learning rate is dynamically adjusted based on the bias metric. ,

[0132] ;

[0133] in, This represents the current learning rate. γ represents the base learning rate, and γ represents the learning rate adjustment factor.

[0134] When the difference between new and old knowledge is detected to be δt > τ, the learning rate is automatically reduced; conversely, the learning rate is increased to accelerate the absorption of new knowledge. This effectively balances the injection of new knowledge with the retention of old knowledge, significantly improving the efficiency of injecting knowledge such as classical Chinese texts and poems, while maintaining the model's general capabilities.

[0135] Constructing a "short input, multiple intents" dataset for cultural tourism scenarios requires combining domain characteristics with the need for few-shot learning, achieved through multi-dimensional data collection, refined annotation, and intelligent enhancement techniques. The specific methodological framework is as follows:

[0136] I. Intent Classification System Design

[0137] Domain division and hierarchical structure

[0138] Referring to the classification logic of the aviation tourism industry, cultural tourism intentions can be divided into two core areas:

[0139] Query domain: Focuses on information retrieval, such as "ticket prices," "opening hours," and "transportation methods";

[0140] Behavior area: Covering action-oriented needs such as "route planning", "ticket booking", and "intangible cultural heritage experience".

[0141] Based on this, a multi-level intent system can be built, for example:

[0142] Primary Intent: Scenic Spot Information;

[0143] Secondary intent: Tickets (price, discounts, booking), routes (hiking routes, driving navigation), weather (probability of sea of ​​clouds, sunrise time);

[0144] Cultural characteristics intended to expand

[0145] In light of the unique characteristics of cultural tourism scenarios, additional intentions related to cultural experiences have been added:

[0146] Cultural courses: such as Huizhou seal carving and Tang Dynasty palace banquet dance instruction;

[0147] Intangible cultural heritage activities: such as fish lantern making and participation in Nuo opera performances;

[0148] Historical interpretation: such as the structure of ancient buildings and the origins of anecdotes about famous people.

[0149] Taking "Huangshan" as an example, it can be associated with distinctive features such as "cloud sea observation techniques" and "Huizhou ink making experience".

[0150] II. Data Collection and Labeling Strategies

[0151] (I) Multi-source data fusion

[0152] Structured data mining

[0153] Extract short inputs and their attributes from publicly available tourism databases (such as a dataset of 352 city attractions):

[0154] Attraction names: "Huangshan", "Terracotta Warriors", "Tang Dynasty Never Sleeps City";

[0155] Related intent: Automatically extract fields such as ticket prices, route recommendations, and intangible cultural heritage activities to generate initial annotations.

[0156] For example, "Huangshan" can obtain ticket prices (corresponding to the "ticket" intent), hiking routes (corresponding to the "route" intent), and cloud sea observation points (corresponding to the "cloud sea probability" intent) from the database.

[0157] User-generated content crawling

[0158] Forums and Q&A platforms: Scrape user questions from platforms such as Reddit, Zhihu, and Mafengwo, such as "How much are the tickets to Huangshan?" and "Best hiking routes in Huangshan", and label them with the corresponding intent ("tickets" and "routes").

[0159] Social Media: Analyze the titles and comments of short videos on platforms such as Douyin and Xiaohongshu to extract high-frequency demands, such as the intent of "sunrise time" and "photography tips" corresponding to "Huangshan sunrise guide".

[0160] Expert knowledge injection

[0161] Invite tourism industry experts to generate typical short inputs and their intent tags:

[0162] Cultural category: "Dunhuang Mogao Grottoes" → "Mural Restoration Course" "Special Grotto Reservation";

[0163] Experiential activities: "Xi'an Datang Everbright City" → "Night Light Show" and "Hanfu Rental".

[0164] (ii) Multi-intent annotation method

[0165] Labeling process design

[0166] Crowdsourced labeling: Through external crowdsourcing platforms or self-built platforms, labelers are required to check all relevant intent tags for each short input and add associated entities (e.g., for "Huangshan", "Bright Summit" and "Welcoming Pine" are required to be labeled as scenic spots).

[0167] Consistency verification: Kappa coefficient is used to evaluate consistency among annotators, and dissenting samples are reviewed by experts.

[0168] Syntactic and semantic auxiliary annotation

[0169] Dependency parsing: Identifies parallel structures (such as "tickets and routes"), verb-object relationships (such as "book tickets"), and automatically tags multi-intent candidates;

[0170] Semantic similarity matching: Sentence-BERT is used to calculate the similarity between short inputs and intent templates. For example, when the similarity between "How to get to Huangshan?" and the "Route planning" template reaches 0.85, the "Route" intent is automatically labeled.

[0171] III. Data Augmentation and Balancing Techniques

[0172] Text enhancement strategies

[0173] Synonym substitution: Expand "Huangshan" to "Huangshan Scenic Area" or "Huangshan Travel Guide" to maintain the same intent but diversify the forms;

[0174] Sentence transformation: Change "Huangshan ticket price" to "How much is the Huangshan ticket?" or "Huangshan ticket fee";

[0175] Entity replacement: Generate cross-scenic spot samples such as "Mount Tai ticket" and "Mount Hua route" to enhance the model's generalization ability.

[0176] Synthesis of few samples

[0177] For low-frequency intents (such as "probability of cloud sea"), MLSMOTE technology is used to synthesize samples:

[0178] Feature vector generation: New samples are generated in the feature space based on the SMOTE algorithm, such as interpolation between "Huangshan" and "Yunhai observation point";

[0179] Label set generation: The intended labels of the synthetic samples are determined by ranking to ensure consistency with the original data distribution.

[0180] Data balancing

[0181] Oversampling: Duplicate samples of minority intentions (such as "intangible cultural heritage experience") and add noise;

[0182] Undersampling: Randomly remove some samples from the majority class intent (such as "tickets") to avoid model overfitting.

[0183] IV. Learning Adaptation and Optimization Tips

[0184] Prompt template design

[0185] General template: "User asks for information about [Huangshan], intent includes: [MASK1], [MASK2], [MASK3]", where [MASK] corresponds to the intent label output by the Softmax layer;

[0186] Domain Template: For cultural experience scenarios, design “[Huangshan] cultural activities include: [Intangible Cultural Heritage Experience], [Historical Explanation]” to guide the model to focus on cultural intentions.

[0187] Verbalizer (Tag Mapping)

[0188] Establish a mapping relationship between intent labels and model output:

[0189] Python

[0190] label_word_map={

[0191] "Tickets":["Ticket Price","How to Purchase Tickets"],

[0192] "Route":["Hiking route","Driving navigation"],

[0193] "Probability of Cloud Sea": ["Cloud Sea Prediction", "Weather Conditions"]

[0194] }

[0195] Use SoftVerbalizer to convert the model's output logits into an intent probability distribution. For example, the probability of "cloud sea probability" corresponding to the tag "cloud sea prediction" is 0.75.

[0196] Few-sample fine-tuning strategy

[0197] Freeze base parameters: Only fine-tune the Prompt vector and Softmax layer to reduce the risk of overfitting;

[0198] Dynamic Prompt Expansion: Based on the performance of the validation set, automatically adjust the number of intent candidates in the template, for example, expand [MASK1] and [MASK2] to [MASK1], [MASK2], and [MASK3] to cover more intents.

[0199] Dataset validation and iteration

[0200] Hierarchical cross-validation

[0201] The dataset is stratified according to intent distribution to ensure that the intent ratio is consistent across the training, validation, and test sets. For example, the intent for "tickets" accounts for 30% in each subset.

[0202] Model performance evaluation

[0203] Multi-label metrics: F1 score and Hamming Loss were used to evaluate the performance of multi-intent recognition;

[0204] Domain adaptability testing: Test the model's generalization ability in unlabeled cultural tourism scenarios (such as "Ancient City of Huizhou"), and correct the labeling rules based on mislabeled samples.

[0205] Continuous iteration mechanism

[0206] Real-time data updates: Regularly crawl data from tourism platforms to supplement intent tags for newly opened attractions (such as "new Huangshan Yung Valley Cableway route");

[0207] User feedback loop: Collect real user satisfaction with the model output through A / B testing, and optimize the quality of dataset annotation in reverse.

[0208] Using the above methods, a "short input-multiple intent" dataset covering the core needs of cultural tourism and adapted to prompt learning can be constructed, providing high-quality training data for the precise fine-tuning of large models in this scenario.

[0209] Multi-turn dialogue simulation: Using dialogue state tracking (DST) technology, the model is trained to actively collect key information and build a complete intent chain when the user input is incomplete.

[0210] Reasoning ability optimization:

[0211] The proposal suggests introducing an MCTS reward term into the reward function:

[0212] For complex problems (such as "What Three Kingdoms cultural attractions are there along the way from Xi'an to Chengdu?"), the large language model needs to generate a reasoning tree (nodes represent attractions, and edges represent relationships).

[0213] The path rationality is evaluated using MCTS (e.g., node coverage ≥ 80%, correct time sequence), and the reward function is:

[0214] R=R answer +αR mcts ;

[0215] Among them, R answer To score the accuracy of the answer, R mcts The quality score of the reasoning tree is represented by α=0.3, which is the weighting coefficient.

[0216] By optimizing the model's ability to solve complex problems through inference trees, the model can be helped to model geospatial relationships, making the model perform more reasonably in scenarios such as spatial understanding and route planning.

[0217] Model Deployment and Lightweighting

[0218] Knowledge distillation is used to compress large models into lightweight versions, which are then deployed on edge devices using quantization technology (4-bit Normal Float) to support real-time question answering in cultural and tourism scenarios.

[0219] The specific implementation method is as follows:

[0220] I. Implementation Environment and Data Preparation

[0221] Hardware environment: 8 NVIDIA A100 GPU servers (80GB of VRAM each), running Ubuntu 20.04, with PyTorch 2.0 as the distributed training framework;

[0222] Dataset Construction:

[0223] Vertical cultural knowledge set: Collected 500,000 data entries in history (such as dynastic culture, celebrity footprints) and literature (such as poetry and allusions, famous works scenes), and constructed a knowledge graph containing a ternary set of "scenic spot-cultural attribute-knowledge tag" (such as "Wuzhen-Jiangnan water town-filming location of 'Like Water Flowing Years'").

[0224] User interaction dataset: Collected 1 million real tourism scenario dialogue data (including fuzzy inputs such as "ancient towns in Jiangnan" and "Silk Road heritage") and 500,000 user tourism preference data (including destination type, length of stay, and cultural interest tags).

[0225] Test sets include: a 2,000+ history / literature-related question test set specified by the user (e.g., "Which city's scenic spots are described in 'A Night Mooring by Maple Bridge'?"), a 1,000+ fuzzy input test set (e.g., "Want to visit a southern town with an ancient bridge"), a GLUE benchmark test set, and a 500-question complex question test set (e.g., "Distribution of World Heritage sites along the Silk Road").

[0226] II. Model Training Process

[0227] 1. Training of Vertical Knowledge Modeling Module (Improving the Accuracy of Cultural Knowledge Question Answering and Recommendation)

[0228] Knowledge integration: Based on pre-trained general large models (such as BERT-base), optimization is achieved through "knowledge graph embedding + contrastive learning": triples in the cultural knowledge graph are transformed into vectors and fused with the model word vectors. A contrastive loss function is designed (to reduce the distance between the vectors of scenic spots with similar cultural labels, such as increasing the similarity between the vectors of "Wuzhen" and "Xitang" by 30%).

[0229] Recommendation optimization: Introduce "cultural matching weight" into the recommendation module, combine user preferences (such as "preference for Ming and Qing architecture") with the cultural attributes of scenic spots (such as "Xitang - Ming and Qing residential buildings") to calculate matching scores and optimize the recommendation ranking logic.

[0230] 2. Training of the User Intent Understanding Module (Deepening Fuzzy Input Parsing and Multi-Round Guidance)

[0231] Fuzzy Input Parsing: Constructing an "Intent Candidate Generator + Semantic Matching Model": For fuzzy input (such as "Ancient Town in Jiangnan"), first generate 10 candidate intent tags (such as "Ancient Town in Jiangnan Water Town" and "Ancient Town Photography"), then calculate the similarity between the candidate tags and the scenic spot database through a pre-trained semantic model, and select the Top 5 related scenic spots (such as "Wuzhen", "Xitang", "Zhouzhuang", "Tongli", and "Luzhi").

[0232] Multi-round guidance optimization: Design intent to complete the dialogue template (e.g., when a user inputs "want to visit cultural attractions", the model asks "do you prefer historical sites, literary scenes or intangible cultural heritage experiences?"), and optimize the guidance strategy based on reinforcement learning (RL), with the reward function being "the reduction in the number of rounds when the user finally confirms the intent".

[0233] 3. Model robustness enhancement training

[0234] Dynamic learning rate adjustment: During the pre-training knowledge fine-tuning stage, the knowledge retention rate (such as "accuracy of historical event timeline") in the GLUE benchmark test is monitored in real time. When the retention rate is lower than 90%, the learning rate is automatically reduced from 1e-5 to 5e-6 to avoid knowledge forgetting.

[0235] RLHF+MCTS Joint Optimization:

[0236] RLHF phase: Invite 20 experts in the field of cultural tourism to score the model output (1-5 points) for complex problems (such as "distribution of World Heritage sites along the Silk Road"). Use "expert scores" as feedback signals to optimize the model inference weights.

[0237] MCTS stage: Construct reasoning path trees for complex problems (such as "Silk Road → segmented regions → heritage in each region → heritage features"), and use Monte Carlo Tree Search (MCTS) to select the optimal reasoning path to improve logical integrity.

[0238] III. Results of Effect Verification

[0239] Vertical knowledge modeling results:

[0240] Accuracy of cultural knowledge questions: On the 2k+ history / literature test set, it improved from 62% to 97% of the general model (e.g., the accuracy of "the scenic spot corresponding to 'The Story of the Drunken Old Man's Pavilion'" improved from 58% to 100%).

[0241] Tourism recommendation results: A / B testing was conducted on 10,000 randomly selected users. The click-through rate of the recommended attractions increased by 45%, and the NPS score in the user satisfaction survey increased from 42 to 89.

[0242] User intent understanding effect:

[0243] Fuzzy input parsing accuracy: On a 1k+ fuzzy input test set, it improved from 55% to 92% (e.g., the proportion of correctly associating "Jiangnan Ancient Town" with the Top 5 scenic spots improved from 52% to 93%).

[0244] Multi-round guidance success rate: For "users with vague needs" (such as only entering "want to travel"), the success rate of clarifying users' needs after multiple rounds of guidance reached 95%, and the average number of times users entered the information decreased from 5 to 1.2.

[0245] Model robustness:

[0246] Pre-training knowledge retention: In the GLUE benchmark test, the knowledge retention rate increased from 78% to 96%, and the probability of catastrophic forgetting decreased by 82%;

[0247] Reasoning accuracy for complex questions: On a test set of 500 complex questions, the reasoning accuracy improved from 61% to 90% (an increase of 29 percentage points). For example, the completeness score of the answer to the question "Distribution of World Heritage Sites along the Silk Road" improved from 5.2 points (out of 10) to 9.1 points.

[0248] The above description is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any variations or substitutions within the technical scope disclosed in the present invention should be covered within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the scope of the claims.

Claims

1. A large model training method based on a cultural tourism scene, characterized in that, include: Vertical corpora are extracted from the database to construct an initial dataset. Information extraction technology is used to extract entities and relationships from the initial dataset to construct a cultural tourism knowledge graph. Question-answer pairs are generated based on manual annotation and cultural tourism knowledge graph. The question-answer pairs are then input into the reward model RM to output quality scores, thereby achieving the initial training of the reward model RM. The interaction of the question-and-answer pairs is enhanced by using a knowledge verification model to form new question-and-answer pairs; The new question-answer pair is scored using the reward model RM that has completed initial training, low-scoring samples are filtered out, and the high-scoring samples obtained after filtering are used as training samples for reinforcement learning. The reward model RM is fine-tuned using reinforcement learning training samples, and the parameters of the reward model RM are updated based on KL divergence.

2. The method of claim 1, wherein: The database includes a literature database, a classical poetry database, a local history database, a travel guide, and a scenic spot database. The vertical corpus includes literary and historical works, general cultural knowledge, classical Chinese poems, scenic spot introductions, and tourist Q&A.

3. The method of claim 2, wherein: The initialization training operations for the reward model RM include, When training a reward model RM using supervised fine-tuning, it is necessary to select the fine-tuning scenario and process the input data. The operations for selecting the scene for fine-tuning include: For the reward model RM in the cultural and tourism scenario, scenario knowledge injection is first performed. The scenario knowledge injection specifically includes the selection of pre-trained models and the design of fine-tuning strategies. The pre-trained model selected is one with strong Chinese semantic understanding capabilities and supports fine-tuning in vertical domains. The design of the fine-tuning strategy includes, A low learning rate and multiple iterations are used to train the reward model RM, while gradient accumulation is used to improve the training stability of the data. The operations that process the input data include, A two-step process of manual annotation and verification using a cultural tourism knowledge graph is used to construct question-answer pairs.

4. The method of claim 3, wherein the method is characterized by: The reward model RM is trained using a supervised fine-tuning method. The input of the model is question-answer pairs generated by manual annotation and verification by the cultural tourism knowledge graph. The output is a question-answer quality score between 0 and 1. The reward model RM outputs a continuous quality score of 0 to 1. During the training process, the loss function is cross-entropy, and BLEU and ROUGE-L are used as evaluation metrics for model performance.

5. The method of claim 4, wherein: It also includes, During the pre-training knowledge fine-tuning phase, the knowledge retention rate in the GLUE benchmark test is monitored in real time. When the retention rate is below 90%, the learning rate is automatically adjusted.

6. The method of claim 5, wherein: Automatic learning rate adjustment includes the following operations: Based on the theory of continuous learning, a dynamic learning rate adjustment strategy is designed to balance the integration of new and old knowledge. During fine-tuning, the cosine similarity and parameter gradient norm between the current batch of data and the pre-trained knowledge are calculated in real time. ； wherein, δ t represents the difference degree of new knowledge and model inherent old knowledge, θ t represents the inference parameter of the current batch, θ pretrain represents the pre-training average parameter, λ1 and λ2 both represent weight coefficients; Using the AdaFactor optimizer, combined with bias metric results to dynamically adjust the learning rate , ； wherein, denotes the current learning rate, denotes the base learning rate, and γ denotes a learning rate adjustment factor; When the difference between new knowledge and old knowledge is detected to be greater than τ, the learning rate is automatically reduced; otherwise, the learning rate is increased to accelerate the absorption of new knowledge.

7. The method for training a large model based on a cultural tourism scenario according to claim 6, characterized in that: In the post-processing stage of the reward model RM, the model's ability to understand intent and reason under fuzzy input is improved by fusing hard sample injection and Monte Carlo tree search.

8. The method of claim 1 or 7, wherein the method is characterized by: The RM score of the low-scoring sample is lower than the first score setting value a, and the RM score of the high-scoring sample is greater than or equal to the second score setting value b, and the first score setting value a < the second score setting value b.

9. The method of claim 8, wherein: The operations for updating the RM parameters of the reward model based on KL divergence include: During the RM parameter update phase of the reward model, KL divergence is used as a dual constraint tool for data distribution constraints and model output constraints; Data distribution constraints include calculating the distribution difference between the newly generated question-answer pairs and the original training data to ensure that the new data learned by the reward model RM does not exceed a reasonable range of the domain. The model output constraints include measuring the difference in the score distribution of the RM before and after the update for the same batch of data. If the standard deviation of the score distribution exceeds the set value, the KL regularization term will trigger a penalty mechanism.

10. The method of claim 9, wherein the method further comprises: The operations for updating the RM parameters of the reward model based on KL divergence also include... New question-and-answer pairs are filtered in two stages, which include primary screening and KL verification. Primary screening specifically retains high-scoring samples with RM scores greater than or equal to the second score setting value b. KL verification specifically calculates the KL divergence between the filtered samples and the original labeled data, and only retains samples with divergence values ​​less than the third score setting value c into the update pool. The third score setting value c is less than the first score setting value a. The loss function for updating the RM parameters of the reward model consists of three parts: data preference ranking loss (Pairwise Loss), KL divergence (KL Divergence), and cross-entropy loss (CrossEntropy Loss). The total loss of the RM for updating the RM parameters of the reward model is: total_loss = α×PairwiseLoss + β×KL_Divergence + γ×CrossEntropyLoss; Where α, β, and γ represent the weighting coefficients of data preference ranking loss, KL divergence, and cross-entropy loss, respectively.

Images