Dialogue strategy optimization method and apparatus based on intent trajectory preference comparison

Abstract

This invention relates to the field of data processing technology, and in particular to a method and apparatus for optimizing dialogue strategies based on intention trajectory preference comparison. The method constructs preference triples from real human-computer interaction logs, using user intention states as anchors, and based on the change in Euclidean distance between the user intention state vector and the target intention state vector during multiple rounds of dialogue. These preference triples are then used as training samples. A pre-trained dialogue strategy model is fine-tuned by introducing a direct preference optimization loss function with intention trajectory quality weights to obtain an optimized model. The intention trajectory quality weights reflect the difference in the magnitude of intention state changes between positive and negative sample pairs. Effective strategies are extracted from the optimized dialogue strategy model and transformed into structured rules. These structured rules are then reviewed and semantically expanded before being added to a rule base and used as training samples for the next round of dialogue strategy model iteration, thereby achieving continuous optimization of the dialogue strategy.

Description

Technical Field

[0001] This invention relates to the field of data processing technology, and in particular to a method and apparatus for optimizing dialogue strategies based on intention trajectory preference comparison. Background Technology

[0002] Natural language-based human-computer dialogue systems have been widely applied in scenarios such as retail sales guidance, financial marketing, medical consultation, education and training, and customer service. In practical applications, dialogue systems not only need to complete information question-and-answer sessions, but also need to guide user intentions towards a target direction through dynamic dialogue strategies, such as increasing purchase intention, reducing doubts, enhancing user trust, or facilitating a final decision. Existing dialogue strategy systems mainly include the following technical approaches:

[0003] The first category is dialogue strategy methods based on human-written rules. These methods typically involve business experts pre-writing rules, such as "trigger a discount explanation strategy when a user expresses price concerns" or "trigger a sales promotion strategy when a user shows high interest." While this method offers some interpretability, it heavily relies on human experience, has high rule-building costs, limited coverage, and struggles to adapt to complex and ever-changing real-world interaction scenarios. Furthermore, the rule update cycle is long, failing to evolve continuously with changes in user behavior.

[0004] The second category comprises dialogue policy optimization methods based on reinforcement learning (RL). These methods define a reward function, allowing the dialogue system to learn the optimal policy through long-term interactions. For example, the policy model can be optimized using reward signals such as conversion rate, user satisfaction, or task completion rate. However, reinforcement learning methods typically suffer from difficulties in reward design, unstable training, high exploration costs, and significant risks associated with online trial and error. Especially in real-world business scenarios, incorrect policies can lead to user churn, making direct deployment challenging.

[0005] The third category comprises policy learning methods based on Human Feedback Reinforcement Learning (RLHF) or Direct Preference Optimization (DPO). These methods align the model's training by constructing preference pairs between "preferred answers" and "inferior answers." Compared to traditional RLHF, DPO eliminates the need to train a reward model, resulting in a more stable training process. However, existing DPO techniques are primarily applied to general text generation tasks, where preference data typically relies on subjective ranking of responses by human annotators, such as "answer A is better than answer B." This approach suffers from several problems: high preference annotation costs requiring significant human involvement; highly subjective annotation results with low consistency among annotators; difficulty in applying to long-term intent evolution tasks in real-world business dialogue scenarios; and inability to effectively utilize implicit behavioral feedback from massive historical interaction logs. Summary of the Invention

[0006] This invention provides a method and apparatus for optimizing dialogue strategies based on intent trajectory preference comparison, in order to solve the problems existing in the current dialogue strategy optimization process, such as strong reliance on human intervention, difficulty in obtaining preference data, poor model optimization stability, uninterpretable strategies, and inability of rules to continuously evolve.

[0007] A first aspect of this invention provides a dialogue strategy optimization method based on intent trajectory preference comparison, comprising: From real human-computer interaction logs, using the same or similar user intent states as anchors, and based on the change in the Euclidean distance between the user intent state vector and the target intent state vector in multi-turn dialogues, preference triples are constructed. The structure of the preference triples is: intent state - positive sample speech - negative sample speech. Using preference triples as training samples, the pre-trained dialogue strategy model is fine-tuned based on the intent trajectory weighted DPO loss function to obtain the optimized dialogue strategy model. The intent trajectory weighted DPO loss function is a direct preference optimization loss function that introduces intent trajectory quality weights. The intent trajectory quality weights are the difference in the magnitude of intent state changes between positive and negative sample pairs in the preference triples. Based on the optimized dialogue strategy model, effective strategies under specific intent states are extracted and transformed into structured rule entries containing rule ID, applicable conditions, recommended strategies, expected effects, supporting cases, and applicable scenarios. A multi-scenario rule base is constructed. Structured rule entries are added to the rule base after review and semantic expansion. The structured rule entries in the rule base are used as training samples for the next round of dialogue strategy model iteration and optimization.

[0008] In one optional implementation, preference triples are constructed from real human-computer interaction logs, using the same or similar user intent states as anchors, based on the change in the Euclidean distance between the user intent state vector and the target intent state vector during multi-turn dialogues. These triples include: Define a target intent state vector, which is a multidimensional normalized vector when the user intent evolves to the ideal final state; Extract the user intent state vector for each round of dialogue from real human-computer dialogue logs, and group the full set of user intent state vectors by DBSCAN clustering to obtain multiple user intent state clusters; For all interaction trajectories under the same user intent state cluster, calculate the Euclidean distance between the user intent state vector and the target intent state vector before and after each round of dialogue action. Based on the change in Euclidean distance after multiple rounds of dialogue, determine the preference direction and output the preference triplet.

[0009] In one optional implementation, user intent state vectors for each round of dialogue are extracted from real human-computer dialogue logs. These vectors are then grouped using DBSCAN clustering to obtain multiple user intent state clusters, including: Data cleaning is performed on real human-computer interaction logs to filter out invalid dialogues with fewer than 3 dialogue rounds, user exits midway, or low-quality responses. The remaining valid dialogues are then input into a pre-trained intent encoder to output a high-dimensional user intent state vector. Calculate the cosine similarity between user intent state vectors, and consider states with a cosine similarity higher than a first preset threshold as the same or similar intent states, thus obtaining multiple user intent state clusters. Select a center point or a random sample from each user intent state cluster as the anchor point of the cluster.

[0010] In one optional implementation, for all interaction trajectories under the same user intent state cluster, the Euclidean distance between the user intent state vector and the target intent state vector before and after each round of dialogue is calculated. The preference direction is determined based on the change in Euclidean distance after multiple rounds of dialogue, and a preference triplet is output, including: For each user intent state cluster, retrieve the corresponding verbal action from the anchor point, and filter out abnormal interaction trajectories such as user-initiated hanging up, chat jumps, and abnormal system replies. Calculate the Euclidean distance between the user intent state vector and the target intent state vector before and after each round of verbal action. If the Euclidean distance between the user's intent state vector and the target intent state vector after multiple rounds of dialogue is less than the Euclidean distance before the action, the dialogue action is classified as a positive sample dialogue. If the Euclidean distance between the user's intent state vector and the target intent state vector after multiple rounds of dialogue is greater than or equal to the Euclidean distance before the action, the dialogue action is classified as a negative sample dialogue. Under the same anchor point, only positive and negative sample pairs with the largest change are retained, and preference triples are output. The structure of the preference triples is: intention state - positive sample speech - negative sample speech.

[0011] In one optional implementation, preference triples are used as training samples, and the pre-trained dialogue policy model is fine-tuned based on the intent trajectory weighted DPO loss function to obtain an optimized dialogue policy model. The intent trajectory weighted DPO loss function is a direct preference optimization loss function that incorporates intent trajectory quality weights. The intent trajectory quality weights are the differences in the magnitude of intent state changes between positive and negative sample pairs in the preference triples, including: The full set of preference triples is divided into training, validation, and test sets. For each preference triple, the intent state quality weight is defined as the L2 norm of the difference between the magnitude of intent state change caused by positive sample verbal actions and the magnitude of intent state change caused by negative sample verbal actions. The standard direct preference optimization loss is multiplied by the intent state quality weight to obtain the intent trajectory weighted DPO loss function. The loss dialogue strategy model is then fine-tuned. The formula for calculating the intent trajectory weighted DPO loss function is as follows:

[0012] in, This is a positive sample script. For negative sample statements, weights , This represents the magnitude of change in intent caused by positive sample verbal actions. π represents the magnitude of change in intent caused by negative sample verbal actions. θ For the fine-tuned policy model, π ref As the reference model, β is a hyperparameter used to control π. θ With π ref The distribution difference, where σ is the Sigmoid function, maps the output to probability values ​​of 0-1; After fine-tuning the model on the training set, the optimized dialogue strategy model is obtained when the improvement in the generation probability of positive sample dialogues and the change in the average intent distance of the dialogues under the anchor intent state reach the preset optimization target on the validation set.

[0013] In one optional implementation, based on the optimized dialogue strategy model, effective strategies under specific intent states are extracted and transformed into structured rule entries containing rule IDs, applicable conditions, recommended strategies, expected effects, supporting cases, and applicable scenarios, including: The optimized model and the reference model are compared on the validation set. For each intent state anchor point, the probability increase of the model generating various predefined dialogue patterns in that state is statistically analyzed. The generation probability increase of which dialogue actions or combinations of dialogue actions is greater than the second preset threshold is analyzed. Dialogue actions or combinations of dialogue actions greater than the second preset threshold are identified as high-gain dialogues. High-gain dialogues are clustered to form reusable strategy templates. Based on the strategy templates, structured rule entries are generated. The fields of the structured rule entries include at least the rule ID, applicable intent state conditions, recommended dialogue strategy type, expected intent state change direction and magnitude, number of supported cases and confidence score, and cross-scenario transfer confidence. For each rule entry, the similarity of the intent state space between the source scenario and the target scenario is calculated as the cross-scenario transfer confidence.

[0014] In one optional implementation, a multi-scenario rule base is constructed. Structured rule entries are added to the rule base after review and semantic augmentation. These structured rule entries are then used as training samples for the next round of dialogue strategy model iteration and optimization, including: The structured rule entries are fully tested on the validation set. The rules to be reviewed are pre-reviewed based on the large language model. Rules that do not conform to business logic or are vaguely expressed are filtered out. The remaining rules are pushed to business experts in the corresponding scenarios for review. The review dimensions include business rationality, effect authenticity and explainability. The approved rules are used as seeds to generate semantically equivalent variant rules through a large language model. The variant rules meet the following requirements: consistent application conditions, semantic equivalence of recommendation strategies, and expected effect deviation less than the preset deviation. The variant rules are added to the rule base after semantic similarity verification. Based on the applicable conditions and recommendation strategies of the new rules, corresponding simulated dialogue samples are generated through a large language model. These simulated dialogue samples are then added to the training set, triggering a new round of model optimization.

[0015] In one alternative implementation, the method further includes: Rules with a cross-scenario migration confidence score greater than a preset confidence threshold are entered into the target scenario verification pool. Small-sample A / B testing is performed in the target scenario. If the rule brings about an improvement in the core metrics, it is officially adopted into the target scenario rule library; otherwise, it is rolled back. Each update to the rule base generates a new version number, records the update content, update time, reviewer, and full effect data. If some rules in the new version are not effective, only the problematic rules are withdrawn.

[0016] A second aspect of the present invention provides a dialogue strategy optimization apparatus based on intent trajectory preference comparison, comprising: The preference signal construction module is used to construct preference triples from real human-computer interaction logs, using the same or similar user intent states as anchors, and based on the change in the Euclidean distance between the user intent state vector and the target intent state vector in multi-turn dialogue. The structure of the preference triple is: intent state - positive sample speech - negative sample speech. The strategy model optimization module is used to fine-tune the pre-trained dialogue strategy model based on the intention trajectory weighted DPO loss function, using preference triples as training samples, to obtain the optimized dialogue strategy model. The intention trajectory weighted DPO loss function is a direct preference optimization loss function that introduces intention trajectory quality weights. The intention trajectory quality weights are the difference in the magnitude of the change in intention state between positive and negative sample pairs in the preference triples. The rule extraction module is used to extract the verbal actions or combinations of verbal actions that significantly increase the probability of generation under specific intent states from the optimized dialogue strategy model as effective strategies, and to transform the effective strategies into structured rule entries that include applicable conditions, recommended strategies, expected effects and confidence levels. The rule accumulation module is used to build a multi-scenario rule base. After reviewing and semantically expanding the structured rule entries, they are added to the rule base, and the structured rule entries in the rule base are used as training samples for the next round of dialogue strategy model iteration and optimization.

[0017] A third aspect of the present invention provides an electronic device, comprising: processor; Memory used to store processor-executable instructions; The processor is configured to invoke instructions stored in memory to execute the method described in the first aspect.

[0018] A fourth aspect of the present invention provides a computer-readable storage medium having computer program instructions stored thereon, wherein the computer program instructions, when executed by a processor, implement the method described in the first aspect.

[0019] The beneficial effects of this application are as follows: The present invention provides a dialogue strategy optimization method and apparatus based on intent trajectory preference comparison. Firstly, it breaks through the reliance on manually labeled data in traditional DPO methods by using the actual change in the Euclidean distance between the user's intent state vector and the target intent state vector as an objective preference signal to automatically construct preference triples. Secondly, it introduces intent trajectory quality weights, prioritizing sample pairs with significant effect differences, making small-scale real-data optimization convergence more stable. Thirdly, through a structured rule extraction mechanism, it transforms the model's implicit policy knowledge into interpretable rule entries containing applicable conditions, recommended strategies, expected effects, and confidence levels. These rules are then reviewed by experts and stored in a database. Furthermore, a large language model is used to generate variant rules to expand the coverage, forming a self-evolving closed loop. Fourthly, it includes a cross-scenario rule transfer mechanism, supporting the transfer of rules learned in scenarios such as retail sales guidance to other fields such as medical consultation and educational tutoring, enabling cross-scenario reuse of policy knowledge. Attached Figure Description

[0020] Figure 1 This is a flowchart illustrating the dialogue strategy optimization method based on intent trajectory preference comparison according to an embodiment of the present invention. Figure 2 This is a schematic diagram of the structure of the dialogue strategy optimization device based on intent trajectory preference comparison according to an embodiment of the present invention; Figure 3 This is a schematic diagram of the structure of an electronic device according to an embodiment of this application. Detailed Implementation

[0021] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0022] The technical solution of the present invention will be described in detail below with reference to specific embodiments. These specific embodiments can be combined with each other, and the same or similar concepts or processes may not be described again in some embodiments.

[0023] To address the issues of high preference annotation costs, poor interpretability of black-box policies, and low efficiency of rigid rule iteration in current task-oriented dialogue systems, this solution designs a complete closed loop from signal acquisition, model optimization, and rule accumulation. This transforms the optimization process of dialogue policies from a "black-box training problem" that relies on manual annotation into a "white-box rule discovery and evolution problem" based on objective environmental feedback (user intent evolution).

[0024] Specifically, by analyzing the actual evolution trajectory of user intent states, and using the change in distance between the intent state and the target state as an objective preference basis, preference triples are constructed to achieve preference signal acquisition without manual annotation. By tracking the changes in intent in subsequent rounds of dialogue, whether the user intent moves closer to the target direction is taken as the core optimization objective, enabling policy learning to directly reflect real business effects. A quality weight based on the magnitude of intent trajectory changes is introduced into the standard DPO loss function, assigning higher training weights to samples with more significant intent improvement effects, thereby improving model training stability and convergence efficiency. By analyzing the parameter changes of the model before and after optimization, high-value dialogue patterns are extracted and transformed into structured rule entries, making the model optimization results interpretable, auditable, and reusable. Furthermore, by introducing rule version records, online A / B testing, and an automatic rollback mechanism, the security and controllability of the rule base update process are improved, enhancing the system's adaptability to new scenarios and new user behaviors.

[0025] Figure 1 This is a flowchart illustrating the dialogue strategy optimization method based on intent trajectory preference comparison according to an embodiment of the present invention. Figure 1 As shown, in step S102, from real human-computer interaction logs, using the same or similar user intent states as anchors, and based on the change in the Euclidean distance between the user intent state vector and the target intent state vector in multi-turn dialogues, preference triples are constructed. The structure of the preference triples is intent state - positive sample speech - negative sample speech.

[0026] This disclosed embodiment uses the actual evolution direction of user intent in real interaction trajectories as the objective basis for preference, no longer relying on manual judgment of "which response is better", but directly observing whether user intent evolves in the direction of the target; whether user interest increases; whether user doubt decreases; whether user trust is enhanced; whether the user is close to completing the transaction / acceptance / achievement of the goal, which not only ensures the authenticity of preference signals, but also eliminates the cost of manual annotation.

[0027] In one optional implementation, the target intent state vector for each scenario is first predefined, which is the multi-dimensional normalized vector when the user intent evolves to the ideal final state. This can be generated in two ways: defining business rules, such as the target state for the "credit card application" intent being that all required fields (name, ID number, mobile phone number, work unit) are filled and the user confirms the application; or, performing K-means clustering on the final intent state vectors of all successfully completed interactions under the same intent, taking the cluster centers as the target vector, adapting to complex non-linear business scenarios. For example, the target intent state vector for a retail sales guide scenario can be defined as [purchase intention: 0.9, doubt index: 0.1, information completeness: 0.8], and the target intent state vector for a medical consultation scenario can be defined as [symptom description completeness: 0.9, willingness to seek medical treatment: 0.8, question index: 0.1].

[0028] The user intent state vector for each round of dialogue is extracted from real human-computer dialogue logs. DBSCAN clustering is used to group all user intent state vectors into multiple user intent state clusters. Specifically, real interaction logs can be cleaned to filter out invalid dialogues with fewer than 3 rounds, user exits midway, or extremely low-quality responses (such as those containing sensitive words or meaningless replies). The remaining valid dialogues are input into a pre-trained intent encoder, and the output user intent state vectors are stored in a vector database. The dimension of the user intent state vectors is consistent with the target intent state representation. It should be noted that dialogue logs can include multimodal signals such as user voice, emotion, operational behavior, and historical profiles. A multimodal intent fusion encoder can be used to improve the accuracy of intent state representation.

[0029] The cosine similarity between user intent state vectors is calculated. States with a cosine similarity higher than a first preset threshold are considered to be the same or similar intent states, resulting in multiple user intent state clusters. A center point or a random sample is selected from each cluster as the anchor point for that cluster. Here, "similarity" is not based on surface vocabulary but on deep intent vector representations. This means that even if users' wording is completely different (e.g., "I want to go to Shanghai" vs. "Book me a ticket to Shanghai"), as long as their core intent vectors are highly similar, they are considered to be under the same intent state anchor point, thus enabling cross-dialogue action comparison. The first preset threshold of 0.85 is an empirical value for multi-scenario verification: too low a threshold leads to inconsistent intents among candidate samples and high noise in the preference signal; too high a threshold leads to insufficient candidate samples and inability to construct effective triples. For the intent state S to be used to construct the preference triple... t Retrieve historical sessions from the vector database that are related to the current session. t S with a cosine similarity ≥ 0.85 t ′, corresponding to historical action A t ′ serves as a candidate positive and negative sample pool.

[0030] For all interaction trajectories under the same user intent state cluster, calculate the Euclidean distance between the user intent state vector and the target intent state vector before and after each round of dialogue. Determine the preference direction based on the change in Euclidean distance after multiple rounds of dialogue and output preference triples. The preference triple format is (S... t A t +,A t ), namely, intent state - positive sample verbal action - negative sample verbal action, the construction process includes: For each user intent state cluster, the corresponding dialogue action is retrieved from the anchor point. Abnormal interaction trajectories such as user-initiated hang-ups, casual chat transitions, and abnormal system responses are filtered out. The Euclidean distance between the user intent state vector and the target intent state vector before and after each dialogue action is calculated. If the Euclidean distance between the user intent state vector and the target intent state vector after multiple rounds of dialogue actions is less than the Euclidean distance before the action, the dialogue action is classified as a positive sample dialogue. If the Euclidean distance between the user intent state vector and the target intent state vector after multiple rounds of dialogue actions is greater than or equal to the Euclidean distance before the action, the dialogue action is classified as a negative sample dialogue.

[0031] In similar or highly similar dialogue scenarios, the system may execute different verbal actions (positive sample candidates) and (negative sample candidates). A comparison window N can be set, which can be dynamically adjusted based on the average number of dialogue rounds in the scenario. For short dialogue scenarios (such as e-commerce shopping guides), N can be set to 3-5, and for long dialogue scenarios (such as medical consultations), N can be set to 5-8. This avoids N being too short to reflect the long-term effect of the action, and N being too long to be affected by other irrelevant factors. If, after N rounds, the intention state is closer to the preset ideal target state than the initial state, for example, high interest and low doubt, then the action is considered effective and positive. Conversely, the action is considered ineffective, negative, or neutral. If only one type of positive or negative sample exists in the anchor state, then the anchor point is skipped; if the user actively terminates the dialogue midway, then the intention state at the time of termination is taken as S. t+N For example, for different verbal actions A under each similar anchor point. t Calculate the intention state S of the N rounds of dialogue following this action. t+N With the target state The Euclidean distance, combined with the current anchor point S t and Distance judgment preference: Positive Sample A t +Judgment condition:∣S t+N |2<|S t |2 indicates that the action drives the user's intention to evolve in the direction of the target, and is therefore judged as a positive sample; Negative sample A t Judgment condition: ∣S t+N |2≥|S t |2 indicates that the action did not promote or even hindered the evolution of the intention, and was judged as a negative sample.

[0032] Under the same anchor point, only the positive and negative sample pairs with the largest changes are retained, and the preference triples are output. That is, for each anchor point S t The positive sample with the best matching effect and the negative sample with the worst matching effect constitute (S) t , , Preference triples are used as inputs for subsequent DPO optimization.

[0033] In step S104, the preference triplet is used as a training sample, and the pre-trained dialogue strategy model is fine-tuned based on the intention trajectory weighted DPO loss function to obtain the optimized dialogue strategy model. The intention trajectory weighted DPO loss function is a direct preference optimization loss function that introduces intention trajectory quality weights. The intention trajectory quality weights are the difference in the magnitude of the change in intention state between positive and negative sample pairs in the preference triplet.

[0034] The standard Direct Preference Optimization (DPO) loss encourages the model to increase the probability of generating positive samples relative to the reference model more than it increases the probability of generating negative samples. It assigns the same weight to all preferred sample pairs. Low-quality sample pairs (with minimal difference between positive and negative effects) can introduce noise and reduce optimization efficiency, especially in small-scale real-world data scenarios where they are prone to overfitting.

[0035] To address the aforementioned issues, an intent trajectory quality weight is introduced on top of the standard DPO loss function to improve training stability in small sample scenarios.

[0036] Specifically, the entire set of preference triples is divided into training, validation, and test sets, and an intent state quality weight is defined for each preference triple. Intent State Quality Weight Let L2 norm be the difference between the magnitude of the change in intention state caused by the verbal actions in positive samples and the magnitude of the change in intention state caused by the verbal actions in negative samples. It quantifies the significance of the fact that "positive samples are much better than negative samples." That is, , The magnitude of change in intent caused by positive sample verbal actions =∣S t |2-|S t+N |2 (the amount by which the distance to the target state decreases after the action; a larger value indicates better performance of positive samples). The magnitude of change in intent caused by negative sample verbal actions |S t+N |2-|S t |2, the increase in distance to the target state after the action; a larger value indicates a worse performance on negative samples. Weights Normalization can be performed to avoid excessively high weights for extreme sample pairs, which could cause training oscillations.

[0037] The dialogue strategy model fine-tunes its loss by multiplying the standard direct preference optimization loss by the intent state quality weight as the intent trajectory weighted DPO loss function. The formula for calculating the intent trajectory weighted DPO loss function is as follows:

[0038] in, This is a positive sample script. For negative sample statements, weights , This represents the magnitude of change in intent caused by positive sample verbal actions. The weights represent the magnitude of the change in intent caused by negative sample verbal actions. Let π be the L2 norm of the difference between the magnitude of the change in intention state caused by the verbal actions in positive samples and the magnitude of the change in intention state caused by the verbal actions in negative samples. θ For the fine-tuned policy model, π ref As the reference model, β is a hyperparameter used to control π. θ With π ref The distribution differences are used to avoid overfitting the model to the training samples. The larger the value of β, the more significant the difference in action performance between positive and negative samples. This pair of samples carries a higher weight during training, and the model will prioritize learning from pairs with large performance differences, avoiding wasting computational resources on noisy samples with small performance differences. This results in more stable convergence on small-scale real-world data. σ is the Sigmoid function, mapping the output to probability values ​​between 0 and 1. In the early stages of training, a larger value for β (e.g., 0.1) is used to ensure model stability. Later, β is decreased with each training iteration (e.g., to 0.01) to expand the optimization space.

[0039] During training, samples with semantic similarity to positive samples but poor performance are prioritized as negative samples to improve training efficiency. Preference triples are reconstructed every two training epochs, incorporating newly generated effective samples to prevent overfitting to older data. The optimized dialogue strategy model is obtained when the increase in the generation probability of positive sample dialogues and the change in the average intent distance corresponding to the dialogues under the anchor intent state reach the preset optimization targets on the validation set. Training stops if the intent achievement rate (the percentage of dialogues where the final intent is achieved) does not improve for three consecutive epochs on the validation set to avoid overfitting. Batch preference triples are input into the model, weighted loss is calculated, and π is updated via backpropagation. θ It typically converges in 3-5 rounds. During training, efficient parameter fine-tuning techniques such as LoRA can be used to fine-tune only the low-rank adaptation layers of the model, significantly reducing computational costs.

[0040] The weighting mechanism can automatically filter low-value sample pairs and optimize samples with significant differences in performance, improving convergence speed on small-scale real data. The optimized model is a neural network, and its decision-making is not transparent. This embodiment transforms the optimized model into structured rules that are readable and interpretable by the business side, and stores them in a rule base to avoid the black-box problem of the model, while providing a foundation for subsequent rule expansion and migration.

[0041] In step S106, based on the optimized dialogue strategy model, effective strategies under specific intent states are extracted and transformed into structured rule entries containing rule ID, applicable conditions, recommended strategies, expected effects, supporting cases, and applicable scenarios.

[0042] In one optional implementation, the optimized model and the reference model are compared on a validation set. For each intent state anchor point, the probability increase of the model generating various predefined dialogue patterns in that state is statistically analyzed. It is then analyzed which dialogue actions or combinations of dialogue actions have a probability increase greater than a second preset threshold, and these dialogue actions or combinations of dialogue actions exceeding the second preset threshold are identified as high-gain dialogues. For example, if a certain type of dialogue pattern in S... t If the generation probability of the given text increases by ≥ the second preset threshold (15%), and the corresponding intent evolution effect (such as the proportion of intent distance reduction in subsequent N rounds) is ≥ 10% higher than the reference model, then the text pattern is determined to be an effective strategy and enters the rule extraction process.

[0043] High-gain dialogue phrases are clustered to form reusable strategy templates. For example, high-gain dialogue phrases are first labeled with business dimensions, such as objection handling, pain point resonance, social proof, and sales guidance. Then, semantic embedding models such as Sentence-BERT are used for clustering, grouping dialogue phrases with consistent logical structure and similar semantics into the same category. Based on the strategy templates, structured rule entries are generated. The fields of the structured rule entries include at least the rule ID, applicable intent state conditions, recommended dialogue strategy type, expected intent state change direction and magnitude, number of supported cases and confidence score, and cross-scenario transfer confidence. Among them, the similarity of the intent state space between the source scenario and the target scenario is calculated for each rule entry as the cross-scenario transfer confidence. For each cluster of dialogue samples, a reusable execution framework is extracted. For example, the template for objection handling + social proof can be standardized as follows: "① Empathize with the objection (avoid denying the user's feelings, such as 'I completely understand your concerns about the lack of after-sales service') → ② Provide targeted solutions (such as 'Our product supports 7-day no-reason return and exchange, and there is also dedicated customer service with 1-hour response) → ③ Provide supporting social proof (prioritize real user feedback in the same scenario + quantitative data, such as 'Last week, 320 users who were worried about after-sales service placed orders, just like you, and the current after-sales problem resolution rate is 94%)", while explicitly prohibiting the use of statements (such as vague promises like 'Our product is definitely fine').

[0044] The final output is structured rules. Each rule contains fields such as unique ID, applicable conditions, recommendation strategy, expected effect, supporting cases, applicable scenarios, confidence level, update date, and expiration conditions. It also comes with a corresponding script template library. The business side can directly match the corresponding rule and invoke the script based on the user intent recognition results, without needing to understand the underlying model logic.

[0045] In step S108, a multi-scenario rule base is constructed. Structured rule entries are added to the rule base after review and semantic expansion, and the structured rule entries in the rule base are used as training samples for the next round of dialogue strategy model iteration optimization.

[0046] In one optional implementation, structured rule entries are fully tested on a validation set. Pre-review of the rules to be reviewed is performed based on a large language model, filtering out rules that do not conform to business logic or are vaguely worded. The remaining rules are then pushed to business experts for review in the corresponding scenarios. Review dimensions include business rationality, effectiveness authenticity, and explainability. For example, rules with a confidence level ≥ 0.9 are automatically approved and directly added to the rule base; rules with a confidence level < 0.7 are directly filtered; rules with a confidence level ∈ [0.7, 0.9] are pushed to the expert review backend, where the backend automatically displays the rule's applicable conditions, supporting cases, expected effects, and corresponding real-world dialogue snippets. Experts only need to confirm / deny, and the review time for a single rule is < 10 seconds. Approved rules are stored in the structured rule base, with fields such as version number, reviewer, entry time, and online effect data. Unapproved rules are returned and trigger model re-analysis.

[0047] Approved rules are used as seeds, and semantically equivalent variant rules are generated through a large language model. Variant rules must meet the following criteria: consistent applicability, semantic equivalence of the recommended strategy, and expected performance deviation less than a preset deviation (e.g., expected performance deviation ≤ 10%). Variant rules are added to the rule base after semantic similarity verification (similarity ≥ 0.9 with the seed rule). For example, the rule recommending "use social proof" can generate variants such as "mention recent best-selling items," "show similar customer reviews," and "cite industry report data." These variants share the same core logic but offer richer expression options. Through embedding similarity calculation, duplicate variants with similarity ≥ 0.9 to the existing rule base are filtered out, ensuring rule diversity. This greatly expands the surface correspondence capability of the rule base, making strategy execution more flexible and natural.

[0048] Based on the applicable conditions and recommendation strategies of the new rules, corresponding simulated dialogue samples are generated through a large language model. These simulated dialogue samples are added to the training set, triggering a new round of model optimization. The new rules and their variations can serve as strong prior knowledge, reinforcing the construction of preference data. For example, based on the applicable conditions of the rules, user intent states for corresponding intervals are generated; compliant verbal actions are generated using the recommendation strategies corresponding to the rules; subsequent N rounds of dialogue and corresponding intent states are generated through a dialogue simulator, forming new preference triplet samples, which are added to the training pool. This process triggers a new round of preference triplet construction → ITDPO optimization → rule extraction, forming a complete self-evolving closed loop. The closed loop trigger threshold is configurable, such as triggering every 100 new interaction data points, weekly fixed triggering, or automatic triggering after rule base updates.

[0049] In one optional implementation, a migration confidence score is calculated for each rule to migrate from a source scenario (e.g., "mobile phone retail") to a target scenario (e.g., "laptop retail"). Rules with migration confidence scores greater than a pre-set confidence threshold are added to a target scenario validation pool. Small-sample A / B testing is performed in the target scenario. If the rule improves core metrics, it is officially adopted into the target scenario rule base; otherwise, it is rolled back. The core metric for migration confidence is the similarity of the intent state space distribution between the two scenarios. For example, this can be achieved by comparing the cluster centers and variances of intent vectors in historical data from both scenarios. Higher similarity results in higher migration confidence. High-confidence rules can be directly pre-embedded in the target scenario rule base and marked as pending validation. In the target scenario, their effectiveness is verified through small-sample validation (at least 20 real-world cases) online testing or A / B testing, i.e., whether they can also trigger positive intent evolution. Once validated, the rule is officially added to the target scenario rule base.

[0050] Each major update to the rule base, such as adding a batch of rules or revising major rules, creates a version snapshot, recording the update content, update time, reviewer, and full performance data. If some rules in the new version are ineffective, only the problematic rules are withdrawn. When a new version of the rule base goes live, it undergoes rigorous online A / B testing, comparing core business metrics such as task completion rate, user satisfaction, and conversion rate with the old version. If a new version experiences a significant decline in core metrics during the testing period or after launch, the system automatically or with administrator confirmation rolls back to the previous stable version to ensure service stability.

[0051] The following section uses a retail sales guide scenario as an example to explain in detail the implementation process of this embodiment: 1. Preference Construction: A user asks "How much is this phone?" At this point, the user's intent state St is [Intent: Inquiry, Interest: 0.7, Doubt Index: 0.3]. Two historical actions in similar states are retrieved from the logs: A1 (positive candidate) is "We have a promotion now, this one is 500 yuan off, and we're also giving away screen breakage insurance," and A2 (negative candidate) is "This one is 3999 yuan." In the subsequent 3 rounds (N=3) of dialogue, the user's interest level rises to 0.9 and their doubt index drops to 0.1, moving closer to the target state. The distance decreased from 0.3 to 0.05; the user interest level that triggered A2 remained at 0.7, the doubt index rose to 0.5, and the distance to the target state rose to 0.4. Therefore, A1 is a positive sample and A2 is a negative sample, forming a triple (St, A1, A2), with a weight wt = |0.25 0.1 | = 0.15.

[0052] 2. Model optimization: Input the triplet into ITDPO for training. The optimized model has a 20% higher probability of generating A1 type dialogue in the St state.

[0053] 3. Rule Extraction: The script pattern was determined to be a valid strategy and extracted as a rule: Rule ID is RETAIL_20240529_001, applicable conditions are "user intent is product inquiry, interest level ≥ 0.6, doubt index < 0.4", recommended strategy is to prioritize the use of the "promotional discount + value-added service commitment" combination script, expected effect is "purchase intention increase ≥ 0.2, doubt index decrease ≥ 0.15", supports 3 real cases, confidence level 0.87, applicable scenario is retail sales guide, confidence level is 0.78 when migrated to home appliance sales guide.

[0054] 4. Closed-loop iteration: After the rules are approved by experts, they are added to the database. LLM generates 3 variant rules and adds them to the database. At the same time, simulated dialogue samples are generated based on the rules to supplement the training set and trigger the next round of optimization.

[0055] As can be seen, the embodiments disclosed herein construct preferences entirely based on objective intent evolution results, requiring no manual annotation and free from subjective bias. The intent trajectory weighting mechanism focuses on high-value samples, improving the model's convergence speed. The self-evolving closed loop eliminates the need for regular manual rule review, supports rule transfer, and is suitable for dialogue systems with clear intent evolution goals, such as sales guidance, medical consultation, government services, intelligent customer service, and educational tutoring.

[0056] In summary, compared with existing technologies, this invention: First, it breaks through the dependence of traditional DPO methods on manually labeled data, using the actual change in the Euclidean distance between the user's intention state vector and the target intention state vector as an objective preference signal to automatically construct preference triples; Second, it introduces intention trajectory quality weights, prioritizing sample pairs with significant differences in effect, making small-scale real data optimization convergence more stable; Third, through a structured rule extraction mechanism, it transforms the model's implicit policy knowledge into interpretable rule entries containing applicable conditions, recommendation strategies, expected effects, and confidence levels. After expert review, these rules are stored in the database, and then variant rules are generated using a large language model to expand the coverage, forming a self-evolving closed loop; Fourth, it includes a cross-scenario rule transfer mechanism, supporting the transfer of rules learned in scenarios such as retail sales guidance to other fields such as medical consultation and educational tutoring, realizing the cross-scenario reuse of policy knowledge.

[0057] Figure 2 This is a schematic diagram of the dialogue strategy optimization device based on intent trajectory preference comparison according to an embodiment of the present invention. (Refer to...) Figure 2 As shown, the device includes: The preference signal construction module 210 is used to construct preference triples from real human-computer interaction logs, using the same or similar user intent states as anchors, based on the change in the Euclidean distance between the user intent state vector and the target intent state vector in multi-turn dialogues. The structure of the preference triples is: intent state - positive sample speech - negative sample speech.

[0058] The strategy model optimization module 220 is used to fine-tune the pre-trained dialogue strategy model based on the intention trajectory weighted DPO loss function, using preference triples as training samples, to obtain the optimized dialogue strategy model. The intention trajectory weighted DPO loss function is a direct preference optimization loss function that introduces intention trajectory quality weights. The intention trajectory quality weights are the difference in the magnitude of the change in intention state between positive and negative sample pairs in the preference triples.

[0059] The rule extraction module 230 is used to extract the verbal actions or combinations of verbal actions that significantly increase the probability of generation under specific intent states from the optimized dialogue strategy model as effective strategies, and to transform the effective strategies into structured rule entries that include applicable conditions, recommended strategies, expected effects and confidence levels.

[0060] The rule accumulation module 240 is used to build a multi-scenario rule base. After reviewing and semantically expanding the structured rule entries, it adds them to the rule base and uses the structured rule entries in the rule base as training samples for the next round of dialogue strategy model iteration optimization.

[0061] Figure 3 This is a schematic diagram of the structure of an electronic device according to an embodiment of this application. Figure 3As shown, the electronic device 310 includes: a storage unit 302 for storing executable instructions; and a processing unit 301 for connecting to the storage unit 302 to execute the executable instructions to complete the dialogue strategy optimization method based on intent trajectory preference comparison.

[0062] Those skilled in the art will understand that the illustration Figure 3 This is merely an example of electronic device 310 and does not constitute a limitation on electronic device 310. It may include more or fewer components than shown, or combine certain components, or different components. For example, electronic device 310 may also include input / output devices, network access devices, buses, etc.

[0063] The processing unit 301 (Central Processing Unit, CPU) can also be other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor can be a microprocessor, or the processing unit 301 can be any conventional processor. The processing unit 301 is the control center of the electronic device 310, connecting all parts of the electronic device 310 through various interfaces and lines.

[0064] Storage unit 302 can be used to store computer-readable instructions. Processing unit 301 implements various functions based on electronic device 310 by running or executing computer-readable instructions or modules stored in storage unit 302 and calling data stored in storage unit 302. Storage unit 302 mainly includes a program storage area and a data storage area. The program storage area can store the operating system, application programs required for at least one function (such as sound playback function, image playback function, etc.), etc.; the data storage area can store data created based on the use of electronic device 310, etc. In addition, storage unit 302 may include hard disk, memory, plug-in hard disk, smart media card (SMC), secure digital card (SD) card, flash card, at least one disk storage device, flash memory device, read-only memory (ROM), random access memory (RAM), or other non-volatile / volatile storage devices.

[0065] If the module integrated in the electronic device 310 is implemented as a software functional module and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, all or part of the processes in the methods of the above embodiments of the present invention can also be implemented by instructing related hardware through computer-readable instructions. The computer-readable instructions can be stored in a computer-readable storage medium, and when executed by a processor, the computer-readable instructions can implement the steps of the various method embodiments described above.

[0066] This embodiment also provides a computer-readable storage medium having a computer program stored thereon; the computer program is executed by a processor to implement the dialogue strategy optimization method based on intent trajectory preference comparison in other embodiments.

[0067] Those skilled in the art will understand that the terminology used in this invention is for the purpose of describing particular embodiments only and is not intended to be limiting of the invention. The singular forms “a,” “an,” and “the” as used in this invention and the appended claims are also intended to include the plural forms unless the context clearly indicates otherwise. It should also be understood that the term “and / or” as used herein refers to and includes any or all possible combinations of one or more of the associated listed items.

[0068] It should be understood that although the present invention may use terms such as first, second, third, etc., to describe various information, this information should not be limited to these terms. These terms are only used to distinguish information of the same type from one another. For example, first information may also be referred to as second information without departing from the scope of the present invention, and similarly, second information may also be referred to as first information. Depending on the context, the word "if" as used herein may be interpreted as "when," "when," or "in response to a determination."

[0069] Although preferred embodiments of this application have been described, those skilled in the art, upon learning the basic inventive concept, can make other changes and modifications to these embodiments. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments as well as all changes and modifications falling within the scope of this application.

[0070] Obviously, those skilled in the art can make various modifications and variations to this application without departing from the spirit and scope of this application. Therefore, if such modifications and variations fall within the scope of the claims of this application and their equivalents, this application also intends to include such modifications and variations.

Claims

1. A dialogue strategy optimization method based on intent trajectory preference comparison, characterized in that, include: From real human-computer interaction logs, using the same or similar user intent states as anchors, and based on the change in the Euclidean distance between the user intent state vector and the target intent state vector in multi-turn dialogues, preference triples are constructed. The structure of the preference triples is: intent state - positive sample speech - negative sample speech. Using the preference triples as training samples, the pre-trained dialogue strategy model is fine-tuned based on the intent trajectory weighted DPO loss function to obtain the optimized dialogue strategy model. The intent trajectory weighted DPO loss function is a direct preference optimization loss function that introduces intent trajectory quality weights. The intent trajectory quality weights are the difference in the magnitude of intent state changes between positive and negative sample pairs in the preference triples. Based on the optimized dialogue strategy model, effective strategies under specific intent states are extracted, and the effective strategies are transformed into structured rule entries containing rule ID, applicable conditions, recommended strategies, expected effects, supporting cases, and applicable scenarios. A multi-scenario rule base is constructed. The structured rule entries are added to the rule base after review and semantic expansion. The structured rule entries in the rule base are used as training samples for the next round of dialogue strategy model iteration optimization.

2. The dialogue strategy optimization method based on intent trajectory preference comparison according to claim 1, characterized in that, The method involves constructing preference triples from real human-computer interaction logs, using the same or similar user intent states as anchors, and based on the change in the Euclidean distance between the user intent state vector and the target intent state vector during multiple rounds of dialogue. This includes: Define a target intent state vector, which is a multidimensional normalized vector when the user intent evolves to the ideal final state; Extract the user intent state vector for each round of dialogue from real human-computer dialogue logs, and group the full set of user intent state vectors into multiple user intent state clusters through DBSCAN clustering. For all interaction trajectories under the same user intent state cluster, calculate the Euclidean distance between the user intent state vector and the target intent state vector before and after each round of dialogue action. Based on the change in the Euclidean distance after multiple rounds of dialogue, determine the preference direction and output the preference triplet.

3. The dialogue strategy optimization method based on intent trajectory preference comparison according to claim 2, characterized in that, The process involves extracting the user intent state vector for each round of dialogue from real human-computer dialogue logs, and then grouping the full set of user intent state vectors using DBSCAN clustering to obtain multiple user intent state clusters, including: Data cleaning is performed on real human-computer interaction logs to filter out invalid dialogues with fewer than 3 dialogue rounds, user exits midway, or low-quality responses. The remaining valid dialogues are then input into a pre-trained intent encoder to output a high-dimensional user intent state vector. Calculate the cosine similarity between user intent state vectors, and consider states with a cosine similarity higher than a first preset threshold as the same or similar intent states, thus obtaining multiple user intent state clusters. Select a center point or a random sample from each user intent state cluster as the anchor point of the cluster.

4. The dialogue strategy optimization method based on intent trajectory preference comparison according to claim 3, characterized in that, For all interaction trajectories under the same user intent state cluster, the Euclidean distance between the user intent state vector and the target intent state vector before and after each round of dialogue is calculated. Based on the change in the Euclidean distance after multiple rounds of dialogue, the preference direction is determined, and preference triples are output, including: For each user intent state cluster, retrieve the corresponding dialogue action from the anchor point, and filter out abnormal interaction trajectories such as user-initiated hanging up, chat jump, and abnormal system reply. Calculate the Euclidean distance between the user intent state vector before and after each round of dialogue action and the target intent state vector. If the Euclidean distance between the user's intent state vector and the target intent state vector is less than the Euclidean distance before the action after multiple rounds of dialogue, the dialogue action is classified as a positive sample dialogue. If the Euclidean distance between the user's intent state vector and the target intent state vector is greater than or equal to the Euclidean distance before the action, the dialogue action is classified as a negative sample dialogue. Under the same anchor point, only the positive and negative sample pairs with the largest change are retained, and the preference triples are output. The structure of the preference triples is: intention state - positive sample speech - negative sample speech.

5. The dialogue strategy optimization method based on intent trajectory preference comparison according to claim 1, characterized in that, The preferred triplet is used as a training sample, and the pre-trained dialogue policy model is fine-tuned based on the intent trajectory weighted DPO loss function to obtain an optimized dialogue policy model. The intent trajectory weighted DPO loss function is a direct preference optimization loss function that incorporates intent trajectory quality weights. The intent trajectory quality weights are the differences in the magnitude of intent state changes between positive and negative sample pairs in the preferred triplet, including: The full set of preference triples is divided into training, validation, and test sets. For each preference triple, the intent state quality weight is defined as the L2 norm of the difference between the magnitude of intent state change caused by positive sample verbal actions and the magnitude of intent state change caused by negative sample verbal actions. The standard direct preference optimization loss is multiplied by the intent state quality weight to obtain the intent trajectory weighted DPO loss function. The loss dialogue strategy model is then fine-tuned. The calculation formula of the intent trajectory weighted DPO loss function is as follows: in, This is a positive sample script. For negative sample statements, weights , This represents the magnitude of change in intent caused by positive sample verbal actions. π represents the magnitude of change in intent caused by negative sample verbal actions. θ For the fine-tuned policy model, π ref As the reference model, β is a hyperparameter used to control π. θ With π ref The distribution difference, where σ is the Sigmoid function, maps the output to probability values ​​of 0-1; After fine-tuning the model on the training set, the optimized dialogue strategy model is obtained when the increase in the generation probability of positive sample dialogues and the change in the average intent distance of the dialogues under the anchor intent state reach the preset optimization target on the validation set.

6. The dialogue strategy optimization method based on intent trajectory preference comparison according to claim 5, characterized in that, Based on the optimized dialogue strategy model, effective strategies under specific intent states are extracted, and these effective strategies are transformed into structured rule entries containing rule IDs, applicable conditions, recommended strategies, expected effects, supporting cases, and applicable scenarios, including: The optimized model and the reference model are compared on the validation set. For each intent state anchor point, the probability increase of the model generating various predefined dialogue patterns in that state is statistically analyzed. The generation probability increase of which dialogue actions or combinations of dialogue actions is greater than the second preset threshold is analyzed. Dialogue actions or combinations of dialogue actions greater than the second preset threshold are identified as high-gain dialogues. The high-gain dialogues are clustered to form reusable strategy templates. Based on the strategy templates, structured rule entries are generated. The fields of the structured rule entries include at least rule ID, applicable intent state conditions, recommended dialogue strategy type, expected intent state change direction and magnitude, number of supported cases and confidence score, and cross-scenario migration confidence. The similarity of the intent state space between the source scenario and the target scenario is calculated for each rule entry as the cross-scenario migration confidence.

7. The dialogue strategy optimization method based on intent trajectory preference comparison according to claim 6, characterized in that, The construction of a multi-scenario rule base involves adding the structured rule entries to the rule base after review and semantic expansion, and using these structured rule entries as training samples for the next round of dialogue strategy model iteration and optimization, including: The structured rule entries are fully tested on the validation set. The rules to be reviewed are pre-reviewed based on the large language model. Rules that do not conform to business logic or are vaguely expressed are filtered out. The remaining rules are pushed to business experts in the corresponding scenarios for review. The review dimensions include business rationality, effect authenticity and explainability. The approved rules are used as seeds to generate semantically equivalent variant rules through a large language model. The variant rules meet the requirements of consistent application conditions, semantic equivalence of recommendation strategies, and expected effect deviation less than the preset deviation. The variant rules are added to the rule base after semantic similarity verification. Based on the applicable conditions and recommendation strategies of the new rules, corresponding simulated dialogue samples are generated through a large language model. These simulated dialogue samples are then added to the training set, triggering a new round of model optimization.

8. The dialogue strategy optimization method based on intent trajectory preference comparison according to claim 7, characterized in that, The method further includes: Rules with a cross-scenario migration confidence level greater than a preset confidence threshold are entered into the target scenario verification pool. Small-sample A / B testing is performed in the target scenario. If the rule brings about an improvement in the core indicators, it is officially adopted into the target scenario rule library; otherwise, it is rolled back. Each update to the rule base generates a new version number, records the update content, update time, reviewer, and full effect data. If some rules in the new version are not effective, only the problematic rules are withdrawn.

9. A dialogue strategy optimization device based on intent trajectory preference comparison, used to implement the method as described in any one of claims 1-8, characterized in that, include: The preference signal construction module is used to construct preference triples from real human-computer interaction logs, using the same or similar user intent states as anchors, and based on the change in the Euclidean distance between the user intent state vector and the target intent state vector in multi-turn dialogues. The structure of the preference triples is: intent state - positive sample speech - negative sample speech. The strategy model optimization module is used to fine-tune the pre-trained dialogue strategy model based on the intention trajectory weighted DPO loss function, using the preference triple as training samples, to obtain the optimized dialogue strategy model. The intention trajectory weighted DPO loss function is a direct preference optimization loss function that introduces intention trajectory quality weights. The intention trajectory quality weights are the difference in the magnitude of the change in intention state between positive and negative sample pairs in the preference triple. The rule extraction module is used to extract the verbal actions or combinations of verbal actions that significantly increase the probability of generation under specific intention state conditions from the optimized dialogue strategy model as effective strategies, and to transform the effective strategies into structured rule entries containing applicable conditions, recommended strategies, expected effects and confidence levels. The rule accumulation module is used to build a multi-scenario rule base. The structured rule entries are added to the rule base after being reviewed and semantically expanded, and the structured rule entries in the rule base are used as training samples for the next round of dialogue strategy model iteration optimization.

10. A computer-readable storage medium having computer program instructions stored thereon, characterized in that, When the computer program instructions are executed by the processor, they implement the method described in any one of claims 1 to 8.

Images