A method and device for modeling an empathetic dialogue based on a strategy mechanism

Abstract

This invention discloses a policy-based empathic dialogue modeling method and apparatus. It relates to the fields of dialogue systems and empathy generation technology. It addresses the problems of incomplete policy coverage, lack of policy support in the reasoning process, and low data annotation quality in existing empathic dialogue models, and solves the defects of uncontrollable model generation results, weak interpretability, and poor empathic effect. This invention can generalize on various open-source large language models and consistently outperforms existing methods in both automatic and manual evaluation metrics, effectively improving the generation quality of empathic dialogues. It can be applied to fields such as intelligent customer service and emotional companion robots.

Description

Technical Field

[0001] This invention relates to the fields of dialogue systems and empathy generation technology, and in particular to a strategy-based empathy dialogue modeling method and apparatus. Background Technology

[0002] Empathic dialogue is an important component of social interaction, requiring the identification of emotional states and the generation of understanding, acceptance, and supportive responses. From a psychological perspective, empathic dialogue is a complex, multi-stage cognitive and decision-making behavior, characterized by strategic and context-sensitive features, and belongs to a complex task beyond surface-level text generation.

[0003] Early research in this area often employed external common sense knowledge or sentiment dictionaries to implicitly enhance dialogue models. For example, sentiment common sense graphs and common sense knowledge selection mechanisms were integrated into neural network architectures to provide auxiliary information for the generation of emotion-aware text. However, these methods did not explicitly model the reasoning and decision-making processes, making it difficult to reveal the intrinsic mechanism by which emotion understanding influences response generation.

[0004] Current research utilizes large-scale language models as its technological vehicle, constructing explicit intermediate reasoning processes through thought chain prompts. Some studies enhance the interpretability of the generation process by guiding the model to output intermediate reasoning evidence. However, thought chain-based methods have limitations; the reasoning process lacks a systematic strategic framework. Existing strategies are mostly geared towards specific application scenarios and negative emotions, resulting in superficial responses that fail to cover the full emotional spectrum and lack higher-order cognitive strategies. Corresponding structured methods also suffer from insufficient reasoning depth and poor strategy coherence.

[0005] The existing technical solutions have three key shortcomings: First, the empathy strategy has a limited coverage and lacks a complete system that adapts to multiple emotions and includes higher-order cognitive strategies, making it difficult to support standardized decision-making; Second, it lacks a multi-stage reasoning mechanism that matches the task and fails to model the dialogue process as an explicit multi-stage reasoning process, which restricts the model's deep reasoning ability and lowers the quality of response generation; Third, there is a lack of strategy-aware supervision information and the training data lacks high-quality labeled content that matches the empathy strategy and reasoning process. Summary of the Invention

[0006] This invention provides a strategy-based empathic dialogue modeling method and apparatus to address the problems of incomplete strategy coverage, lack of strategy support in the reasoning process, and low data annotation quality in existing empathic dialogue models, thereby improving the shortcomings of uncontrollable model generation results, weak interpretability, and poor empathic effect.

[0007] This invention provides a strategy-based empathic dialogue modeling method, comprising:

[0008] A set of empathy strategies covering the entire emotional spectrum is established, and the set of empathy strategies is divided according to cognitive complexity to obtain a structured empathy strategy guidance framework;

[0009] Using a large language model as an automatic annotation expert, four types of structured labels are generated based on annotation prompts and dialogue history context. The dialogue history context and the four types of structured labels are combined into a single sample, and multiple single samples are combined into an initial full-scale annotation dataset.

[0010] At least three large language models are selected as evaluation models, and the quality of individual samples included in the initial fully labeled dataset is scored. The reliability weight of each evaluation model and the final quality score of an individual sample are determined based on the Spearman correlation coefficient. Based on the final quality score, a set number of samples are selected from the initial fully labeled dataset to form a high-quality candidate dataset.

[0011] Based on the empirical frequency and difficulty weight of the empathy strategy for each individual sample in the initial fully labeled dataset, the target sampling ratio is determined; for the high-quality candidate dataset, the largest effective subset size that satisfies the policy distribution deviation constraint is determined from the high-quality candidate dataset by binary search; based on the largest effective subset size and the target sampling ratio, the high-quality candidate dataset is refined by hierarchical random sampling to obtain a refined training dataset;

[0012] The remaining subsets of the refined training dataset and the initial full-label dataset are used as the training set. With minimizing the negative log-likelihood as the training objective, the basic large language model is supervised and fine-tuned to obtain an intermediate model.

[0013] Based on the high-quality candidate dataset, a near-end policy optimization algorithm is used with a multi-dimensional reward function as the optimization objective to perform secondary optimization on the intermediate model, resulting in the final empathic dialogue model.

[0014] This invention provides a policy-based empathic dialogue modeling device, comprising:

[0015] A unit is established to build a set of empathy strategies covering the entire emotional spectrum. The set of empathy strategies is divided according to cognitive complexity to obtain a structured empathy strategy guidance framework.

[0016] The annotation unit is used to use the large language model as an automatic annotation expert, generate four types of structured labels based on annotation prompts and dialogue history context, and combine the dialogue history context and the four types of structured labels into a single sample. Multiple single samples are used to form the initial full-scale annotation dataset.

[0017] An evaluation unit is used to select at least three large language models as evaluation models, score the quality of individual samples included in the initial fully labeled dataset, determine the reliability weight of each evaluation model and the final quality score of an individual sample based on the Spearman correlation coefficient, and select a set number of samples from the initial fully labeled dataset to form a high-quality candidate dataset based on the final quality score.

[0018] The first obtaining unit is used to determine the target sampling ratio based on the empirical frequency and difficulty weight of the empathy strategy of each individual sample in the initial full-label dataset; for the high-quality candidate dataset, the unit uses binary search to determine the size of the largest effective subset that satisfies the policy distribution deviation constraint; based on the size of the largest effective subset and the target sampling ratio, the unit uses hierarchical random sampling to refine the high-quality candidate dataset to obtain a refined training dataset.

[0019] The second obtaining unit is used to take the remaining subset of the refined training dataset and the initial full-label dataset as the training set, and to perform supervised fine-tuning on the basic large language model with the goal of minimizing the negative log-likelihood, to obtain an intermediate model.

[0020] The third obtaining unit is used to perform secondary optimization on the intermediate model based on the high-quality candidate dataset, using a near-end strategy optimization algorithm and a multi-dimensional reward function as the optimization objective, to obtain the final empathic dialogue model.

[0021] This invention provides a computer device, which includes a memory and a processor. The memory stores a computer program, and when the computer program is executed by the processor, the processor performs any of the above-described empathic dialogue modeling methods based on policy mechanisms.

[0022] This invention provides a computer-readable storage medium storing a computer program, which, when executed by a processor, causes the processor to perform any of the above-described empathic dialogue modeling methods based on a policy mechanism.

[0023] This invention provides a strategy-based empathic dialogue modeling method and apparatus. The method employs a data filtering step involving multi-model consistency scoring and strategy-aware sampling, combined with automatic annotation using a large language model. It filters data using a formula that integrates reliability weights from multiple evaluation models and penalties for standard deviation of scores, eliminating low-quality samples and improving the consistency and reliability of training data. This addresses the problems of low efficiency and strong subjectivity inherent in traditional manual filtering, and compensates for insufficient strategy-aware supervision, laying the foundation for model training. Secondly, it constructs a comprehensive empathic strategy system covering the entire emotional spectrum and different cognitive difficulties. Combining a strategy-guided structured output architecture and a step-by-step cognitive chain reasoning process, the output sequence is defined as a multi-module structured form, simulating the human empathic reasoning process and achieving end-to-end interpretability. This solves the problems of incomplete strategy coverage and empathic misalignment in existing methods, improving the rationality and empathic ability of responses. Finally, it adopts a two-stage training paradigm of supervised fine-tuning and proximal policy optimization, combined with multi-objective reinforcement learning, enabling the model to perform well in emotion alignment and strategy execution, generating responses that are relevant, diverse, and fluent. Experimental verification shows that the method provided in this invention can generalize on a variety of open-source large language models, and its automatic and manual evaluation metrics are superior to existing methods. It significantly improves the quality of empathic dialogue generation and can be widely applied in fields such as intelligent customer service, emotional companionship robots, and psychological support dialogue systems, with extremely high application value. Attached Figure Description

[0024] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0025] Figure 1 A schematic diagram of a strategy-based empathic dialogue modeling method provided in an embodiment of the present invention;

[0026] Figure 2 A schematic diagram of the single-modal text reasoning process of STRIDE-ED (Strategy-Grounded Reasoning Interpretability Deep Empathetic Dialogue) provided for embodiments of the present invention;

[0027] Figure 3 This is a schematic diagram of the overall architecture of STRIDE-ED provided in an embodiment of the present invention;

[0028] Figure 4 This is a schematic diagram of a strategy-based empathic dialogue modeling device provided in an embodiment of the present invention. Detailed Implementation

[0029] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0030] In practical applications, addressing the shortcomings mentioned in the background technology involves multiple adjustments: First, the dynamic and complex nature of implicit relationships makes traditional rules or static structures difficult to adapt; implicit associations within a single modality require dynamic inference, and cross-modal relationships need to consider heterogeneity. Second, the continuous expression of emotional semantics and cross-modal alignment require meticulous design, preserving emotional category information while encoding contextual relevance, and balancing commonalities and differences between modalities. Finally, there is a contradiction between the efficiency of fusing multimodal heterogeneous data and computational cognitive complexity; dynamically constructing hybrid relationship graphs requires consideration of multi-level interactions, otherwise it can easily lead to a surge in resource consumption and a decrease in generalization. These challenges collectively constrain the ability of existing methods to perform deep emotional analysis and generate empathetic responses in complex dialogue scenarios.

[0031] To address the aforementioned problems, embodiments of the present invention provide a strategy-based empathic dialogue modeling method. Figure 1 This is a schematic diagram of a policy-based empathic dialogue modeling method provided in an embodiment of the present invention; the following is in conjunction with... Figure 1 Taking an example, this invention provides a detailed description of an empathic dialogue modeling method based on a policy mechanism. Figure 1 As shown, the method includes the following steps:

[0032] Step 101: Establish a set of empathy strategies covering the entire emotional spectrum, and divide the set of empathy strategies according to cognitive complexity to obtain a structured empathy strategy guidance framework;

[0033] Step 102: Using the large language model as an automatic annotation expert, four types of structured labels are generated based on the annotation prompts and the dialogue history context. The dialogue history context and the four types of structured labels are combined into a single sample, and multiple single samples are combined into an initial full-scale annotation dataset.

[0034] Step 103: Select at least three large language models as evaluation models, and score the quality of individual samples included in the initial full-scale labeled dataset; determine the reliability weight of each evaluation model and the final quality score of an individual sample based on the Spearman correlation coefficient; and select a set number of samples from the initial full-scale labeled dataset to form a high-quality candidate dataset based on the final quality score.

[0035] Step 104: Determine the target sampling ratio based on the empirical frequency and difficulty weight of the empathy strategy for each individual sample in the initial full-label dataset; for the high-quality candidate dataset, determine the size of the largest effective subset that satisfies the policy distribution deviation constraint from the high-quality candidate dataset through binary search; based on the size of the largest effective subset and the target sampling ratio, refine the high-quality candidate dataset by using stratified random sampling to obtain a refined training dataset.

[0036] Step 105: Using the remaining subset of the refined training dataset and the initial full-label dataset as the training set, and with the goal of minimizing the negative log-likelihood, the basic large language model is supervised and fine-tuned to obtain an intermediate model.

[0037] Step 106: Based on the high-quality candidate dataset, the intermediate model is optimized a second time using a near-end strategy optimization algorithm with a multi-dimensional reward function as the optimization objective, to obtain the final empathic dialogue model.

[0038] It should be noted that the subject of this method is a computer device, server, or artificial intelligence training and inference system with data processing and model running capabilities.

[0039] This method involves a total of five large language models: a large language model as the automatic annotation expert, three large language modules for quality scoring, and a large language model as the foundational model. The large language model as the automatic annotation expert generates four types of structured labels—scene summary, target sentiment, empathy strategy, and execution action—based on the dialogue history context. The large language model for quality scoring is used to assign quality scores of 1-5 to the samples included in the initial fully annotated dataset. The large language model as the foundational model is the original base model that is fine-tuned.

[0040] In step 101, an empathy strategy set covering the entire emotional spectrum is established, and the empathy strategy set is divided according to cognitive complexity to obtain a structured empathy strategy guidance framework.

[0041] In this embodiment of the invention, the full emotional spectrum includes positive emotions, neutral emotions, and negative emotions; that is, the empathy strategy set consists of positive emotions, neutral emotions, and negative emotions. Furthermore, since cognitive complexity is divided into three difficulty levels: difficulty I, difficulty II, and difficulty III, and the empathy strategy set contains 14 types of empathy strategies, the empathy strategies are divided according to cognitive complexity, that is, the 14 types of empathy strategies included in the empathy strategy set are divided into three difficulty levels according to the three difficulty levels.

[0042] Specifically, difficulty level I is the basic level strategy, which mainly includes gratitude prompts, restatements or interpretations, other, and information provision; difficulty level II is the advanced level strategy, which includes neutral validation, positive reinforcement, exploring behavior and intentions, self-disclosure, affirmation and comfort; difficulty level III is the high-level strategy, which includes emotional reflection, cognitive restructuring, exploring feelings and emotions, exploring thoughts and cognitions, and providing advice.

[0043] It should be noted that Difficulty Level I is the basic level and is mainly applicable to simple emotional scenarios. Specifically, gratitude prompts guide the speaker to focus on positive experiences or supportive aspects of life, cultivating positive emotional cognition; restatement or paraphrasing restates the speaker's core points, demonstrating listening and understanding; other indicates response types that do not fit other predefined strategies; and information provision provides objective facts or relevant knowledge to help the speaker understand the situation or make decisions.

[0044] Difficulty Level II is the advanced level, requiring basic emotional understanding and interaction skills. Specifically, neutral validation affirms the legitimacy of neutral or low-intensity emotions and discourages emotional reinforcement; positive reinforcement highlights the speaker's strengths, efforts, or constructive behaviors to strengthen confidence and motivation; exploring behaviors and intentions involves clarifying the speaker's behavior, plans, and underlying intentions through targeted questioning; self-disclosure involves sharing limited but relevant personal experiences to build empathy and mutual trust; and affirmation and comfort acknowledge the speaker's feelings and provide emotional support and comfort.

[0045] Difficulty Level III represents advanced cognitive level, suitable for complex emotional scenarios. Specifically, Emotional Reflection represents identifying and expressing the speaker's implicit but unexpressed emotions; Cognitive Restructuring represents providing alternative perspectives on the dilemma while respecting the original emotions; Exploring Feelings and Emotions represents encouraging the speaker to express their emotional experiences more deeply through open-ended questions; Exploring Thoughts and Cognition represents exploring the speaker's beliefs, interpretations, and thought processes; and Providing Suggestions represents offering practical and actionable suggestions tailored to the speaker's needs.

[0046] In step 102, multi-turn interactive dialogue history data between the user and the dialogue agent is obtained as the dialogue history context. In this embodiment of the invention, the multi-turn interactive dialogue history data does not exceed 10 rounds. The dialogue history context can be represented as:

[0047] (1)

[0048] in, Indicates the historical context of the dialogue, consisting of multiple rounds. Composed of (utterance, number of rounds / statements); Indicates the first A single turn of speech in a dialogue is the first in the historical context of the dialogue. The content of messages spoken by an individual user or conversation agent; This represents the dialogue round index, with a maximum of 10 rounds. ; This indicates the current dialogue turn in which a response is pending. This indicates the previous speaker in the current round. ; ; Indicates the first Round of speech The Middle Each token is the smallest text processing unit in a large language model. ; Indicates the first Round of speech The total number of word segments, that is, the total number of word units obtained after word segmentation in this round of speech.

[0049] In this embodiment of the invention, the length of a single-round word segmentation unit sequence does not exceed 2048, that is, for all... ,satisfy Ensure that the input conforms to the context window requirements of the large language model.

[0050] It should be noted that, in the embodiments of the present invention, utterance refers to a single-turn dialogue speech / single sentence, which refers to a complete natural language text spoken by one party (user or dialogue agent) in an interaction in the context of the dialogue history, and is the basic unit of dialogue.

[0051] Furthermore, a large language model with strong reasoning capabilities is selected as the automatic annotation expert to standardize the annotation of the acquired dialogue history context. Before annotation, annotation prompts need to be designed. In this embodiment of the invention, the annotation prompts include: setting the annotation expert role for the large language model, clarifying the structured annotation task, explaining the dialogue history input format, defining the meaning and specifications of four types of annotation fields: scene summary, target emotion, empathy strategy, and execution action; providing an optional list of emotion tags and empathy strategy tags; and specifying a unified structured output format to ensure that the automatic annotation results are standardized, consistent, and parsable.

[0052] Specifically, the aforementioned dialogue history context is input into a large language model that serves as an automated annotation expert. The automated annotation expert processes the dialogue context in batches based on standardized annotation prompts, generating four types of structured tags: scene summary, target emotion, empathy strategy, and execution action.

[0053] Among them, the annotation prompts are standardized instruction templates that drive the automatic annotation process. The core consists of five modules to ensure the standardization, consistency and accuracy of annotation:

[0054] 1) Task definition and objective description: Clarify the role of professional annotation experts in the large language model, require all annotations to be 100% consistent with the dialogue history context, and prohibit fabrication of information or deviation from the original text, and set a general outline for the annotation process;

[0055] 2) Input format description: Specify the input as the dialogue history context, limit the information boundaries of the annotation, and require the model to generate annotations only based on the input content and not introduce external information;

[0056] 3) Label Field Definition and Requirements: Clearly define the specific rules and label libraries for the four types of labels, providing executable labeling standards for the model. Specifically:

[0057] Scene summary: Use 1-2 sentences to objectively summarize the core dialogue scene, event background, and user needs, without subjective evaluation;

[0058] Target emotion: Select one tag from a library of 32 preset emotion tags that best matches the user's state, and output only the tag name;

[0059] Empathy Strategy: Select one optimal response strategy from the 14 categories of empathy strategies with difficulty levels, and output only the strategy name;

[0060] Execution Actions: Generate 2-3 specific executable response actions based on the selected strategy to clarify the direction of subsequent responses;

[0061] 4) Output format constraints: The large language model is forced to output structured results with four fixed labels: scene summary, target emotion, empathy strategy, and execution action. Labels cannot be modified or fields are omitted to ensure that the annotation results can be directly parsed.

[0062] 5) Constraints and verification rules: Supplement the labeling quality assurance requirements, including that labels must be selected from the preset list, the labeling logic must be consistent, and no blanks are allowed, to reduce model illusions and provide a reliable basis for subsequent quality scoring.

[0063] It should be noted that the dialogue history context includes four types of structured labels: scene summary, target emotion, empathy strategy, and execution action. These are the labeled attribute information of each dialogue sample. In this embodiment of the invention, each dialogue history context and the corresponding four types of structured labels together form a complete labeled single sample. Furthermore, all labeled single samples are summarized to form the initial full labeled dataset.

[0064] It should be noted that the large language model mentioned above as an automatic annotation expert can be DeepSeek-R1. In this embodiment of the invention, the specific type of large language model is not specifically limited.

[0065] In step 103, at least three large language models are selected as evaluation models to score the quality of individual samples included in the initial full-scale labeled dataset. The higher the score, the stronger the semantic coherence and alignment of the scene summary, target emotion, empathy strategy, execution action and dialogue history context labeled in step 102. That is, the labeling results in step 102 are more accurate, reasonable and fit the dialogue context.

[0066] Before scoring the quality of each sample in the initial full-scale labeled dataset, evaluation prompts need to be designed. Specifically, three large language models are used as neutral evaluation models. Based on the dialogue history context, they judge the semantic coherence and alignment of the four types of structured labels labeled in step 102 with the dialogue history context, and output a score of 1-5 (1=very poor, 5=excellent).

[0067] Furthermore, the reliability weight of each evaluation model and the final quality score of a single sample are determined based on the Spearman correlation coefficient. Based on the final quality score, a set number of samples are selected from the initial fully labeled dataset to form a high-quality candidate dataset.

[0068] Specifically, the reliability weights of the evaluation model are determined using the following formula:

[0069] (2)

[0070] The final quality score is determined using the following formula:

[0071] (3)

[0072] in, Indicates the first Reliability weights for each evaluation model Indicating the evaluation model and Spearman correlation coefficient of the rating vector Represents the set of evaluation models. Indicates the first Each evaluation model scores a vector for all individual samples. Indicates the first Each evaluation model scores a vector for all individual samples. , Indicates the evaluation model number. Represents a set All evaluation models All are established. Indicates sample The final quality score, This represents a single sample in the initial fully labeled dataset. This indicates the total number of evaluation models. Indicates the first Reliability weights for each evaluation model This indicates that each evaluation model is applied to a single sample. The standard deviation of the rating Indicates the first Evaluation Model For a single sample The rating, This represents the standard deviation of the scores for each evaluation model. This represents the penalty hyperparameter, with a value of 0.1.

[0073] It should be noted that, based on the final quality score, all samples included in the initial fully labeled dataset are sorted in descending order, and a predetermined number of samples are selected from them to form a high-quality candidate dataset. The specific value of the predetermined number is not limited here.

[0074] It should be noted that the three large language models here can be DeepSeek-R1, Qwen3-8B, and Llama-3.1-8B-Instruct, respectively. In this embodiment of the invention, the specific type of the large language model is not specifically limited.

[0075] In step 104, the target sampling ratio is determined based on the empirical frequency and difficulty weight of the empathy strategy for each individual sample in the initial full-scale labeled dataset. The empirical frequency of the empathy strategy represents the proportion of occurrence of each of the 14 types of strategies in the initial full-scale labeled dataset. In the difficulty weight, the strategy weight for difficulty I is set to 1, the strategy weight for difficulty II is set to 2, and the strategy weight for difficulty III is set to 3.

[0076] Specifically, the target sampling ratio is determined by the following formula:

[0077] (4)

[0078] in, Indicates the first The target sampling ratio of the empathy strategy Indicates the first The empirical frequency of this strategy in the initial fully labeled dataset. Indicates the first The difficulty weight of each strategy Indicates the first The frequency of experience with this strategy Indicates the first The difficulty weight of each strategy This represents the total number of empathy strategies. This represents the complete set of empathy strategies.

[0079] Furthermore, for the high-quality candidate dataset, the size of the largest effective subset that satisfies the policy distribution deviation constraint is determined from the high-quality candidate dataset by binary search.

[0080] In practical applications, a subset is selected from the high-quality candidate dataset. This subset includes 14 empathy policy distributions that are as close as possible to the previously calculated target sampling ratio. This is achieved through binary search, continuously trying subsets of different sizes. The smallest subset can be zero, and the largest can be equal to the number of high-quality candidate datasets. For subsets of different sizes, stratified random sampling is performed according to the target ratio. The deviation between the policy distribution of each subset and the target sampling ratio is calculated. When the deviation meets the policy distribution deviation constraint (assuming the deviation is ≤5%), the size of this subset is determined as the maximum effective subset size.

[0081] It should be noted that, in the embodiments of the present invention, determining the size of the maximum effective subset can make full use of existing data and ensure training stability while ensuring accurate alignment of policy distribution. At the same time, it avoids the problems of insufficient sample quantity for high-difficulty policies (Difficulty III) and excessive proportion of low-difficulty policy samples, thereby achieving balanced policy distribution and improving generation effect.

[0082] Furthermore, based on the maximum effective subset size and the target sampling ratio, stratified random sampling is used to refine the high-quality candidate dataset to obtain a refined training dataset.

[0083] Specifically, the fixed size of the refined training dataset can be determined based on the number of samples corresponding to the size of the maximum effective subset. Once the fixed size of the refined training dataset is determined, stratified random sampling is used to select samples from the high-quality candidate dataset that are equal to the size of the maximum effective subset, and these samples are used to form the refined training dataset.

[0084] In this embodiment of the invention, the size of the maximum effective subset is determined by binary search, which can ensure that the policy distribution is accurately aligned with the target ratio and obtain the maximum amount of usable data. Then, based on this size, a refined training set is constructed by hierarchical random sampling, which can balance the policy distribution and difficulty, avoid the low-difficulty policy samples from having too high a proportion and overdoing the training process, and ensure that the model can learn various empathy policies in a balanced way.

[0085] In step 105, a training set is constructed with the goal of minimizing the negative log-likelihood, and the basic large language model is fine-tuned under supervision to obtain an intermediate model.

[0086] The training set consists of two parts: the refined training set obtained in step 104 and the remaining subset of the initial full-label dataset. This dataset is based on the initial full-label dataset. Specifically, all samples included in the refined training dataset are first removed from the initial full-label dataset, and then the empathy strategy and action labels in the remaining samples are removed, leaving only the scene summary and target emotion fields, thus forming the remaining subset of the initial full-label dataset.

[0087] In this invention, the two parts of the training set are mixed in a 1:4 ratio.

[0088] Furthermore, DeepSeek-7B-Chat was selected as the base model, and the training parameters were set as follows: the initial learning rate was set to... The batch size is 16, the training epochs are 20, a learning rate scheduler is used, and the weight decay coefficient is 0.01. The training objective is to minimize the negative log-likelihood, and supervised fine-tuning is performed on the basic large language model to obtain an intermediate model. The training objective is determined by the following formula:

[0089] (5)

[0090] in, This represents the loss function for supervised fine-tuning. This represents the total number of training samples. Indicates the number of the training sample. Represents logarithmic operations. Indicates in model parameters Below, the historical context of the dialogue Generate target sequence The conditional probability, This represents all trainable parameters of a large language model. This represents the target output sequence of the model.

[0091] In step 106, based on the high-quality candidate dataset, the intermediate model is optimized a second time using a near-end policy optimization algorithm with a multi-dimensional reward function as the optimization objective, to obtain the final empathic dialogue model.

[0092] This step is the multi-objective reinforcement learning optimization stage. Its purpose is to make the model more accurately align with emotions, empathy strategies and output formats based on the supervised fine-tuning in step 105, so as to obtain the final usable model.

[0093] Specifically, based on a high-quality candidate dataset, a proximal strategy optimization is used to perform secondary optimization on the intermediate model. During training, a multi-dimensional reward function composed of format, emotion, and strategy guides the model update, so that the model output is accurately aligned with the target format, emotional state, and empathy strategy, and finally a final model that can be directly used for empathic dialogue generation is obtained.

[0094] The multi-dimensional reward function is determined by the following formula:

[0095] (6)

[0096] (7)

[0097] (8)

[0098] (9)

[0099] in, Indicates a formatted reward. Indicates emotional reward, Indicates strategy reward, This indicates the final reward.

[0100] Furthermore, after inputting the dialogue history context that conforms to the format requirements of step 102, the final model generates a response according to the following process:

[0101] The system analyzes the dialogue history context to generate a scene summary; infers the speaker's target emotion based on the scene summary and dialogue history context; selects an appropriate empathy strategy based on the target emotion and the cognitive complexity of the dialogue scene; plans specific actions to express the target emotion; and generates a natural and coherent empathic response based on the actions.

[0102] The final model provided in this embodiment of the invention adopts a step-by-step cognitive chain reasoning mechanism of scene summarization, emotion recognition, strategy inference, action planning and response generation. It achieves explicit reasoning through structured guidance of dialogue context, scene summary, target emotion, empathy strategy, execution action and final response, ensuring that the response generation process is interpretable and the strategy execution is traceable.

[0103] To more clearly illustrate this invention, the following section uses a strategy-based empathic dialogue deep reasoning modeling method as an example to describe the method in detail:

[0104] Basic information included in the implementation examples:

[0105] Base model: DeepSeek-7B-Chat.

[0106] Dataset: EMPATHETICDIALOGUES (empathic dialogue open source dataset), containing approximately 25,000 dialogues and 32 emotions.

[0107] Total number of strategies: 14, divided into 3 difficulty levels (I:4, II:5, III:5).

[0108] Hyperparameter: Penalty coefficient Learning rate = 1e-4, batch size = 16.

[0109] Step 201: Construct an empathy strategy set, in which the full emotional spectrum includes positive emotions, neutral emotions, and negative emotions; the empathy strategy set contains 14 types of empathy strategies; and the cognitive complexity is divided into three difficulty levels: difficulty I, difficulty II, and difficulty III.

[0110] Specifically, Difficulty I (weight 1) includes gratitude prompts, restatements, other, and information provision (4 categories in total); Difficulty II (weight 2) includes neutral verification, positive reinforcement, exploring behavioral intentions, self-disclosure, and affirmation and comfort (5 categories in total); Difficulty III (weight 3) includes emotional reflection, cognitive restructuring, exploring feelings and emotions, exploring thoughts and cognition, and providing suggestions (5 categories in total).

[0111] Step 202, input the dialogue history context from the EMPATHETICDIALOGUES test dialogue, such as: "I made an appointment with a friend to watch a movie today, but he stood me up at the last minute, so I watched the whole movie by myself."

[0112] The expression for the dialogue history context is: C={u1}, and the sequence length of the word segmentation unit is ≤2048.

[0113] Step 203: Designate DeepSeek-R1 as an automatic annotation expert and input the following into DeepSeek-R1: "I made plans to see a movie with a friend today, but he stood me up at the last minute, so I watched the whole movie by myself."

[0114] DeepSeek-R1 outputs four types of annotations:

[0115] Scenario Summary: A user is stood up by a friend and watches a movie alone, feeling disappointed.

[0116] Target emotion: Disappointment.

[0117] Empathy strategy: affirmation and comfort (Difficulty II).

[0118] Action to take: Acknowledge the user's grievances and express understanding.

[0119] DeepSeek-R1 outputs the initial fully annotated dataset ED-CSA-all (Empathetic Dialogue-Context-Strategy-Action (all samples)) (approximately 25,000 samples).

[0120] Step 204: DeepSeek-R1, Qwen3, and Llama-3.1 are selected as evaluation models. The scoring rules for the evaluation models are as follows: scores range from 1 to 5 points, based on the alignment of scene summary, target emotion, empathy strategy, and the performance of actions with the dialogue history context. Assume the score for a sample x is: Then, according to the weight formula of the evaluation model, Assuming correlation: , , Similarly, , .

[0121] According to the sample quality score formula ,in, ; ; ; ; .

[0122] Based on the initial fully labeled dataset ED-CSA-all The samples are sorted in descending order, and the first 12,000 samples are selected to form a high-quality candidate set ED-CSA-12k.

[0123] Step 205, using the target sampling ratio formula Statistical analysis of strategies for the initial fully labeled dataset ED-CSA-all, including... Representation Strategy Frequency in ED-CSA-all This represents the difficulty weights (I:1, II:2, III:3); for example, a difficulty III strategy. , =3, then we have The calculated proportions of the 14 policy objectives are as follows: .

[0124] Using ED-CSA-12k (Empathetic Dialogue-Context-Strategy-Action (12k high-quality samples), a high-quality candidate set of 12k empathetic dialogue datasets for context, strategy, and action) as the object, we use binary search to determine the size of the largest effective subset that satisfies the policy distribution bias constraint from ED-CSA-12k, where the policy distribution bias is ≤5% and the maximum effective size is 5000.

[0125] According to the target sampling ratio formula 5000 samples were extracted from ED-CSA-12k to obtain the refined training set ED-CSA-5k (Empathetic Dialogue-Context-Strategy-Action (5k refined samples)).

[0126] Step 206: The remaining portions of ED-CSA-5k and ED-CSA-all are combined into a training set. The remaining portions of ED-CSA-all represent the subset of the initial fully labeled dataset after removing ED-CSA-5k samples, deleting the empathy strategies and action labels from the remaining samples, and finally retaining the scene summary and target emotion. The training set includes 20,000 samples.

[0127] Through the training target formula The training objective is to minimize the negative log-likelihood. Supervised fine-tuning is then performed on the basic large language model to obtain an intermediate model. The training parameters are set as follows: learning rate... The batch size was 16, and the number of training epochs was 3. Finally, the intermediate model STRIDE-ED-M-SFT (Strategy-Grounded Reasoning Interpretability Deep Empathetic Dialogue Model (Supervised Fine-Tuning)) was obtained.

[0128] Step 207: Based on the ED-CSA-12k dataset, a near-end policy optimization algorithm is used, with a multi-dimensional reward function as the optimization objective, to perform secondary optimization on the STRIDE-ED-M-SFT model, thereby obtaining the final model STRIDE-ED-M (Strategy-Grounded Reasoning Interpretability Deep Empathetic Dialogue Model). The multi-dimensional reward functions are as follows: =1 (correct format), otherwise 0; =1 (correct emotion), otherwise 0; =1 (strategy correct), otherwise 0. Optimal reward is .

[0129] Step 208: Input the dialogue history context from Step 202, "I made plans to see a movie with a friend today, but he stood me up at the last minute, and I watched the whole movie by myself." into STRIDE-ED-M. Its reasoning process includes: generating a scene summary → identifying the target emotion: disappointment → selecting empathy strategies: affirmation and comfort → planning the action: acknowledging the emotion and expressing understanding.

[0130] The reply was: "Being stood up is really upsetting, and watching a movie alone is so lonely. I totally understand how you feel."

[0131] This invention conducted extensive experiments on the EMPATHETICDIALOGUES dataset, which contains approximately 25,000 dialogue scenarios covering 32 emotion labels. Three baseline models—traditional architecture, external knowledge enhancement methods, and reflexive decision integration methods—were compared. Automatic evaluation metrics included BLEU (Bilingual Evaluation Understudy) (n=1-4), Accemo (Accuracy of Emotion), Distinct (Distinctness of Generated Responses) (n=1-2), and PPL (Perplexity). Human evaluation used A / B testing to score empathy, relevance, and fluency.

[0132] Table 1 shows the experimental results on the EMPATHETICDIALOGUES dataset.

[0133]

[0134] Table 2 Statistics of the EMPATHETICDIALOGUES dataset

[0135]

[0136] Table 3. Main hyperparameters of STRIDE-ED-M

[0137]

[0138] Table 4 Performance of STRIDE-ED-M in ablation experiments

[0139]

[0140] Table 5. Performance of STRIDE-ED-M in the training set size sensitivity experiment.

[0141]

[0142] Table 6. Performance of STRIDE-ED-M in various backbone LLM (Large Language Model) experiments

[0143]

[0144] Table 7 Performance of STRIDE-ED compared to baseline model in experiments.

[0145]

[0146] Table 1 compares the performance of STRIDE-ED-M with other baseline models on the EMPATHETICDIALOGUES dataset. In Table 1, B-1, B-2, B-3, and B-4 correspond to BLEU-1, BLEU-2, BLEU-3, and BLEU-4 respectively, and their Chinese meanings are 1-gram BLEU value, 2-gram BLEU value, 3-gram BLEU value, and 4-gram BLEU value respectively; Acc_emo corresponds to Accuracy of Emotion; D-1 and D-2 correspond to Distinct-1 and Distinct-2 respectively, and their Chinese meanings are 1-gram uniqueness and 2-gram uniqueness respectively; Transformer represents the basic encoder-decoder architecture; MoEL is the English abbreviation of Mixture of Empathy Listeners, and its Chinese meaning is the Mixture of Empathy Listeners model; EmpDG is the English abbreviation of Empathetic Dialogue Generation, and its Chinese meaning is the Empathetic Dialogue Generation model; KEMP is the English abbreviation of Knowledge-Enhanced Memory-Enhanced Prompting, and its Chinese meaning is the Knowledge-Enhanced Memory Prompting model; DCKS is the English abbreviation of Deep Contextual Knowledge Selection, and its Chinese meaning is the Deep Contextual Knowledge Selection model; E-CORE is the English abbreviation of Emotional Context Representation Learning, and its Chinese meaning is the Emotional Context Representation Learning model; Emp-USIR is the English abbreviation of Empathetic Unsupervised Iterative Refinement, and its Chinese meaning is the Empathetic Unsupervised Iterative Optimization model; CAB is the English abbreviation of Context-Aware Bert, and its Chinese meaning is the Context-Aware BERT model; IAMM is the English abbreviation of Interactive Attention Mixture Model, and its Chinese meaning is the Interactive Attention Mixture Model; ReflectDiffu is the English abbreviation of Reflective Diffusion Model, and its Chinese meaning is the Reflective Diffusion Model; Ours is the English abbreviation of Our Proposed Model, and its Chinese meaning is the proposed model STRIDE-ED-M in this invention.

[0147] Table 2 provides the statistical information of the EMPATHETICDIALOGUES dataset. In Table 2, Train corresponds to the English full name Training Set, and its Chinese meaning is the training set; Val corresponds to the English full name Validation Set, and its Chinese meaning is the validation set; test corresponds to the English full name Test Set, and its Chinese meaning is the test set.

[0148] Table 3 lists the main hyperparameter settings of STRIDE-ED-M on the EMPATHETICDIALOGUES dataset. In Table 3, Epoch refers to the number of training rounds / iteration cycles, Batch size refers to the batch size, and Warmup ratio refers to the learning rate warm-up ratio.

[0149] Table 4 shows the ablation results of STRIDE-ED on EMPATHETICDIALOGUES. Among them, w / o oe stands for without Emotion; w / o sum stands for without Summary; w / o stra stands for without Strategy; w / o CoT stands for without Context; w / o R.&S. stands for without Rating and Sampling; w / o S. stands for without Sampling; and w / o PPO stands for without Proximal Policy Optimization.

[0150] Table 5 shows the results of the STRIDE-ED model in the training set size sensitivity experiment. When the training data was reduced to 1 / 8, the model metrics remained relatively robust; however, when the data was further reduced to 1 / 16, BLEU-1 and Acc plummeted, and PPL showed an abnormal decrease, indicating severe overfitting and a significant decline in task performance. Under zero-data conditions, although the model exhibited high generation fluency, it was unable to complete the empathy task.

[0151] Table 6 shows that STRIDE-ED maintains strong and stable performance on various backbone LLMs, verifying its architecture independence and parameter efficiency. Specifically, Qwen3-0.6B stands for Qiantong Wen3-0.6 Billion Model; Qwen3-4B stands for Qiantong Wen3-4 Billion Model; Qwen3-4B-In. stands for QiantongWen3-4 Billion-Inference Model; LLama3.2-3B stands for Large Language Model AI3.2-3 Billion Model; and GLM-Z1-9B stands for General Language Model-Z1-9 Billion. The 9 billion parameter version; as shown in Table 6, the Acc_em of the Qwen series steadily increases with scale; B-1 does not increase monotonically, indicating that the lightweight model can also achieve higher fluency after being optimized by this framework. LLama3.2-3B has the best sentiment accuracy and diversity.

[0152] Table 7 presents the A / B human evaluation results of STRIDE-ED and the baseline model in three categories: empathy, relevance (Rel.), and fluency (Flu.). Compared to MoEL, EmpDG, and CAB, STRIDE-ED achieved significantly higher win rates in both empathy and relevance dimensions, while maintaining or slightly outperforming fluency, indicating that its generated responses are more empathetic, context-relevant, and readable. In the table, Comparison refers to the comparison object; Aspects refers to the evaluation dimensions; Win refers to the percentage of wins; Lose refers to the percentage of losses; Tie refers to the percentage of draws; Emp. stands for Empathy; Relevance refers to Relevance; and Fluency refers to Fluency.

[0153] Experimental results show that the method provided in this invention consistently outperforms baseline models across all automatic evaluation metrics. Compared to the strongest baseline, ReflectDiffu, BLEU-1 shows an improvement of 4.5%, BLEU-2 an improvement of 8.7%, BLEU-3 an improvement of 35.9%, and BLEU-4 an improvement of 31.5%, with an 18.1% improvement in emotional accuracy. Distinct-1 and Distinct-2 are significantly improved, and perplexity is greatly reduced, indicating that the response has significant advantages in relevance, emotional controllability, diversity, and fluency. In manual evaluation, this invention leads in win rate across the three dimensions of empathy, relevance, and fluency when compared with baseline models such as MOEL, EmpDG, and CAB, verifying the high quality of the generated response.

[0154] Ablation experiments show that removing any component—such as step-by-step cognitive chain reasoning, data scoring and sampling, or PPO training—leads to a decrease in model performance across one or more dimensions, confirming the importance of each module. Training set size analysis reveals that the model's performance stabilizes before the data volume is reduced to 1 / 8; further reduction results in a sharp performance drop, indicating that sufficient high-quality data is crucial for model adaptation. Backbone model analysis demonstrates that the framework of this invention is adaptable to various open-source LLMs, achieving excellent performance on models of different architectures and sizes, and exhibiting good generalization ability.

[0155] Figure 2 This is a schematic diagram of the STRIDE-ED single-modal text reasoning process provided in an embodiment of the present invention, as shown below. Figure 2 As shown, in S21, the user's dialogue history context in the form of speech is input, such as "I originally planned to go to the movies with a group of friends. But they all canceled, so now I have to go alone.", as the original input for reasoning; in S22, based on the input dialogue history context, a scene summary is generated, such as "This speaker originally planned to go to the movies with friends, but now, because the reservations were canceled, he has to go alone," completing the structured extraction of the core information of the dialogue; in S23, combining the scene summary and the complete dialogue history context, the user's target emotion is identified, such as the identified emotion of "loneliness," providing a basis for subsequent strategy selection; in S24, based on the identified target emotion, an appropriate empathy strategy is selected from the preset set of empathy strategies, such as the "cognitive reconstruction" strategy selected for the emotion of "loneliness"; in S25, based on the selected empathy strategy, the corresponding execution action is planned, such as "Using the method of cognitive reconstruction, shifting attention to the potential value of the movie itself...", and finally generating a natural and fluent empathic response based on the execution action: "If this movie is worth watching, you definitely won't miss it." Figure 2 The provided method is used to perform empathic reasoning on the historical context of single-turn or multi-turn dialogues and generate empathic responses.

[0156] Figure 3 The schematic diagram of the overall architecture of STRIDE-ED provided in the embodiment of the present invention is divided into two parts: the data chain and the training chain. The data chain part includes S31-S33, and the training chain part includes S34-S35.

[0157] Specifically, in S31, a large language model is used as an automatic annotation expert to annotate the original dialogue data and generate an initial full-scale annotated dataset containing four types of structured labels: scene summary, target emotion, empathy strategy, and execution action, which is the dataset ED-CSA-all shown in the figure.

[0158] In S32, at least three independent large language models are selected as evaluation models to score the quality of the labeled data. The consistency score between the judging models is calculated by statistical methods and the corresponding weights are determined. After obtaining the weighted average score, the models are sorted in descending order to select high-quality candidate datasets, namely the dataset ED-CSA-12k shown in the figure.

[0159] In S33, the target sampling ratio is determined based on the empirical frequency and difficulty weight of the empathy strategy. The size of the largest effective subset that satisfies the policy distribution deviation constraint is determined by binary search. Stratified random sampling is used to select high-quality candidate datasets, so that there are more samples with high difficulty and fewer samples with low difficulty, resulting in a refined training dataset, namely the dataset ED-CSA-5k shown in the figure.

[0160] In S34, the model is trained in two stages: the first stage uses the refined training dataset and the remaining subset of the initial full-label dataset as the training set, with the goal of minimizing the negative log-likelihood, and supervises the fine-tuning of the basic large language model to enable the model to learn the output content and obtain the intermediate model; the second stage is based on a high-quality candidate dataset and optimizes the training model through a proximal strategy to improve the accuracy of format, sentiment and policy, and complete the secondary optimization.

[0161] In S35, after two stages of training, the final empathic dialogue model is obtained, namely the STRIDE-ED-DeepSeek-7B model shown in the figure.

[0162] Based on the same inventive concept, this invention provides an empathic dialogue modeling device based on a strategy mechanism. Since the principle by which this device solves the technical problem is similar to that of an empathic dialogue modeling method based on a strategy mechanism, the implementation of this device can refer to the implementation of the method, and the repeated parts will not be described again.

[0163] like Figure 4 As shown, the device includes a creation unit 401, a labeling unit 402, an evaluation unit 403, a first obtaining unit 404, a second obtaining unit 405, and a third obtaining unit 406.

[0164] Unit 401 is established to build a set of empathy strategies covering the entire emotional spectrum. The set of empathy strategies is divided according to cognitive complexity to obtain a structured empathy strategy guidance framework.

[0165] The annotation unit 402 is used to use the large language model as an automatic annotation expert, generate four types of structured labels based on annotation prompts and dialogue history context, and combine the dialogue history context and the four types of structured labels into a single sample, and make up the initial full annotation dataset from multiple single samples.

[0166] Evaluation unit 403 is used to select at least three large language models as evaluation models, score the quality of individual samples included in the initial full-label dataset, determine the reliability weight of each evaluation model and the final quality score of an individual sample based on the Spearman correlation coefficient, and select a set number of samples from the initial full-label dataset to form a high-quality candidate dataset based on the final quality score.

[0167] The first obtaining unit 404 is used to determine the target sampling ratio based on the empirical frequency and difficulty weight of the empathy strategy of each individual sample in the initial full-label dataset; for the high-quality candidate dataset, the largest effective subset size that satisfies the strategy distribution deviation constraint is determined from the high-quality candidate dataset by binary search; based on the largest effective subset size and the target sampling ratio, the high-quality candidate dataset is selected by hierarchical random sampling to obtain a refined training dataset.

[0168] The second obtaining unit 405 is used to take the remaining subset of the refined training dataset and the initial full-label dataset as the training set, and to perform supervised fine-tuning on the basic large language model with the goal of minimizing the negative log-likelihood, to obtain an intermediate model.

[0169] The third unit 406 is used to perform secondary optimization on the intermediate model based on the high-quality candidate dataset, using a near-end strategy optimization algorithm and a multi-dimensional reward function as the optimization objective, to obtain the final empathic dialogue model.

[0170] It should be understood that the units included in the above-described empathic dialogue modeling device based on a strategy mechanism are merely a logical division based on the functions implemented by the device. In practical applications, the above units can be superimposed or split. Furthermore, the functions implemented by the empathic dialogue modeling device based on a strategy mechanism provided in this embodiment correspond one-to-one with the empathic dialogue modeling method based on a strategy mechanism provided in the above-described embodiment. The more detailed processing flow implemented by this device has been described in detail in the first embodiment of the method described above, and will not be described in detail here.

[0171] Another embodiment of the present invention provides a computer device, the computer device including: a processor and a scene database; the scene database is used to store computer program code, the computer program code including computer instructions; when the processor executes the computer instructions, the electronic device executes the various steps of the empathic dialogue modeling method based on a policy mechanism shown in the above method embodiment.

[0172] Another embodiment of the present invention provides a computer-readable storage medium storing computer instructions that, when executed on a computer device, cause the computer device to perform the various steps of the policy-based empathic dialogue modeling method shown in the above method embodiment.

[0173] Obviously, those skilled in the art can make various modifications and variations to this invention without departing from its spirit and scope. Therefore, if these modifications and variations fall within the scope of the claims of this invention and their equivalents, this invention also intends to include these modifications and variations.

Claims

1. A strategy-based empathic dialogue modeling method, characterized in that, include: A set of empathy strategies covering the entire emotional spectrum is established, and the set of empathy strategies is divided according to cognitive complexity to obtain a structured empathy strategy guidance framework; Using a large language model as an automatic annotation expert, four types of structured labels are generated based on annotation prompts and dialogue history context. The dialogue history context and the four types of structured labels are combined into a single sample, and multiple single samples are combined into an initial full-scale annotation dataset. At least three large language models are selected as evaluation models, and the quality of individual samples included in the initial fully labeled dataset is scored. The reliability weight of each evaluation model and the final quality score of an individual sample are determined based on the Spearman correlation coefficient. Based on the final quality score, a set number of samples are selected from the initial fully labeled dataset to form a high-quality candidate dataset. The target sampling ratio is determined based on the empirical frequency and difficulty weight of the empathy strategy for each individual sample in the initial full-label dataset. For the high-quality candidate dataset, the size of the largest effective subset that satisfies the policy distribution deviation constraint is determined from the high-quality candidate dataset by binary search; Based on the maximum effective subset size and the target sampling ratio, hierarchical random sampling is used to refine the high-quality candidate dataset to obtain a refined training dataset. The remaining subsets of the refined training dataset and the initial full-label dataset are used as the training set. With minimizing the negative log-likelihood as the training objective, the basic large language model is supervised and fine-tuned to obtain an intermediate model. Based on the high-quality candidate dataset, a near-end policy optimization algorithm is used with a multi-dimensional reward function as the optimization objective to perform secondary optimization on the intermediate model, resulting in the final empathic dialogue model.

2. The method as described in claim 1, characterized in that, The full emotional spectrum includes positive emotions, neutral emotions, and negative emotions; the set of empathy strategies includes 14 types of empathy strategies; the cognitive complexity is divided into three difficulty levels: difficulty I, difficulty II, and difficulty III. The difficulty level I is a basic strategy, including gratitude prompts, restatements or interpretations, and information provision; the difficulty level II is an advanced strategy, including neutral validation, positive reinforcement, exploring behaviors and intentions, self-disclosure, affirmation and comfort; and the difficulty level III is a high-level strategy, including emotional reflection, cognitive restructuring, exploring feelings and emotions, exploring thoughts and cognitions, and providing advice.

3. The method as described in claim 1, characterized in that, The four types of structured tags include scene summary, target emotion, empathy strategy, and performance action; The dialogue history context represents the multi-turn interactive dialogue history data between the user and the dialogue agent, and the dialogue history context is represented by the following formula: in, Indicates the context of the dialogue history. Indicates the first A single round of speaking in a round-robin dialogue. ; This indicates the previous speaker in the current round. ; Indicates the first Round of speech The Middle Each word segmentation unit , Indicates the first Round of speech The total number of word segmentation units.

4. The method as described in claim 1, characterized in that, The reliability weights of the evaluation model are determined by the following formula: The final quality score of a single sample is determined by the following formula: in, Indicates the first Reliability weights for each evaluation model Indicating the evaluation model and Spearman correlation coefficient of the rating vector Represents the set of evaluation models. Indicates the first Evaluation Model For the score vector of all individual samples, Indicates the first Evaluation Model For the score vector of all individual samples, , Indicates the evaluation model number. Indicates sample The final quality score, This represents a single sample in the initial fully labeled dataset. This indicates the total number of evaluation models. Indicates the first Reliability weights for each evaluation model This indicates that each evaluation model is applied to a single sample. The standard deviation of the rating Indicates the first Evaluation Model For a single sample The rating, This represents the penalty hyperparameter, with a value of 0.

1.

5. The method as described in claim 1, characterized in that, The target sampling ratio is determined by the following formula: in, Indicates the first The target sampling ratio of the empathy strategy Indicates the first The frequency of experience with this strategy Indicates the first The difficulty weight of each strategy Indicates the first The frequency of experience with this strategy Indicates the first The difficulty weight of each strategy This represents the total number of empathy strategies. This represents the complete set of empathy strategies.

6. The method as described in claim 1, characterized in that, The training objective is determined by the following formula: in, This represents the loss function for supervised fine-tuning. This represents the total number of training samples. Indicates the number of the training sample. Represents logarithmic operations. Indicates in model parameters Below, the historical context of the dialogue Generate target sequence The conditional probability, This represents all trainable parameters of a large language model. This represents the target output sequence of the model. Indicates the context of the dialogue history.

7. The method as described in claim 1, characterized in that, The multi-dimensional reward function is determined by the following formula: in, Indicates a formatted reward. Indicates emotional reward, Indicates strategy reward, This indicates the final reward.

8. A policy-based empathic dialogue modeling device, characterized in that, include: A unit is established to build a set of empathy strategies covering the entire emotional spectrum. The set of empathy strategies is divided according to cognitive complexity to obtain a structured empathy strategy guidance framework. The annotation unit is used to use the large language model as an automatic annotation expert, generate four types of structured labels based on annotation prompts and dialogue history context, and combine the dialogue history context and the four types of structured labels into a single sample. Multiple single samples are used to form the initial full-scale annotation dataset. An evaluation unit is used to select at least three large language models as evaluation models and to score the quality of individual samples included in the initial full-scale labeled dataset. The reliability weight of each evaluation model and the final quality score of a single sample are determined based on the Spearman correlation coefficient. Based on the final quality score, a set number of samples are selected from the initial fully labeled dataset to form a high-quality candidate dataset. The first obtaining unit is used to determine the target sampling ratio based on the empirical frequency and difficulty weight of the empathy strategy for each individual sample in the initial full-scale labeled dataset; For the high-quality candidate dataset, the size of the largest effective subset that satisfies the policy distribution deviation constraint is determined from the high-quality candidate dataset by binary search; Based on the maximum effective subset size and the target sampling ratio, hierarchical random sampling is used to refine the high-quality candidate dataset to obtain a refined training dataset. The second obtaining unit is used to take the remaining subset of the refined training dataset and the initial full-label dataset as the training set, and to perform supervised fine-tuning on the basic large language model with the goal of minimizing the negative log-likelihood, to obtain an intermediate model. The third obtaining unit is used to perform secondary optimization on the intermediate model based on the high-quality candidate dataset, using a near-end strategy optimization algorithm and a multi-dimensional reward function as the optimization objective, to obtain the final empathic dialogue model.

9. A computer-readable storage medium, characterized in that, The system stores a computer program that, when executed by a processor, causes the processor to perform the policy-based empathic dialogue modeling method as described in any one of claims 1-7.

10. A computer device, characterized in that, The computer device includes a memory and a processor. The memory stores a computer program, which, when executed by the processor, causes the processor to perform the empathic dialogue modeling method based on any one of claims 1-7.

Images