Large language model high-context co-empathy enhanced reply generation method and device

Abstract

The application provides a large language model high-context co-empathy enhanced reply generation method and device, which has the following characteristics: step S1, constructing a co-empathy dialogue data set according to existing co-empathy dialogue data, the co-empathy dialogue data set including multiple task subsets of different difficulties; step S2, training an existing large language model according to the co-empathy dialogue data set to obtain a co-empathy reply model and a reinforcement learning model; step S3, inputting a specified dialogue into the co-empathy reply model to obtain an initial reply; and step S4, inputting the initial reply into the reinforcement learning model to obtain a co-empathy reply. In summary, the method can generate a co-empathy reply that is more in line with emotions, roles and personalities in a high-context culture.

Description

Technical Field

[0001] This invention belongs to the field of machine high-order cognition, specifically relating to a method and apparatus for generating responses with enhanced high-context empathy in large language models. Background Technology

[0002] Empathy is a high-level cognitive ability in humans, broadly defined as a set of connections that link one individual's response to another's experience. Therefore, it is crucial for large language models to demonstrate genuine empathy and understanding in human interactions. The expression and understanding of empathy are unique across different cultures and contexts, and also vary depending on the role and personality. Therefore, it is necessary to further optimize and improve existing large language models to enable them to respond with empathy in various dialogue scenarios.

[0003] Existing technologies have already made some progress in empathy: In 2023, Jiang et al. found that GPT-3.5 had a surprisingly good performance in empathic responses; in 2007, Williams et al. and in 2023, U large language model an explored the similarity between large language models and human emotion processing, and emphasized the excellent ability of GPT series models to predict human emotions; in 2022, Lee, Lim and Choi studied the empathic dialogue generation of GPT-3 through learning in context, while in 2023, Zhao et al. demonstrated the potential of ChatGPT in emotion response generation.

[0004] While some existing large language models, such as ChatGPT and GPT4, have demonstrated high capabilities in certain theory of mind tasks, they frequently fail in complex emotional dialogues. This implies a discrepancy between performance on non-dialogue tasks and genuinely empathetic responses. Although some studies suggest that large language models can effectively detect human emotions in dialogue and provide empathetic responses, the perception of empathy can vary across different cultural contexts. For example, English dialogues with low-context cultural characteristics tend to be more explicit, while Chinese dialogues with high-context cultural characteristics tend to be more implicit and concise. This makes it possible for large language models to fail to provide responses that truly align with specific cultural empathy in certain contexts. Furthermore, empathy in dialogue is closely related to the participant's assigned role and personality. Although there has been research on the application of large language models in role-playing, the main focus of these studies has been ensuring that the content is consistent with the assigned role and context. Whether these large language models can empathize with assigned roles and personalities remains unresolved.

[0005] In summary, existing technologies still face many challenges in improving dialogue in high-context cultural contexts using large language models and incorporating roles and personalities into empathetic responses. Summary of the Invention

[0006] This invention is made to solve the above-mentioned problems, and aims to provide a method and apparatus for generating responses with enhanced empathy in a large language model.

[0007] This invention provides a method for generating highly contextualized empathic responses using a large language model. This method generates corresponding empathic responses based on a specified dialogue and includes the following steps: Step S1, constructing an empathic dialogue dataset based on existing empathic dialogue data, the dataset comprising multiple task subsets of varying difficulty; Step S2, training an existing large language model using the empathic dialogue dataset to obtain an empathic response model and a reinforcement learning model; Step S3, inputting the specified dialogue into the empathic response model to obtain an initial response; Step S4, inputting the initial response into the reinforcement learning model to obtain an empathic response. Step S2 includes the following sub-steps: Step S2-1, performing supervised fine-tuning of the large language model based on all task subsets to obtain a fine-tuned large language model; Step S2-2, generating multiple responses corresponding to each task subset based on the fine-tuned large language model. For the response data; Step S2-3, construct the corresponding reward model based on all response data of each task subset; Step S2-4, randomly sort all task subsets to obtain the first sort; Step S2-5, based on the first sort, perform single-turn dialogue meta-learning on the fine-tuned large language model through each task subset to obtain the first EM model φ, and perform multi-turn dialogue meta-learning on the fine-tuned large language model to obtain the second EM model λ; Step S2-6, re-sort the first sort based on the first EM model φ and the second EM model λ to obtain the second sort from easy to difficult; Step S2-7, based on the second sort, perform reinforcement learning on the fine-tuned large language model through each task subset until a preset performance threshold is met, thus obtaining the empathic response model; Step S2-8, perform reinforcement learning on the empathic response model based on the reward model to obtain the reinforcement learning model.

[0008] The large language model high-context empathy enhancement response generation method provided by this invention may also have the following features: In steps S2-3, the existing large language model with a softmax layer is replaced by a linear layer in the reward model; the input of the reward model is the response; the output of the reward model is the response quality score corresponding to that response; and the expression for the loss of the reward model during training is: In the formula, x represents a response in the response data, D is the set of all ranked pairs constructed from pairwise responses in the response data, and y w and y l For the two responses in the ranked pair, and y w For the higher-ranked responses, y l For lower-ranked responses, r θ This is the scalar output of the reward model, where k is the total number of responses in the response data.

[0009] The large language model high-context empathy enhancement response generation method provided by this invention may also have the following feature: wherein, in steps S2-5, the first EM model φ is in the task subset T i The parameters are updated using gradient descent, and its expression is: In the formula, α is the learning rate. To fine-tune the large language model on task T i The performance energy obtained from training is used to fine-tune the parameters of the large language model by combining the trained first EM model φ with gradient descent.

[0010] The large language model high-context empathy enhancement response generation method provided by this invention may also have the following feature: In steps S2-5, a memory module M is set in the multi-turn dialogue meta-learning to store past empathy feedback and dialogue history; the second EM model λ is in the task subset T. i The parameters are updated using gradient descent, and its expression is: In the formula, β is the learning rate. To fine-tune the large language model and memory module M in task T i The energy obtained through training.

[0011] The method for generating responses with enhanced empathy in a large language model provided by this invention may also have the following feature: wherein, in steps S2-7, according to the second sorting, all task subsets constitute a task sequence. Its expression is: In the formula For the i-th task subset, where n is the total number of task subsets, fine-tune the large language model on the task subset. Performance metrics and task subset Performance metrics on previous task subsets When all preset performance thresholds are met, the fine-tuned large language model is enabled in the task subset. Training was performed on the large language model until fine-tuning was applied to a subset of tasks. If the preset performance threshold is also met, then the empathic response model is obtained.

[0012] The high-context empathy enhancement response generation method for large language models provided by this invention may also have the following feature: wherein, in steps S2-8, reinforcement learning includes a first loss for enabling the reinforcement learning model to generate output responses that are more human-like based on the input responses, a second loss for maintaining the similarity between the reinforcement learning model and the empathy response model, and a third loss for a general task. The expression for the first loss is: In the formula, x is the input response, y is the output response, and r θ (x, y) represents the scores of the reward model for the input response x and the output response y. The expression for the second loss is: In the formula For the responses generated by the reinforcement learning model, π SFT (y|x) represents the response generated by the empathy response model, and the expression for the third loss is: In the formula D pretrain The dataset selected for pre-training.

[0013] The high-context empathy enhancement response generation method for large language models provided in this invention may also have the following features: Multiple task subsets include SER task subsets, SRER task subsets, SPER task subsets, SRPER task subsets, BER task subsets, BPER task subsets, and ERPM task subsets. The SER task subset is language-oriented empathy response, focusing on generating responses based on dialogue history. It obtains user questions related to specific emotions through existing large language models and formulates corresponding empathy responses. The SRER task subset is language-oriented role-guided empathy response, constructed based on the SER task subset by integrating roles into the dialogue history. The SPER task subset... The task subsets are: Language-oriented personality-guided empathic responses, based on the SER task subset and constructed by integrating dialogue history and personality; Language-oriented role-personality-guided empathic responses, based on SER and constructed by combining emotion, role, and personality for empathic responses; Behavior-oriented empathic responses, focusing on behavioral responses; Behavior-oriented personality-guided empathic responses, focusing on how different personality traits influence behavior-driven responses; and Emotion-Role-Personality Pattern Determination, obtained by merging the SER, SRER, SPER, and SRPER task subsets.

[0014] This invention also provides a large language model high-context empathy enhancement response generation device for generating corresponding empathic responses based on a specified dialogue. It features the following characteristics: a data input module for user input of a specified dialogue; an initial response generation module containing a trained empathic response model for obtaining an initial response based on the specified dialogue; and an empathic response generation module containing a trained reinforcement learning model for obtaining an empathic response based on the initial response. The empathic response model and the reinforcement learning model are trained on an existing large language model using an empathic dialogue dataset. The empathic dialogue dataset is constructed from existing empathic dialogue data and includes multiple task subsets of varying difficulty. The training process of the empathic response model and the reinforcement learning model includes the following steps: Step S2-1, supervised fine-tuning of the large language model based on all task subsets to obtain a fine-tuned large language model; Step S2-2, based on the fine-tuned large language model... For each task subset, generate multiple corresponding responses as response data; Step S2-3: Construct a corresponding reward model based on all response data of each task subset; Step S2-4: Randomly sort all task subsets to obtain a first sort; Step S2-5: Based on the first sort, sequentially perform single-turn dialogue meta-learning on the fine-tuned large language model through each task subset to obtain a first EM model φ, and perform multi-turn dialogue meta-learning on the fine-tuned large language model to obtain a second EM model λ; Step S2-6: Re-sort the first sort based on the first EM model φ and the second EM model λ to obtain a second sort from easy to difficult; Step S2-7: Based on the second sort, sequentially perform reinforcement learning on the fine-tuned large language model through each task subset until a preset performance threshold is met, thus obtaining an empathic response model; Step S2-8: Perform reinforcement learning on the empathic response model based on the reward model to obtain a reinforcement learning model.

[0015] The role and effect of invention

[0016] According to the method and apparatus for generating high-context empathic enhanced responses using a large language model, this invention achieves the following: First, it creates a comprehensive Chinese empathic dialogue dataset based on existing data, comprising multiple task subsets oriented towards language and behavior with different focuses such as emotion, role, and personality. The model is trained using this dataset, enabling it to provide more appropriate empathic responses based on different emotions, roles, and personalities within high-context cultures. Second, it comprehensively trains the large language model through supervised fine-tuning, empathic-guided meta-curriculum learning, and reinforcement learning, allowing the model to provide empathic responses based on actual dialogue scenarios. Therefore, the method and apparatus for generating high-context empathic enhanced responses using a large language model of this invention can generate more empathic responses that conform to emotions, roles, and personalities within high-context cultures. Attached Figure Description

[0017] Figure 1This is a block diagram of the large language model high-context empathy enhancement response generation device in an embodiment of the present invention;

[0018] Figure 2 This is a schematic diagram illustrating the principle of training the empathic response model and the reinforcement learning model in an embodiment of the present invention;

[0019] Figure 3 This is a schematic diagram illustrating the process of training the empathic response model and the reinforcement learning model in an embodiment of the present invention;

[0020] Figure 4 This is a flowchart illustrating the method for generating responses using a large language model with enhanced empathy in a specific context, as described in an embodiment of the present invention. Detailed Implementation

[0021] To make the technical means, creative features, objectives and effects of the present invention easy to understand, the following embodiments, in conjunction with the accompanying drawings, provide a detailed description of the method and apparatus for generating responses with enhanced empathy in a large language model.

[0022] This embodiment provides a large language model high-context empathy enhancement response generation device, which is used to generate corresponding empathic responses based on a specified dialogue.

[0023] Figure 1 This is a block diagram of the response generation device for a large language model with enhanced empathy in a specific context, as described in an embodiment of the present invention.

[0024] like Figure 1 As shown, the large language model high-context empathy enhancement response generation device 100 includes a data input module 10, an initial response generation module 20, an empathy response generation module 30, and an output module 40.

[0025] The data input module 10 is used for users to input specified dialogue.

[0026] The initial response generation module 20 contains a trained empathic response model, which is used to generate an initial response based on a specified dialogue.

[0027] The empathic response generation module 30 contains a trained reinforcement learning model used to generate empathic responses based on the initial responses.

[0028] Among them, the empathic response model and the reinforcement learning model are trained on the existing large language model based on the empathic dialogue dataset. The empathic dialogue dataset is constructed based on the existing empathic dialogue data and includes multiple task subsets of different difficulty levels.

[0029] Among them, several task subsets include the SER task subset, SRER task subset, SPER task subset, SRPER task subset, BER task subset, BPER task subset, and ERPM task subset.

[0030] The SER task subset is language-oriented empathic response, which focuses on generating responses based on dialogue history. It obtains user questions related to specific emotions through existing large language models and formulates corresponding empathic responses. In this embodiment, GPT3.5 is mainly used to obtain user questions related to specific emotions.

[0031] The SRER task subset is a language-oriented role-guided empathic response, built upon the SER task subset by integrating roles into the dialogue history.

[0032] The SPER task subset is a language-oriented personality-guided empathic response, which is constructed by integrating dialogue history and personality based on the SER task subset.

[0033] The SRPER task subset is a language-oriented role-personality guided empathic response, based on SER, and constructed by combining emotion, role, and personality to create empathic responses.

[0034] The BER task subset is behavior-oriented empathic response, which focuses on behavior-based responses. In this embodiment, behavior testing is performed on the large language model, with each question asked five times to determine the baseline behavior of the large language model.

[0035] The BPER task subset is a behavior-oriented personality-guided empathic response, focusing on how different personality types influence behavior-driven responses. In this embodiment, volunteers are recruited to conduct behavioral tests using MBTI, with five volunteers assigned to each personality type to ensure that each personality type has a clear behavioral pattern tendency, thereby constructing the BPER task subset.

[0036] The ERPM task subset is determined by the emotion-role-personality pattern and is obtained by merging the SER task subset, SRER task subset, SPER task subset, and SRPER task subset.

[0037] Figure 2 This is a schematic diagram illustrating the principles of training the empathic response model and the reinforcement learning model in an embodiment of the present invention.

[0038] like Figure 2 As shown, in this embodiment, supervised fine-tuning, empathic-guided meta-curriculum learning, and reinforcement learning are sequentially applied to the existing large language model to obtain an empathic response model and a reinforcement learning model. Meta-curriculum learning includes empathic-guided meta-learning (ML) and curriculum learning (CL). In meta-learning (ML), a first EM model φ and a second EM model λ are constructed. All task subsets are then sorted from easy to difficult using the first EM model φ and the second EM model λ. In curriculum learning (CL), the supervised fine-tuned large language model is then trained according to this sorting from easy to difficult.

[0039] Figure 3 This is a schematic diagram illustrating the process of training the empathic response model and the reinforcement learning model in an embodiment of the present invention.

[0040] like Figure 3 As shown, the training process for the empathic response model and the reinforcement learning model includes the following steps:

[0041] Step S2-1: Perform supervised fine-tuning of the large language model based on all task subsets to obtain the fine-tuned large language model.

[0042] In this embodiment, supervised fine-tuning is used to enhance the ability of the large language model to generate responses that match real-world data, and the model is updated using the cross-entropy loss function. To specify a particular task during training, this embodiment adds a short instruction before the model's input, such as "This is a... (SER / SRER / SPER / SRPER / BER / BPER / ERPM) task. The goal of this task is...".

[0043] Step S2-2: Based on the fine-tuned large language model, generate multiple corresponding responses for each task subset as response data.

[0044] Steps S2-3: Construct the corresponding reward model based on all response data for each task subset, i.e., for each task subset T i A corresponding reward model R i .

[0045] In this embodiment, the reward model is an existing large language model in which the softmax layer is replaced with a linear layer. The existing large language model in this embodiment is baichuan. The input of the reward model is the response, and the output of the reward model is the response quality score corresponding to the response.

[0046] The expression for the loss of the reward model during training is:

[0047]

[0048] In the formula, x represents a response in the response data, D is the set of all ranked pairs constructed from pairwise responses in the response data, and y w and y l For the two responses in the ranked pair, and y w For the higher-ranked responses, y l For lower-ranked responses, r θ This is the scalar output of the reward model, where k is the total number of responses in the response data.

[0049] In this embodiment, set D contains a ranking sequence {r1, r2, ..., r...} formed by human participants evaluating 1% of the responses generated from each task subset.k-1 r k In the}, k represents the total number of 1% of responses corresponding to the ranking sequence. The ranking sequence is sorted according to the human participants' approval of the responses, so that the reward model can more effectively mimic human preferences, that is, assigning higher response quality scores (rewards) to high-quality responses and assigning lower response quality scores (penalties) to low-quality responses.

[0050] Step S2-4: Randomly sort all task subsets to obtain the first sort.

[0051] Steps S2-5: Based on the first sorting, the first EM model φ is obtained by performing single-turn dialogue meta-learning on the fine-tuned large language model through each task subset. Then, the second EM model λ is obtained by performing multi-turn dialogue meta-learning on the fine-tuned large language model. In this embodiment, the parameters of the fine-tuned large language model are adjusted by combining the trained first EM model φ with gradient descent.

[0052] Among them, the first EM model φ is in the task subset T i The parameters are updated using gradient descent, and its expression is:

[0053]

[0054] In the formula, α is the learning rate. To fine-tune the large language model on task T i The energy obtained through training.

[0055] In the multi-turn dialogue meta-learning, a memory module M is set up to store past empathic feedback and dialogue history. The second EM model λ is used in the task subset T. i The parameters are updated using gradient descent, and its expression is:

[0056]

[0057] In the formula, β is the learning rate. To fine-tune the large language model and memory module M in task T i The attribute energy obtained from training is the cross-entropy loss for both single-turn dialogue meta-learning and multi-turn dialogue learning in this embodiment.

[0058] In this embodiment, the memory module M is a key-value storage structure, with dialogue history as the key and corresponding empathic feedback as the value, thus stored in the memory module M. The memory module M stores the empathic feedback and dialogue history of each task subset for subsequent responses. The parameters of the second EM model λ are adjusted to better utilize the dialogue history and empathic feedback stored in the memory module M.

[0059] Step S2-6: Reorder the first sorting according to the first EM model φ and the second EM model λ to obtain a second sorting from easy to difficult.

[0060] In this embodiment, steps S2-4 to S2-6 are equivalent to performing meta-learning (ML).

[0061] Step S2-7: According to the second sorting, the fine-tuned large language model is subjected to reinforcement learning through each task subset until the preset performance threshold is met, thus obtaining the empathic response model. In this embodiment, step S2-7 is to execute the course learning CL.

[0062] According to the second sorting, all task subsets constitute the task sequence. Its expression is:

[0063]

[0064] In the formula Let be the i-th task subset, and n be the total number of task subsets.

[0065] Fine-tuning large language models on task subsets Performance metrics and task subset Performance metrics on previous task subsets When all preset performance thresholds are met, the fine-tuned large language model is enabled in the task subset. Training was performed on the large language model until fine-tuning was applied to a subset of tasks. If the preset performance threshold is also met, that is, if the fine-tuned large language model is trained on all task subsets according to the second sort and meets the preset performance threshold, then the empathic response model is obtained.

[0066] In this implementation, the empathic-guided meta-curriculum learning enables the large language model to automatically adjust the order of training tasks, training from easy to difficult tasks one by one. As more complex tasks are gradually introduced, it can achieve faster adaptation and generalization, thereby improving the model training speed and model accuracy.

[0067] Steps S2-8 involve performing reinforcement learning on the empathy response model based on the reward model to obtain a reinforcement learning model. In this embodiment, steps S2-2 to S2-3 are used to construct the reward model for reinforcement learning, and further combined with step S2-8 to realize reinforcement learning of the model.

[0068] In this embodiment, the purpose of reinforcement learning is to correct biases in the reward model's scoring, as the distribution of responses generated during supervised fine-tuning may change. For each task subset, a large language model is optimized using reinforcement learning. First, the expected value of the reward model is maximized to consistently generate responses that humans consider best in the corresponding reinforcement learning model. The responses x generated by supervised fine-tuning using empathy-guided meta-curriculum learning are then input into the reinforcement learning model. In this process, a more human-friendly response, y, is obtained. y changes each time the reinforcement learning model is updated. Then, (x, y) is input into the reward model r. θ A score, or reward, is calculated, representing real-time feedback from the reward model. This process constitutes the calculation of the first loss, which enables the reinforcement learning model to generate output responses that are more human-like based on the input responses. The expression for the first loss is:

[0069]

[0070] In the formula, x is the input response, y is the output response, and r θ (x, y) represents the score of the reward model for the input response x and the output response y.

[0071] Next, the goal of this proposal is to maintain the similarity between the reinforcement learning model and the fine-tuned large language model. When the reinforcement learning model updates its parameters, it causes the generated response y to change based on a given response x. If there are significant differences between the reinforcement learning model and the fine-tuned large language model, the reward model's estimate of the score may be inaccurate. To address this issue, Kullback-Leibler (KL) divergence is used to measure the distance between the cues generated by the RL model and the SFT model. The objective is to minimize the KL divergence, i.e., to compute a second loss to maintain the similarity between the reinforcement learning model and the empathic response model. The expression for the second loss is:

[0072]

[0073] In the formula For the responses generated by the reinforcement learning model, π SFT (y|x) represents the response generated by the empathy response model.

[0074] To ensure that the large language model performs well not only on specific tasks but also on general NLP tasks, thus mitigating the problem of catastrophic forgetting, 10% of the data is extracted from general NLP tasks during the pre-training phase. Here, the Firefly-Train-1.1M dataset is used. The goal of pre-training is to generate outputs that are as good as or better than the original output based on the original query; that is, to compute a third loss for the general task. The expression for the third loss is:

[0075]

[0076] In the formula D pretrain The dataset selected for pre-training.

[0077] Output module 40 is used to output empathic responses.

[0078] The following description, in conjunction with the accompanying drawings, explains the process of generating a response using the large language model high-context empathy enhancement response device according to this embodiment.

[0079] Figure 4 This is a flowchart illustrating the method for generating responses using a large language model with enhanced empathy in a specific context, as described in an embodiment of the present invention.

[0080] like Figure 4 As shown, the method for generating highly contextualized empathic responses using a large language model is used to generate corresponding empathic responses based on a given dialogue. It includes the following steps:

[0081] Step S1: Construct an empathic dialogue dataset based on existing empathic dialogue data. The empathic dialogue dataset includes multiple task subsets with different levels of difficulty.

[0082] Step S2: Train the existing large language model based on the empathic dialogue dataset to obtain the empathic response model and the reinforcement learning model.

[0083] Step S3: The initial response is obtained by sending the specified dialogue input received by the data input module 10 into the initial response generation module 20, which contains the empathic response model.

[0084] Step S4: Input the initial response into the empathic response generation module 30 containing the reinforcement learning model to obtain the empathic response, and output the empathic response through the output module 40.

[0085] The role and effect of the embodiments

[0086] According to the method and apparatus for generating highly empathetic responses using a large language model in this embodiment, on the one hand, a comprehensive Chinese empathetic dialogue dataset is created based on existing data. This dataset includes multiple task subsets that focus on language and behavior and have different empathetic empathy, role, and personality emphases. The model is trained using this dataset to obtain responses that are more appropriate based on different emotions, roles, and personalities in highly empathetic cultural contexts. On the other hand, the large language model is comprehensively trained through supervised fine-tuning, empathy-guided meta-curriculum learning, and reinforcement learning, enabling the model to provide empathetic responses based on actual dialogue scenarios. In summary, this method can generate more empathetic responses that are consistent with emotions, roles, and personalities in highly empathetic cultural contexts.

[0087] The above embodiments are preferred embodiments of the present invention and are not intended to limit the scope of protection of the present invention.

Claims

1. A method for generating highly contextualized empathic responses using a large-scale language model, used to generate corresponding empathic responses based on a specified dialogue, characterized in that... Includes the following steps: Step S1: Construct an empathic dialogue dataset based on existing empathic dialogue data. The empathic dialogue dataset includes multiple task subsets of different difficulty levels. Step S2: Train the existing large language model based on the empathic dialogue dataset to obtain the empathic response model and the reinforcement learning model; Step S3: Input the specified dialogue into the empathic response model to obtain the initial response; Step S4: Input the initial response into the reinforcement learning model to obtain the empathic response. Step S2 includes the following sub-steps: Step S2-1: Perform supervised fine-tuning of the large language model based on all the task subsets to obtain the fine-tuned large language model; Step S2-2: Generate multiple responses as response data for each of the task subsets based on the fine-tuned large language model; Steps S2-3: Construct the corresponding reward model based on all the response data of each task subset; Step S2-4: Randomly sort all the task subsets to obtain the first sort; Steps S2-5: Based on the first sorting, the fine-tuned large language model is sequentially subjected to single-turn dialogue meta-learning through each of the task subsets to obtain the first EM model. The fine-tuned large language model is subjected to multiple rounds of dialogue meta-learning to obtain the second EM model. ; Steps S2-6: Based on the first EM model and the second EM model The first sort is reordered to obtain a second sort from easy to difficult. Steps S2-7: According to the second sorting, the fine-tuned large language model is subjected to reinforcement learning through each of the task subsets in turn until the preset performance threshold is met, and the empathic response model is obtained. Steps S2-8: Perform reinforcement learning on the empathy response model based on the reward model to obtain the reinforcement learning model. In steps S2-3, the reward model is replaced by a linear layer in the existing large language model, replacing the softmax layer. The input to the reward model is the response, and the output of the reward model is the response quality score corresponding to that response. The expression for the loss of the reward model during training is: ， In the formula This is one of the responses in the response data. It is the set of all ranked pairs constructed from the pairwise responses in the response data. and For the two responses in the ranked pair, and For the higher-ranked replies, For the lower-ranked replies, This is the scalar output of the reward model. The total number of replies in the response data. The multiple task subsets include the SER task subset, SRER task subset, SPER task subset, SRPER task subset, BER task subset, BPER task subset, and ERPM task subset. The SER task subset is language-oriented empathic response, which focuses on generating responses based on dialogue history. It uses existing large language models to obtain user questions related to specific emotions and formulates corresponding empathic responses. The SRER task subset is a language-oriented, role-guided empathic response, constructed by integrating roles into the dialogue history based on the SRER task subset. The SPER task subset is a language-oriented, personality-guided empathic response, constructed by integrating dialogue history and personality traits based on the SER task subset. The SRPER task subset is a language-oriented role-personality guided empathic response, constructed based on the SER task subset by combining emotion, role, and personality to create empathic responses. The subset of BER tasks is behavior-oriented empathic response, focusing on behavior-based responses. The BPER task subset is behavior-oriented personality-guided empathic responses, focusing on how different personality traits influence behavior-driven responses. The ERPM task subset is determined by the emotion-role-personality pattern and is obtained by merging the SER task subset, the SRER task subset, the SPER task subset, and the SRPER task subset.

2. The method for generating responses with enhanced high-context empathy in large-scale language models according to claim 1, characterized in that: in, In steps S2-5, the first EM model In the task subset The parameters are updated using gradient descent, and its expression is: ， In the formula For learning rate, For fine-tuning the large language model in the task The energy obtained through training on the above, The first EM model was trained The parameters of the fine-tuned large language model are adjusted using gradient descent.

3. The method for generating responses with enhanced high-context empathy in large-scale language models according to claim 1, characterized in that: in, In steps S2-5, a memory module is provided in the multi-turn dialogue meta-learning. It is used to store past empathic feedback and dialogue history. Second EM model In the task subset The parameters are updated using gradient descent, and its expression is: ， In the formula For learning rate, For the fine-tuning of the large language model and the memory module In the mission The energy obtained through training.

4. The method for generating responses with enhanced empathy in a large-scale language model according to claim 1, characterized in that: in, In steps S2-7, according to the second sorting, all the task subsets constitute a task sequence. Its expression is: ， In the formula For the first A subset of tasks This represents the total number of task subsets. The fine-tuning of the large language model in the task subset Performance metrics and the aforementioned task subset Performance metrics on previous task subsets When all preset performance thresholds are met, the fine-tuned large language model is used in the task subset. Training was performed on the model until the fine-tuned large language model performed well on the task subset. If the preset performance threshold is also met, then the empathic response model is obtained.

5. The method for generating responses with enhanced empathy in a large-scale language model according to claim 1, characterized in that: in, In steps S2-8, the reinforcement learning includes a first loss for enabling the reinforcement learning model to generate output responses that are more human-like based on the input responses, a second loss for maintaining the similarity between the reinforcement learning model and the empathic response model, and a third loss for a general task. The expression for the first loss is: ， In the formula Reply to the input. For the output response, The reward model responds to the input. and output reply The score, The expression for the second loss is: ， In the formula The response generated by the reinforcement learning model. The response generated by the empathy response model. The expression for the third loss is: ， In the formula The dataset selected for pre-training.

6. A large-scale language model high-context empathy enhancement response generation device, used to generate corresponding empathic responses based on a specified dialogue using the large-scale language model high-context empathy enhancement response generation method as described in any one of claims 1-5, characterized in that, include: The data input module is used for the user to input the specified dialogue; An initial response generation module includes a trained empathic response model, used to generate an initial response based on the specified dialogue; The empathic response generation module includes a trained reinforcement learning model used to generate the empathic response based on the initial response. The empathic response model and the reinforcement learning model are trained on existing large language models based on the empathic dialogue dataset. The empathic dialogue dataset was constructed based on existing empathic dialogue data and includes multiple task subsets of varying difficulty. The training process of the empathic response model and the reinforcement learning model includes the following steps: Step S2-1: Perform supervised fine-tuning of the large language model based on all the task subsets to obtain the fine-tuned large language model; Step S2-2: Generate multiple responses as response data for each of the task subsets based on the fine-tuned large language model; Steps S2-3: Construct the corresponding reward model based on all the response data of each task subset; Step S2-4: Randomly sort all the task subsets to obtain the first sort; Steps S2-5: Based on the first sorting, the fine-tuned large language model is sequentially subjected to single-turn dialogue meta-learning through each of the task subsets to obtain the first EM model. The fine-tuned large language model is subjected to multiple rounds of dialogue meta-learning to obtain the second EM model. ; Steps S2-6: Based on the first EM model and the second EM model The first sort is reordered to obtain a second sort from easy to difficult. Steps S2-7: According to the second sorting, the fine-tuned large language model is subjected to reinforcement learning through each of the task subsets in turn until the preset performance threshold is met, and the empathic response model is obtained. Steps S2-8: Perform reinforcement learning on the empathy response model based on the reward model to obtain the reinforcement learning model.

Images