Customer service dialogue abstract model training method and generation method based on fusion training

Abstract

The invention relates to a customer service dialogue abstract model training method and a customer service dialogue abstract model generating method based on fusion training. The training method comprises the following steps: acquiring a customer service dialogue sample data set; performing fusion training on the pre-training language model by taking the customer service dialogue sample data set as training data to obtain a customer service dialogue abstract expert model; wherein the fusion training comprises the following steps: in a single training stage, optimizing the pre-training language model by jointly using a supervised fine tuning loss function and a reinforcement learning loss function; the supervision fine tuning loss function is used for enabling the pre-training language model to learn the element extraction capability in the customer service dialogue sample data set, and the reinforcement learning loss function is used for enabling the pre-training language model to optimize a customer service dialogue abstract generation strategy based on the reward signal; through deep fusion of supervised fine tuning and reinforcement learning in a single training stage, the model maintains high-precision element extraction capability, the generalization performance is significantly enhanced, and the training efficiency and stability are also synchronously improved.

Description

Technical Field

[0001] This invention relates to the fields of natural language processing and artificial intelligence, and in particular to a training method and generation method for a customer service dialogue summary model based on fusion training. Background Technology

[0002] In e-commerce, finance, and internet services, online customer service systems handle massive amounts of human-computer dialogue daily. To improve service efficiency and reduce labor costs, companies generally adopt a tiered service architecture: first-line agents handle real-time customer inquiries, record customer requests, and generate work orders; complex issues are then routed to second-line agents. Before closing a case, second-line agents often need to review multiple rounds of historical communication records under the same work order. Therefore, the system needs to automatically generate a summary for each dialogue. This summary must not only outline the facts but also accurately extract high-value elements such as time, amount, and decision from the conversational and fragmented dialogue.

[0003] In recent years, text generation technology based on large language models has been introduced into customer service dialogue summarization scenarios. The current mainstream approach in the industry is to use supervised fine-tuning, which involves fine-tuning the pre-trained large language model using manually annotated dialogue-summary sample pairs. This method improves the accuracy of feature extraction to some extent.

[0004] However, this approach suffers from a significant "catastrophic forgetting" problem. During fine-tuning, the model overfits to specific patterns and expressions in the training data, causing its generalization ability to plummet when faced with unfamiliar new business scenarios or new ways of expressing customers. This manifests as generated summaries omitting key elements, extracting incorrect elements, or fabricating non-existent information (illusion), which in turn increases the verification costs for second-line agents.

[0005] To alleviate the above problems, existing technologies have proposed an improvement scheme, attempting to adopt a two-stage serial training process: the first stage performs supervised fine-tuning to enable the model to have basic feature extraction capabilities; the second stage introduces reinforcement learning, using human feedback or rule-based rewards to further optimize the model, aiming to restore its generalization performance while maintaining accuracy.

[0006] Although the two-stage approach aims to balance accuracy and generalization, it suffers from the following insurmountable drawbacks in engineering practice: One issue is overfitting, which leads to a loss of the foundation for exploration: During the supervised fine-tuning phase, after long-term training, the model tends to memorize training data rather than learn general inference rules, resulting in overfitting. This leaves the model without a solid foundation for effective exploration in subsequent reinforcement learning phases.

[0007] Secondly, the training process is unstable: if the exploration intensity is not properly controlled during the reinforcement learning phase, it will cause the model training process to fluctuate violently, and the overall performance may even be lower than that of a single supervised fine-tuning model.

[0008] Thirdly, the training cycle is long and the iteration efficiency is low: the two-stage training process is carried out in series, which is long and costly, and is not conducive to rapid iteration and model updates in business scenarios. Summary of the Invention

[0009] This invention provides a training method and generation method for a customer service dialogue summary model based on fusion training, in order to solve the technical problems of model overfitting, training instability and long cycle caused by traditional two-stage training methods.

[0010] In a first aspect, the present invention provides a training method for a customer service dialogue summarization model based on fusion training, comprising: acquiring a customer service dialogue sample dataset; using the customer service dialogue sample dataset as training data to perform fusion training on a pre-trained language model to obtain a customer service dialogue summarization expert model; wherein, the fusion training includes: in a single training phase, jointly using a supervised fine-tuning loss function and a reinforcement learning loss function to optimize the pre-trained language model; the supervised fine-tuning loss function is used to enable the pre-trained language model to learn the feature extraction capability in the customer service dialogue sample dataset, and the reinforcement learning loss function is used to enable the pre-trained language model to optimize its customer service dialogue summarization generation strategy based on a reward signal.

[0011] In some embodiments, obtaining the customer service dialogue sample dataset includes: pre-annotating the original customer service dialogue using multiple large language models; integrating the pre-annotation results of the multiple large language models based on a majority voting mechanism to obtain preliminary annotation results; and verifying the preliminary annotation results to form the final customer service dialogue sample dataset.

[0012] In some embodiments, the supervised fine-tuning loss function is the proximal supervised fine-tuning loss function. The calculation formula is as follows:

[0013] in, This represents the model parameters of the pre-trained language model. This indicates the expectation calculated for samples sampled from the customer service dialogue sample dataset D. , representing the first standard answer summary of the corresponding sample t The importance sampling ratio of each word position Indicates the corresponding sample in the th t The generation state of each lexical position, including Hints and generated previoust -1 abstract terminology, Indicates the corresponding sample in the th t The standard answer word element corresponding to each word element position The parameter is The current model strategy, This represents the old model strategy before the model parameters were updated. Used to characterize the current model policy and the old model policy in the same generation state. Choose the correct answer word. The ratio of the probabilities, This represents the clipping function, used to clip... Limited to Within the interval, This represents the preset clipping hyperparameters.

[0014] In some embodiments, the reinforcement learning loss function is a dynamic pruning policy optimization loss function. The calculation formula is as follows:

[0015] in, This represents the model parameters of the pre-trained language model. This indicates the number of candidate summaries generated for a single customer service conversation prompt. An index representing candidate abstracts. Indicates the first A sequence of terms from candidate summaries, express Length, m Indicates the term position index in the candidate summary. , indicating the first i The candidate abstract m The importance sampling ratio of each word position Indicates the first i The candidate abstract in the ... m The generation state of each lexical position, including Hints and generated previous m -1 candidate summary terms, Indicates the first i The candidate abstract in the ... m Target word at each word position The parameter is The current model strategy, This represents the old model strategy before the model parameters were updated. Used to characterize the current model policy and the old model policy in the same generation state. Select target words below The ratio of the probabilities, Indicates the i-th candidate summary at the i-th position. m The advantage function estimate at the nth lexical position is used to evaluate the advantage function at the nth lexical position. The candidate abstract in the ... m A parameter representing the quality of the generation action taken at a given word position relative to the average quality of all candidate summaries generated at that word position under the same customer service dialogue prompt. and These represent the lower and upper boundary hyperparameters of the dynamic clipping, respectively. This represents the clipping function, used to clip... Limited to the range Within the range.

[0016] In some embodiments, the joint use of the supervised fine-tuning loss function and the reinforcement learning loss function is achieved through the total loss function, the calculation formula of which is:

[0017] in, Represents the total loss function. This represents the reinforcement learning loss function. This indicates the supervised fine-tuning loss function. This represents the weighting coefficient for dynamic decay.

[0018] In some embodiments, after performing fusion training on the pre-trained language model, the method further includes: identifying difficult samples whose reward value is lower than a predetermined threshold after the fusion training; using the difficult samples as training data, and using a loss function incorporating KL divergence constraints to enhance and fine-tune the customer service dialogue summary expert model.

[0019] Secondly, the present invention provides a customer service dialogue summary generation method, which applies a customer service dialogue summary expert model trained by any one of the methods described in the first aspect. The method includes: receiving customer service dialogue text to be processed; inputting the customer service dialogue text into the customer service dialogue summary expert model to generate a corresponding customer service dialogue summary.

[0020] In some embodiments, generating the corresponding customer service dialogue summary includes: splitting the elements to be extracted into a simple element set and a complex element set according to a predefined set of summary elements; directly extracting the elements in the simple element set using a fast thinking channel, and extracting the elements in the complex element set using a slow thinking channel after performing thought chain reasoning; merging the extraction results of the fast thinking channel and the slow thinking channel to obtain the corresponding customer service dialogue summary.

[0021] In some embodiments, the fast thinking channel disables the thought chain reasoning function of the customer service dialogue summary expert model by configuring model generation parameters; the slow thinking channel enables the thought chain reasoning function of the customer service dialogue summary expert model by configuring model generation parameters.

[0022] In some embodiments, the method further includes: constructing a structured prompt word template, the structured prompt word template integrating at least one: thought chain instructions for guiding the model's reasoning process, one or more Few-shot examples, hot word prompts for clarifying key expressions, and formatting requirements for standardizing the output format; the step of inputting the customer service dialogue text into the customer service dialogue summarization expert model includes: combining the structured prompt word template with the customer service dialogue text to form a combined input; and inputting the combined input into the customer service dialogue summarization expert model.

[0023] Thirdly, the present invention provides a training device for a customer service dialogue summarization model based on fusion training, comprising: an acquisition module for acquiring a customer service dialogue sample dataset; and a training module for using the customer service dialogue sample dataset as training data to perform fusion training on a pre-trained language model to obtain a customer service dialogue summarization expert model; wherein the fusion training includes: in a single training phase, jointly using a supervised fine-tuning loss function and a reinforcement learning loss function to optimize the pre-trained language model; the supervised fine-tuning loss function is used to enable the pre-trained language model to learn the feature extraction capability in the customer service dialogue sample dataset, and the reinforcement learning loss function is used to enable the pre-trained language model to optimize its customer service dialogue summarization generation strategy based on a reward signal.

[0024] Fourthly, the present invention provides a customer service dialogue summary generation device, which applies a customer service dialogue summary expert model trained by any one of the methods described in the first aspect. The device includes: a receiving module for receiving customer service dialogue text to be processed; and a generation module for inputting the customer service dialogue text into the customer service dialogue summary expert model to generate a corresponding customer service dialogue summary.

[0025] Fifthly, the present invention provides an electronic device, characterized in that it includes a processor, a communication interface, a memory, and a communication bus, wherein the processor, the communication interface, and the memory communicate with each other through the communication bus; the memory is used to store computer programs; and the processor, when executing the program stored in the memory, implements the steps of the customer service dialogue summary model training method based on fusion training as described in any one of the first aspects or the customer service dialogue summary generation method as described in any one of the second aspects.

[0026] In a sixth aspect, the present invention provides a computer-readable storage medium having a computer program stored thereon, characterized in that, when the computer program is executed by a processor, it implements the steps of the customer service dialogue summary model training method based on fusion training as described in any of the first aspects or the customer service dialogue summary generation method as described in any of the second aspects.

[0027] The customer service dialogue summary model training method and generation method based on fusion training provided by this invention deeply integrates supervised fine-tuning and reinforcement learning in a single training stage. This enables the model to maintain high-precision feature extraction capabilities while significantly enhancing generalization performance, effectively overcoming the technical challenges of "catastrophic forgetting" and "training instability". Moreover, this fusion training mechanism avoids the lengthy cycle and performance fluctuation problems of traditional two-stage serial training, significantly improving training efficiency and stability, and is more conducive to rapid iteration and application deployment in actual business. Attached Figure Description

[0028] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with the invention and, together with the description, serve to explain the principles of the invention.

[0029] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, for those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0030] Figure 1 A flowchart illustrating a customer service dialogue summary model training method based on fusion training provided in an embodiment of the present invention; Figure 2 A flowchart illustrating a customer service dialogue summary generation method provided in an embodiment of the present invention; Figure 3 A complete flowchart of a customer service dialogue summary model training and generation method based on fusion training provided in an embodiment of the present invention; Figure 4 A schematic diagram of the structure of a customer service dialogue summary model training device based on fusion training provided in an embodiment of the present invention; Figure 5 This is a schematic diagram of a customer service dialogue summary generation device provided in an embodiment of the present invention; Figure 6 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present invention. Detailed Implementation

[0031] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0032] Figure 1 This is a flowchart illustrating a customer service dialogue summary model training method based on fusion training, provided in an embodiment of the present invention. It is applied to a customer service dialogue summary model training device based on fusion training, or to an electronic device equipped with such a device. Figure 1 As shown, the training method includes: Step S101: Obtain customer service dialogue sample dataset.

[0033] Specifically, a customer service dialogue sample dataset is formed by collecting multiple "customer service dialogue prompts" (mainly customer service dialogue texts) and their corresponding "summaries".

[0034] In some embodiments, step S101 includes: pre-annotating the original customer service dialogue using multiple large language models; integrating the pre-annotation results of the multiple large language models based on a majority voting mechanism to obtain preliminary annotation results; and verifying the preliminary annotation results to form the final customer service dialogue sample dataset.

[0035] Specifically, after obtaining the original customer service dialogue data, multiple high-performance and heterogeneous large language models (such as Qwen2.5-72B, DeepSeek-V3, DeepSeek-R1, Qwen3-32B, etc.) are invoked in parallel. Each model independently pre-annotates the elements that need to be summarized in the dialogue. Then, the pre-annotation results of all models for the same dialogue are collected and integrated using a majority voting mechanism. That is, for each element to be summarized, the answer with the most votes is selected as its initial annotation result. Subsequently, the annotation personnel check and verify the initial annotation results. Based on their professional knowledge and contextual understanding, they correct complex errors or semantic ambiguities that the voting mechanism may not be able to handle, ultimately forming high-quality "customer service dialogue-summary" sample data pairs.

[0036] It should be noted that multiple large language models are used for pre-annotation to avoid errors caused by the inherent biases and blind spots of a single model; the results are integrated through a majority voting mechanism, which can effectively reduce the random errors and systematic biases of individual models by leveraging cross-validation and mutual error correction between models; the introduction of manual verification further eliminates the possible blind spots in machine collective understanding, thereby ensuring that the constructed customer service dialogue sample dataset meets the requirements for subsequent high-precision model training in terms of quality.

[0037] Step S102: Using the customer service dialogue sample dataset as training data, perform fusion training on the pre-trained language model to obtain a customer service dialogue summary expert model; wherein, the fusion training includes: in a single training phase, jointly using a supervised fine-tuning loss function and a reinforcement learning loss function to optimize the pre-trained language model; the supervised fine-tuning loss function is used to enable the pre-trained language model to learn the feature extraction ability in the customer service dialogue sample dataset, and the reinforcement learning loss function is used to enable the pre-trained language model to optimize its customer service dialogue summary generation strategy based on the reward signal.

[0038] Specifically, in this fusion training step, the customer service dialogue sample dataset is used as the training data, and the supervised fine-tuning loss function and the reinforcement learning loss function are used simultaneously and jointly to optimize the pre-trained language model in a single training phase.

[0039] The supervised fine-tuning loss function drives the pre-trained language model to learn the standard answers from the training data, enabling it to accurately extract key elements such as "amount" and "time" from dialogues, providing a reliable factual basis for summary generation. Meanwhile, the reinforcement learning loss function does not rely on fixed answers but is based on external reward signals (such as human feedback or preset rules), guiding the pre-trained language model to explore and generate higher-quality summaries that better meet business needs. This continuously optimizes its summary generation strategy and significantly improves the model's generalization ability in unseen scenarios.

[0040] In some embodiments, the supervised fine-tuning loss function is the proximal supervised fine-tuning loss function. The calculation formula is as follows: (1) in, This represents the model parameters of the pre-trained language model. This indicates the expectation calculated for samples sampled from the customer service dialogue sample dataset D. , representing the first standard answer summary of the corresponding sample t The importance sampling ratio of each word position Indicates the corresponding sample in the th t The generation state of each lexical position, including Hints and generated previous t -1 abstract terminology, This indicates that the corresponding sample corresponds to the t-th word position. The parameter is The current model strategy, This represents the old model strategy before the model parameters were updated. Used to characterize the current model policy and the old model policy in the same generation state. Choose the correct answer word. The ratio of the probabilities, This represents the clipping function, used to clip... Limited to Within the interval, This represents the preset clipping hyperparameters.

[0041] Specifically, the summary of each sample in the customer service dialogue sample dataset D is used as the standard answer summary, and each word in the summary is used as the standard answer word (or expert action) to perform supervised fine-tuning of the pre-trained model. In this embodiment, the supervised fine-tuning adopts the Proximal Supervised Fine-Tuning (PSFT) method. This method enables the model to have basic information extraction capabilities (corresponding to the ability to generate summary words in customer service dialogue text) through fine-tuning, achieving a "cold start" of the task; at the same time, it imposes constraints on the update of model parameters to avoid overfitting. This not only preserves the model's original powerful general capabilities but also lays a solid foundation for the subsequent reinforcement learning optimization stage. This method draws on the idea of ​​Proximal Policy Optimization (PPO). In each step of the parameter update during fine-tuning, the current policy is used to optimize the model. Changes are limited to the old strategy Within the vicinity of the target area, the optimization process is stabilized. This approach not only suppresses drastic policy fluctuations but also maintains high policy entropy, leaving ample room for exploration in subsequent reinforcement learning stages.

[0042] The derivation of the PSFT loss function (i.e., formula (1)) is as follows: First, let's look at the objective function of the standard supervised fine-tuning (SFT), as shown in formula (2): (2) in, The model parameters are: The loss in the supervised fine-tuning task (the fine-tuning goal is to minimize this loss); This represents the empirical expectation calculated from samples sampled from dataset D; Represents a training sample pair; Indicates the training sample at the th tThe generation status of each lexical position, including customer service dialogue prompts. x and the already generated previous One abstract term; Let represent the standard answer term at the t-th term position; D represents the offline supervised fine-tuning dataset; Let log be the logarithm of the model's predicted probability for the standard answer term. Maximizing this log-likelihood is equivalent to minimizing the model's cross-entropy loss on this task.

[0043] Secondly, the objective function of the policy gradient is shown in equation (3): (3) in, This represents the objective function of the policy gradient method; The state-action pair represents the state from the current policy. Samples obtained from interactions with the environment, rather than a fixed dataset; This represents the estimated value of the advantage function at the t-th word position, used to measure performance in state t. Take action below How significant is the advantage relative to the average level? If >0 indicates an action Better than average performance should be encouraged; if <0 indicates an action No, it should be suppressed.

[0044] In fact, SFT can be seen as a special case of policy gradient, and the core differences between the two lie in two points: 1. Sampling source: Sample pairs of SFT State-action pairs of policy gradients are sampled from a fixed training set. From the current strategy Online interaction with the environment; 2. Advantage Value Setting: SFT implicitly sets all standard answer terms. A constant advantage value was set. =1. Based on the objective function of SFT and the objective function of policy gradient, combined with the core idea of ​​PPO, the PSFT method can be derived: retaining the setting of SFT to sample from a fixed customer service dialogue sample dataset D, while continuing the logic of setting a constant advantage value for expert actions in SFT, and introducing the mechanism of PPO to prune the importance sampling ratio to constrain the policy update amplitude, finally obtaining the PSFT loss function shown in formula (1).

[0045] The core function of PSFT is to update the parameters of the policy model during training. At that time, through the clipping function Limiting the range of importance sampling ratios to constrain the current strategy Update magnitude: Avoid it being relative to the old strategy This approach avoids significant changes, thus ensuring training stability. It not only improves the model's generalization ability in "non-training distribution scenarios" (i.e., out-of-domain generalization) but also lays a solid foundation of high entropy for subsequent reinforcement learning stages: sufficient policy entropy provides a richer exploration space, ultimately helping the model achieve better performance on tasks such as customer service dialogue summarization.

[0046] In some embodiments, the reinforcement learning loss function is a dynamic pruning policy optimization loss function. The calculation formula is as follows: (4) in, This represents the model parameters of the pre-trained language model. This indicates the number of candidate summaries generated for a single customer service conversation prompt. An index representing candidate abstracts. Indicates the first A sequence of terms from candidate summaries, express Length, m Indicates the term position index in the candidate summary. , indicating the first i The candidate abstract m The importance sampling ratio of each word position Indicates the first i The candidate abstract in the ... m The generation state of each lexical position, including Hints and generated previous m -1 candidate summary terms, Indicates the first i The candidate abstract in the ... m Target word at each word position The parameter is The current model strategy, This represents the old model strategy before the model parameters were updated. Used to characterize the current model policy and the old model policy in the same generation state. Select target words below The ratio of the probabilities, Indicates the i-th candidate summary at the i-th position. m The advantage function estimate at the nth lexical position is used to evaluate the advantage function at the nth lexical position. The candidate abstract in the ... mA parameter representing the quality of the generation action taken at a given word position relative to the average quality of all candidate summaries generated at that word position under the same customer service dialogue prompt. and These represent the lower and upper boundary hyperparameters of the dynamic clipping, respectively. This represents the clipping function, used to clip... Within the range.

[0047] Specifically, The length refers to the first i The number of tokens in the word sequence of each candidate summary; include hint q and the previously generated m -1 candidate summary terms During training, the customer service dialogue prompts of the corresponding training samples in the customer service dialogue sample dataset D are input into the current model, which will generate G candidate summaries. Based on this, the loss function value of the Dynamic Clipping Policy Optimization (DCPO) for the current training round can be calculated according to formula (4).

[0048] In this embodiment, the DCPO loss function is constructed based on three core technologies: Dynamic Adaptive Clipping (DAC), Smooth Advantage Standardization (SAS), and Only Token Mean (OTM). These three technologies work together to solve the problems of training stability, exploration efficiency, and length bias, as detailed below: First, dynamic adaptive pruning is the core constraint mechanism of DCPO. Unlike traditional methods that use fixed pruning boundaries, DAC introduces a probability that varies with the old policy. q ( x The dynamically changing boundaries allow for more reasonable constraints on strategy updates.

[0049] Traditional methods typically constrain the importance sampling ratio to satisfy | r ( x ) 1 |≤ DCPO proposes that a more appropriate constraint should act on the probability scale itself, as shown below: ∣( r ( x ) 1)· p ( x )∣≤ (5) in p ( x )= r ( x )· q ( x ) represents the probability of the new strategy. r ( x )= p ( x ) / q ( x () represents the importance sampling ratio.

[0050] Will p ( x )= r ( x )· q ( x Substituting into the above equation to perform boundary derivation, we get: ∣( r ( x ) 1)· r ( x )· q ( x )∣≤ (6) By solving this inequality, we can derive... r ( x The dynamic clipping boundary of ) is [1 ε low, 1+ ε [high], among which ε low and ε `high` is a hyperparameter.

[0051] During training, the upper and lower boundaries [1] ε low, 1+ ε The actual width of [high] depends on the probability of the old strategy. q ( x Change: When q ( x When the number of rare words is small, the pruning boundary automatically widens, providing the model with a wider exploration space and helping to stimulate reasoning ability; when .... q ( x When the number of terms is large (corresponding to common terms), the clipping boundary shrinks to a narrower range to suppress drastic fluctuations in the strategy and thus maintain training stability.

[0052] Secondly, smooth advantage normalization aims to address the "zero gradient" problem caused by homogenization of single-step rewards in offline training. By fusing the current reward signal with the cumulative historical reward distribution, it makes advantage estimation more stable, making it particularly suitable for dialogue summarization scenarios where each prompt corresponds to multiple large model votes and labels.

[0053] The calculation of the advantage function value used for policy updates is performed in four steps: (1) Calculate the current step advantage Based on the current training step (the 1st training step) i (Step) generated for the same prompt G For each response's reward, calculate the mean and standard deviation, and then standardize: (7) in, Indicates the first i In the training step, the first j Rewards for each response; and The first i training steps G The mean and standard deviation of each response reward.

[0054] (2) Calculate the cumulative historical advantage Integration from step 1 to step 2 i Step, same prompt generated G × i For each response's reward, calculate its cumulative mean and standard deviation, and then standardize it: (8) in, and These represent the mean and standard deviation of the historical cumulative rewards, respectively.

[0055] (3) Calculate the smoothing advantage: Take a weighted average of the current step advantage and the cumulative historical advantage to obtain two smoothing advantage estimates: (9) (10) The weights are dynamically adjusted with the number of training steps i to balance immediate feedback and long-term experience. (4) Determine the final advantage: In order to suppress the estimation fluctuation, the smaller absolute value of the two smooth advantages is selected as the final advantage value. : (11) SAS accumulates reward signals across training steps, avoiding gradient vanishing caused by homogenization of single-step rewards, while taking into account the generalization value of historical rewards and the immediacy of current rewards, making model training more stable.

[0056] Finally, the term mean method is a loss normalization strategy used in DCPO, aiming to eliminate optimization bias caused by differences in response length and ensure that the model can treat candidate outputs of different lengths fairly. The core mechanism of OTM is to average the loss of all tokens within a single candidate response, rather than performing a global averaging at the batch level. This allows the loss of each response to be calculated independently, fully preserving the relative quality relationships between different responses. As shown in the DCPO objective function, the specific implementation of OTM is as follows: for each candidate response, the average of all its token losses is calculated independently first, and then these independent means are directly added together (rather than averaging) as part of the loss function. This design is reflected in the summation form. , rather than .

[0057] By avoiding averaging the loss across all responses within a batch, OTM prevents the model from generating verbose content due to the "high sum of losses for long responses." Instead, it focuses the model on the average quality of the responses themselves, thus giving stronger optimization signals to outputs that are "shorter but of higher quality," helping to improve the simplicity and accuracy of the generated results while ensuring information integrity.

[0058] In summary, dynamic adaptive pruning updates the magnitude through dynamic boundary constraint strategies, solves the gradient problem in calculating the advantage function through smooth advantage standardization, and optimizes the loss normalization method through word mean method. These three elements work together to form the core technical support of the DCPO loss function, achieving the goal of "stable training, sufficient exploration, and high-quality generation", and adapting to the optimization needs of "accurate elements + diverse expressions" in customer service dialogue summary scenarios.

[0059] In some embodiments, the combined use of the supervised fine-tuning loss function and the reinforcement learning loss function through the total loss function To achieve this, the calculation formula is as follows: (12) in, This represents the reinforcement learning loss function. This indicates the supervised fine-tuning loss function. This represents the weighting coefficient for dynamic decay.

[0060] Specifically, this embodiment adopts A decay strategy is used to coordinate the learning focus at different training stages: in the early stages of training, a larger decay setting is used. A value (such as close to 1) causes the total loss function to change from This approach, primarily driven by the desire to allow the model to prioritize learning from labeled data, quickly grasp the basic patterns of feature extraction through imitation learning, and lay a solid foundation for subsequent exploration, is crucial. As training progresses... The value gradually decays to a smaller value, at which point the total loss function transforms into... This approach prioritizes autonomous exploration through reinforcement learning in the later stages of training, optimizing generation strategies based on reward signals. This avoids overfitting the training data and effectively improves the model's generalization ability. For example, this can be achieved in the early stages of training (e.g., 0-200 steps). The value linearly decays from 0.9 to 0.05 in the later stages of training (after 200 steps). The value is fixed at 0.05.

[0061] In some embodiments, after the fusion training of the pre-trained language model, the method further includes: identifying difficult samples whose reward values ​​are below a predetermined threshold after the fusion training; and using the difficult samples as training data to fine-tune the customer service dialogue summarization expert model using a loss function incorporating KL divergence constraints. To further improve the model's performance in handling edge cases, after completing the fusion training and obtaining the basic expert model, a fine-tuning stage for difficult samples may also be included. Specifically, it is first necessary to identify difficult samples whose reward values ​​remain consistently below a predetermined threshold after the fusion training. These samples typically exhibit low reward scores even when the model generation parameters are adjusted, such as increasing the temperature parameter to increase output diversity. Subsequently, using these selected difficult samples as training data, the customer service dialogue summarization expert model is fine-tuned in a targeted manner.

[0062] To ensure that this fine-tuning does not compromise the generalization ability of the underlying expert model, this invention employs a special training strategy: firstly, a smaller learning rate is used for parameter updates to control the optimization pace and prevent excessive adjustments; secondly, a KL divergence constraint is introduced into the loss function. The loss function is designed as follows: (13) in, This represents supervised fine-tuning loss based on difficult samples (such as cross-entropy loss). This represents the KL divergence calculation, used to measure the performance of the enhanced, fine-tuned model. ) and the original expert model ( The difference between the output distributions is represented by β, which is a preset hyperparameter used to balance the importance of the two losses.

[0063] This loss function constrains the model's behavior from deviating excessively from the original, stable, and highly generalizable expert model while it learns to adapt to difficult samples. This approach allows the model to improve its ability to handle difficult cases while maintaining its strong generalization ability, achieving a balanced development of model performance.

[0064] The customer service dialogue summary model training method based on fusion training provided in this invention fundamentally solves the problems of "catastrophic forgetting" and "training instability" in traditional two-stage methods by deeply integrating supervised fine-tuning and reinforcement learning in a single training stage. This enables the final expert model to simultaneously possess high-precision feature extraction capabilities and strong scene generalization capabilities, and significantly improves training efficiency.

[0065] Furthermore, at the data source, a data construction method combining multi-model voting and manual verification ensured the high quality and reliability of the training set, laying a solid foundation for the model's excellent performance. During training, the application of proximal supervision fine-tuning suppressed overfitting, maintained the model's high policy entropy, and preserved crucial exploration flexibility for reinforcement learning; while dynamic pruning strategy optimization ensured the stability of the reinforcement learning process through an adaptive mechanism, effectively preventing training oscillations. In training scheduling, the dynamically weighted total loss function achieved a smooth and intelligent transition from imitation learning to autonomous exploration, optimizing the overall training trajectory. In the later optimization stage, the reinforcement fine-tuning strategy for difficult samples improved the model's ability to handle challenging cases while strictly protecting its acquired generalization ability from being compromised through KL divergence constraints. Ultimately, a comprehensive, stable, and reliable customer service dialogue summarization expert model was obtained.

[0066] Figure 2 This is a flowchart illustrating a customer service dialogue summary generation method provided in an embodiment of the present invention. It is applied to a customer service dialogue summary generation device or an electronic device equipped with such a device. Figure 2 As shown, the generation method includes: Step S201: Receive the customer service dialogue text to be processed.

[0067] Step S202: Input the customer service dialogue text into the customer service dialogue summary expert model to generate the corresponding customer service dialogue summary.

[0068] Specifically, the customer service dialogue summary expert model trained based on the aforementioned embodiments can be put into practical application. In the application stage, the original customer service dialogue text to be processed is first received, and then the text is input into the trained expert model. The expert model automatically calls upon its mastered element extraction and summary generation capabilities to intelligently analyze and process the input dialogue text content and output a customer service dialogue summary.

[0069] In some embodiments, generating the corresponding customer service dialogue summary in step S202 includes: splitting the elements to be extracted into a simple element set and a complex element set according to a predefined set of summary elements; directly extracting the elements in the simple element set using a fast thinking channel, and extracting the elements in the complex element set using a slow thinking channel after performing thought chain reasoning; merging the extraction results of the fast thinking channel and the slow thinking channel to obtain the corresponding customer service dialogue summary.

[0070] Specifically, in generating customer service dialogue summaries, the model employs a highly efficient parallel extraction mechanism. First, based on a predefined set of summary elements, the model intelligently categorizes the elements to be extracted into two types: simple elements containing explicit information and complex elements requiring in-depth understanding. For the simple element set, the model activates a fast-thinking channel, which directly outputs the element content without intermediate reasoning, achieving instantaneous extraction of summary information. Simultaneously, for the complex element set, the model uses a slow-thinking channel, employing a chain-of-thought reasoning approach to generate logical basis before outputting the final answer, ensuring accurate extraction of complex elements. Finally, following a predefined structured order, the outputs from the fast and slow thinking channels are intelligently combined and merged, maintaining the logical connections between elements and ensuring the completeness and consistency of the final summary. This clearly defined processing mechanism significantly improves overall generation efficiency while maintaining extraction accuracy.

[0071] In some embodiments, the fast thinking channel disables the thought chain reasoning function of the customer service dialogue summary expert model by configuring model generation parameters; the slow thinking channel enables the thought chain reasoning function of the customer service dialogue summary expert model by configuring model generation parameters.

[0072] Specifically, by setting model configuration parameters (such as the key parameter enable_thinking), the reasoning mode of the customer service dialogue summary expert model can be intelligently controlled: when this parameter is set to the disabled state, the model starts the fast thinking channel, at which time the model will bypass the complex thinking chain reasoning process and directly output the extraction results of simple elements; when this parameter is set to the enabled state, the model starts the slow thinking channel, at which time the model will first perform step-by-step reasoning and output the reasoning basis, and then generate the final answer.

[0073] In some embodiments, the method further includes: constructing a structured prompt word template, the structured prompt word template integrating at least one: thought chain instructions for guiding the model's reasoning process, one or more Few-shot examples, hot word prompts for clarifying key expressions, and formatting requirements for standardizing the output format; the step S202 of inputting the customer service dialogue text into the customer service dialogue summarization expert model includes: combining the structured prompt word template with the customer service dialogue text to form a combined input; and inputting the combined input into the customer service dialogue summarization expert model.

[0074] To further improve the accuracy and error tolerance of summary generation, this embodiment introduces the construction and application of a structured prompt template. This prompt template integrates multiple guidance mechanisms: it explicitly requires the model to demonstrate the reasoning process through thought chain instructions, ensuring the transparency and traceability of the analysis logic of complex elements; it provides few-shot examples, offering high-quality samples for the model to learn from, enabling it to master the correct summary format and content standards through contextual learning; it includes hot word prompts for speech-to-text conversion, pre-correcting potential homophonic errors or ambiguous expressions that may arise from speech recognition; and it enforces Markdown and other formatting requirements, significantly reducing the interference of text noise on summary accuracy through a unified structured output template.

[0075] In the specific implementation process, the input stage of step S202 includes two key operations: First, the constructed structured prompt word template is intelligently combined with the customer service dialogue text to be processed to form a combined input containing task instructions, example references, and data to be processed; then, this combined input is submitted to the customer service dialogue summarization expert model. This method explicitly guides the model's generation direction through prompt engineering, further enhancing its accuracy and robustness in handling complex scenarios based on the basic capabilities gained from fusion training, ultimately ensuring that the generated summary is both high-quality and highly readable.

[0076] The customer service dialogue summary generation method provided in this embodiment transforms advanced machine learning capabilities into an efficient summary generation service by applying an expert model that has been fused and trained.

[0077] Furthermore, the employed fast and slow thinking mechanism achieves optimized results by intelligently splitting and routing summary elements, enabling rapid extraction of simple elements and in-depth analysis of complex elements. It also allows for flexible switching of inference modes through model parameter configuration, significantly improving overall generation efficiency while maintaining the depth of inference for complex elements, achieving an ideal balance between accuracy and speed. The constructed comprehensive Prompt framework integrates advanced prompting technologies such as thought chains and Few-shot examples. Through clear guidance instructions and standardized structured output requirements, it provides comprehensive quality assurance for the generation process. This framework effectively reduces text noise interference, ensuring the accuracy and standardization of the output results. Finally, through the organic synergy of model capabilities, processing mechanisms, and prompting engineering, the customer service dialogue summary generated in this embodiment not only possesses high accuracy in element extraction but also maintains good readability and structural standardization.

[0078] Figure 3 A complete flowchart of a customer service dialogue summary model training and generation method based on fusion training provided in an embodiment of the present invention is shown below. Figure 3 As shown, this process includes two core stages: model training and summary generation. (1) Start and base model: The process begins with obtaining a general pre-trained language model as a base to lay the foundation for subsequent specialized training.

[0079] (2) Integrated training: This stage is based on a high-quality customer service dialogue sample dataset. In a single training stage, supervised fine-tuning and reinforcement learning loss functions are used together to enable the model to simultaneously master high-precision feature extraction capabilities and optimized summary generation strategies, ultimately forming a dedicated customer service dialogue summary expert model.

[0080] (3) Parallel extraction: In the model application stage, a parallel extraction mechanism is adopted. Based on the predefined set of elements, the summary task is intelligently divided into simple elements and complex elements, which are processed in parallel through the fast thinking channel (direct extraction) and the slow thinking channel (thinking chain reasoning) to achieve the best balance between generation speed and reasoning depth.

[0081] (4) Prompt Project: Through a carefully designed prompt project, a structured prompt template integrating thought chain instructions, Few-shot examples, hot word prompts and formatting requirements is constructed. After being combined with customer service dialogue text, the input model is significantly improved, enhancing the accuracy and standardization of the output results.

[0082] (5) Merge summary and results: Integrate all the element results generated by the fast and slow thinking channels according to the predetermined structured format, and output an accurate, complete and structurally consistent customer service dialogue summary.

[0083] The innovative breakthroughs of this invention are mainly reflected in three aspects: First, in terms of model training methods, a deep integration paradigm of supervised fine-tuning and reinforcement learning is proposed. Through a specific loss function and dynamic balancing strategy, the model effectively integrates training signals from different sources, achieving an optimal balance between imitation learning and autonomous exploration, significantly improving the model's generalization and reasoning capabilities. Second, in terms of task reasoning architecture, an intelligent splitting and routing mechanism based on element difficulty is proposed. Through a collaborative "fast and slow thinking" working mode, computational efficiency is optimized while ensuring the accuracy of element extraction. Finally, in terms of application-layer interaction technology, a composite framework integrating advanced prompting engineering technology is constructed. Through structured prompt templates and thought chain guidance processes, text noise interference is effectively suppressed, providing a reliable guarantee for the accuracy of dialogue summary generation.

[0084] Figure 4 This is a schematic diagram of the structure of a customer service dialogue summary model training device based on fusion training, provided in an embodiment of the present invention. Figure 4 As shown, the training device includes: Module 401 is used to acquire a customer service dialogue sample dataset; Training module 402 is used to perform fusion training on a pre-trained language model using the customer service dialogue sample dataset as training data to obtain a customer service dialogue summary expert model; wherein, the fusion training includes: in a single training phase, jointly using a supervised fine-tuning loss function and a reinforcement learning loss function to optimize the pre-trained language model; the supervised fine-tuning loss function is used to enable the pre-trained language model to learn the feature extraction ability in the customer service dialogue sample dataset, and the reinforcement learning loss function is used to enable the pre-trained language model to optimize its customer service dialogue summary generation strategy based on reward signals.

[0085] In some embodiments, the acquisition module 401 is specifically used for: Multiple large language models were used to pre-annotate the elements of the original customer service dialogue; Based on the majority voting mechanism, the pre-annotation results of the multiple large language models are integrated to obtain preliminary annotation results; the preliminary annotation results are verified to form the final customer service dialogue sample dataset.

[0086] In some embodiments, the supervised fine-tuning loss function is the proximal supervised fine-tuning loss function. The calculation formula is as follows:

[0087] in, This represents the model parameters of the pre-trained language model. This indicates the expectation calculated for samples sampled from the customer service dialogue sample dataset D. , representing the first standard answer summary of the corresponding sample t The importance sampling ratio of each word position Indicates the corresponding sample in the th t The generation state of each lexical position, including Hints and generated previous t -1 abstract terminology, Indicates the corresponding sample in the th t The standard answer word element corresponding to each word element position The parameter is The current model strategy, This represents the old model strategy before the model parameters were updated. Used to characterize the current model policy and the old model policy in the same generation state. Choose the correct answer word. The ratio of the probabilities, This represents the clipping function, used to clip... Limited to Within the interval, This represents the preset clipping hyperparameters.

[0088] In some embodiments, the reinforcement learning loss function is a dynamic pruning policy optimization loss function. The calculation formula is as follows:

[0089] in, This represents the model parameters of the pre-trained language model. This indicates the number of candidate summaries generated for a single customer service conversation prompt. An index representing candidate abstracts. Indicates the first A sequence of terms from candidate summaries, express Length, m Indicates the term position index in the candidate summary. , indicating the first i The candidate abstract m The importance sampling ratio of each word position Indicates the first i The candidate abstract in the ... m The generation state of each lexical position, including Hints and generated previous m -1 candidate summary terms, Indicates the first i The candidate abstract in the ... m Target word at each word position The parameter is The current model strategy, This represents the old model strategy before the model parameters were updated. Used to characterize the current model policy and the old model policy in the same generation state. Select target words below The ratio of the probabilities, Indicates the i-th candidate summary at the i-th position. m The advantage function estimate at the nth lexical position is used to evaluate the advantage function at the nth lexical position. The candidate abstract in the ... m A parameter representing the quality of the generation action taken at a given word position relative to the average quality of all candidate summaries generated at that word position under the same customer service dialogue prompt. and These represent the lower and upper boundary hyperparameters of the dynamic clipping, respectively. This represents the clipping function, used to clip... Limited to the range Within the range.

[0090] In some embodiments, the joint use of the supervised fine-tuning loss function and the reinforcement learning loss function is achieved through the total loss function, the calculation formula of which is:

[0091] in, Represents the total loss function. This represents the reinforcement learning loss function. This indicates the supervised fine-tuning loss function. This represents the weighting coefficient for dynamic decay.

[0092] In some embodiments, the training module 402 is further configured to: After performing fusion training on the pre-trained language model, identify difficult samples whose reward value is lower than a predetermined threshold after the fusion training. Using the aforementioned difficult samples as training data, the customer service dialogue summarization expert model is enhanced and fine-tuned using a loss function incorporating KL divergence constraints. Those skilled in the art will readily understand that, for the sake of convenience and brevity, the specific working process and corresponding beneficial effects of the customer service dialogue summarization model training device based on fusion training described above can be found in the corresponding process in the aforementioned method examples, and will not be repeated here.

[0093] Figure 5 This is a schematic diagram of a customer service dialogue summary generation device provided in an embodiment of the present invention, as shown below. Figure 5 As shown, the generating apparatus includes: The receiving module 501 is used to receive the customer service dialogue text to be processed; the generating module 502 is used to input the customer service dialogue text into the customer service dialogue summary expert model to generate the corresponding customer service dialogue summary.

[0094] In some embodiments, the generation module 502 is specifically used for: Based on a predefined set of summary elements, the elements to be extracted are split into a simple element set and a complex element set; elements in the simple element set are directly extracted using a fast thinking channel, while elements in the complex element set are extracted using a slow thinking channel after thought chain reasoning; the extraction results from the fast thinking channel and the slow thinking channel are merged to obtain the corresponding customer service dialogue summary.

[0095] In some embodiments, the fast thinking channel disables the thought chain reasoning function of the customer service dialogue summary expert model by configuring model generation parameters; The slow thinking channel enables the thought chain reasoning function of the customer service dialogue summary expert model by configuring model generation parameters.

[0096] In some embodiments, the generating apparatus further includes a configuration module 503, the configuration module 503 being configured to: Construct a structured prompt word template, which integrates at least one of the following: thought chain instructions for guiding the model reasoning process, one or more Few-shot examples, hot word prompts for clarifying key expressions, and formatting requirements for standardizing the output format; The generation module 502 is specifically used for: The structured prompt template is combined with the customer service dialogue text to form a combined input; The combination is input into the customer service dialogue summary expert model.

[0097] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working process and corresponding beneficial effects of the customer service dialogue summary generation device described above can be referred to the corresponding process in the aforementioned method example, and will not be repeated here.

[0098] like Figure 6 As shown, this embodiment of the invention provides an electronic device, including a processor 601, a communication interface 602, a memory 603, and a communication bus 604, wherein the processor 601, the communication interface 602, and the memory 603 communicate with each other via the communication bus 604. Memory 603 is used to store computer programs; In one embodiment of the present invention, when the processor 601 executes the program stored in the memory 603, it implements the steps of any of the aforementioned customer service dialogue summary model training methods or customer service dialogue summary generation methods based on fusion training.

[0099] The electronic device provided in this embodiment of the invention has a similar implementation principle and technical effect to the above embodiments, and will not be described again here.

[0100] The aforementioned memory 603 may be an electronic memory such as flash memory, EEPROM (Electrically Erasable Programmable Read-Only Memory), EPROM, hard disk, or ROM. Memory 603 has storage space for program code used to perform any of the method steps described above. For example, the storage space for program code may include individual program codes for implementing the various steps in the methods described above. This program code can be read from or written to one or more computer program products. These computer program products include program code carriers such as hard disks, optical discs (CDs), memory cards, or floppy disks. Such computer program products are typically portable or fixed storage units. The storage unit may have storage segments or storage spaces arranged similarly to memory 603 in the aforementioned electronic device. The program code may be compressed, for example, in a suitable form. Typically, the storage unit includes programs for performing the method steps according to embodiments of the invention, i.e., code that can be read by a processor such as 601, which, when run by the electronic device, causes the electronic device to perform the various steps in the methods described above.

[0101] Embodiments of the present invention also provide a computer-readable storage medium. The computer-readable storage medium stores a computer program, which, when executed by a processor, implements the steps of the customer service dialogue summary model training method or the customer service dialogue summary generation method based on fusion training as described above.

[0102] The computer-readable storage medium may be included in the device / apparatus described in the above embodiments; or it may exist independently and not assembled into the device / apparatus. The computer-readable storage medium carries one or more programs that, when executed, implement the method according to the embodiments of the present invention.

[0103] According to embodiments of the present invention, the computer-readable storage medium may be a non-volatile computer-readable storage medium, such as including, but not limited to: portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof. In the present invention, the computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, apparatus, or device.

[0104] It should be noted that, in this document, relational terms such as "first" and "second" are used merely to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0105] The above description is merely a specific embodiment of the present invention, enabling those skilled in the art to understand or implement the invention. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the invention. Therefore, the present invention is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features claimed herein.

Claims

1. A training method for a customer service dialogue summarization model based on fusion training, characterized in that, include: Obtain a sample dataset of customer service conversations; Using the customer service dialogue sample dataset as training data, a pre-trained language model is fused and trained to obtain a customer service dialogue summary expert model. The fused training includes: in a single training phase, jointly using a supervised fine-tuning loss function and a reinforcement learning loss function to optimize the pre-trained language model. The supervised fine-tuning loss function enables the pre-trained language model to learn the feature extraction capabilities from the customer service dialogue sample dataset, and the reinforcement learning loss function enables the pre-trained language model to optimize its customer service dialogue summary generation strategy based on reward signals.

2. The method according to claim 1, characterized in that, The process of obtaining the customer service dialogue sample dataset includes: Multiple large language models were used to pre-annotate the elements of the original customer service dialogue; Based on the majority voting mechanism, the pre-annotation results of the multiple large language models are integrated to obtain preliminary annotation results; the preliminary annotation results are verified to form the final customer service dialogue sample dataset.

3. The method according to claim 1, characterized in that, The supervised fine-tuning loss function is the near-end supervised fine-tuning loss function. The calculation formula is as follows: in, This represents the model parameters of the pre-trained language model. This indicates the expectation calculated for samples sampled from the customer service dialogue sample dataset D. , representing the first standard answer summary of the corresponding sample t The importance sampling ratio of each word position Indicates the corresponding sample in the th t The generation status of each lexical position, including customer service dialogue prompts and the generated preceding words. t -1 summary terminology, Indicates the corresponding sample in the th t The standard answer word element corresponding to each word element position The parameter is The current model strategy, This represents the old model strategy before the model parameters were updated. Used to characterize the current model policy and the old model policy in the same generation state. Choose the correct answer word. The ratio of the probabilities, This represents the clipping function, used to clip... Limited to Within the interval, This represents the preset clipping hyperparameters.

4. The method according to claim 1, characterized in that, The reinforcement learning loss function is the dynamic pruning strategy optimization loss function. The calculation formula is as follows: in, This represents the model parameters of the pre-trained language model. This indicates the number of candidate summaries generated for a single customer service conversation prompt. An index representing candidate abstracts. Indicates the first A sequence of terms from candidate summaries, express Length, m Indicates the term position index in the candidate summary. , indicating the first i The candidate abstract m The importance sampling ratio of each word position Indicates the first i The candidate abstract in the ... m The generation state of each lexical position, including Hints and generated previous m -1 candidate summary terms, Indicates the first i The candidate abstract in the ... m Target word at each word position The parameter is The current model strategy, This represents the old model strategy before the model parameters were updated. Used to characterize the current model policy and the old model policy in the same generation state. Select target words below The ratio of the probabilities, Indicates the i-th candidate summary at the 1st... m The advantage function estimate at the nth lexical position is used to evaluate the advantage function at the nth lexical position. The candidate abstract in the ... m A parameter representing the quality of the generation action taken at a given word position relative to the average quality of all candidate summaries generated at that word position under the same customer service dialogue prompt. and These represent the lower and upper boundary hyperparameters of the dynamic clipping, respectively. This represents the clipping function, used to clip... Limited to the range Within the range.

5. The method according to any one of claims 1-4, characterized in that, The combined use of the supervised fine-tuning loss function and the reinforcement learning loss function is achieved through the total loss function, the calculation formula of which is as follows: in, Represents the total loss function. This represents the reinforcement learning loss function. This indicates the supervised fine-tuning loss function. This represents the weighting coefficient for dynamic decay.

6. The method according to any one of claims 1-4, characterized in that, After performing fusion training on the pre-trained language models, the process also includes: Identify difficult samples whose reward value is lower than a predetermined threshold after the fusion training; Using the difficult samples as training data, the customer service dialogue summary expert model is enhanced and fine-tuned using a loss function incorporating KL divergence constraints.

7. A method for generating customer service dialogue summaries, characterized in that, The method of training a customer service dialogue summary expert model according to any one of claims 1-6 is used, the method comprising: Receive the customer service dialogue text to be processed; input the customer service dialogue text into the customer service dialogue summary expert model to generate the corresponding customer service dialogue summary.

8. The method according to claim 7, characterized in that, The generation of the corresponding customer service dialogue summary includes: Based on a predefined set of summary elements, the elements to be extracted are split into a simple element set and a complex element set; elements in the simple element set are directly extracted using a fast thinking channel, while elements in the complex element set are extracted using a slow thinking channel after thought chain reasoning; the extraction results from the fast thinking channel and the slow thinking channel are merged to obtain the corresponding customer service dialogue summary.

9. The method according to claim 8, characterized in that, The fast thinking channel disables the thought chain reasoning function of the customer service dialogue summary expert model by configuring model generation parameters; The slow thinking channel enables the thought chain reasoning function of the customer service dialogue summary expert model by configuring model generation parameters.

10. The method according to claim 8 or 9, characterized in that, The method further includes: Construct a structured prompt word template, which integrates at least one of the following: thought chain instructions for guiding the model reasoning process, one or more Few-shot examples, hot word prompts for clarifying key expressions, and formatting requirements for standardizing the output format; The step of inputting the customer service dialogue text into the customer service dialogue summary expert model includes: The structured prompt template is combined with the customer service dialogue text to form a combined input; The combination is input into the customer service dialogue summary expert model.

11. A training device for a customer service dialogue summarization model based on fusion training, characterized in that, include: The acquisition module is used to acquire a sample dataset of customer service conversations. The training module is used to perform fusion training on the pre-trained language model using the customer service dialogue sample dataset as training data to obtain a customer service dialogue summary expert model. The fusion training includes: in a single training phase, jointly using a supervised fine-tuning loss function and a reinforcement learning loss function to optimize the pre-trained language model. The supervised fine-tuning loss function enables the pre-trained language model to learn the feature extraction capabilities from the customer service dialogue sample dataset, and the reinforcement learning loss function enables the pre-trained language model to optimize its customer service dialogue summary generation strategy based on reward signals.

12. A customer service dialogue summary generation device, characterized in that, The apparatus, which employs a customer service dialogue summary expert model trained by the method of any one of claims 1-6, comprises: The receiving module is used to receive the customer service dialogue text to be processed; the generating module is used to input the customer service dialogue text into the customer service dialogue summary expert model to generate the corresponding customer service dialogue summary.

13. An electronic device, characterized in that, It includes a processor, a communication interface, a memory, and a communication bus, wherein the processor, the communication interface, and the memory communicate with each other through the communication bus; Memory, used to store computer programs; When a processor executes a program stored in memory, it implements the steps of the customer service dialogue summary model training method based on fusion training as described in any one of claims 1-6 or the customer service dialogue summary generation method as described in any one of claims 7-10.

14. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the steps of the customer service dialogue summary model training method based on fusion training as described in any one of claims 1-6 or the customer service dialogue summary generation method as described in any one of claims 7-10.

Images