Task processing model training, dialogue task processing and virtual character dialogue method

Abstract

This specification provides embodiments of a task processing model training method, a dialogue task processing method, and a virtual character dialogue method. The task processing model training method includes: acquiring sample dialogue parameters; inputting the sample dialogue parameters into an initial processing model to obtain sample dialogue data, and extracting unexpected dialogue data from the sample dialogue data; using the initial processing model to generate expected dialogue data corresponding to the unexpected dialogue data; and training the initial processing model based on the unexpected and expected dialogue data to obtain a task processing model. By automatically generating dialogue data and using it for model training, costs and workload are reduced. Since both the unexpected and expected dialogue data are generated by the model, the model can automatically adjust based on actual performance, achieving continuous improvement. By comparing and learning from the unexpected and expected dialogue data, the model can more accurately understand dialogue intent, improving model performance.

Description

Technical Field

[0001] The embodiments in this specification relate to the field of artificial intelligence technology, and in particular to methods for training task processing models, processing dialogue tasks, and conducting dialogue with virtual characters. Background Technology

[0002] With the development of artificial intelligence technology, dialogue systems (such as chatbots and voice assistants) have been widely used in various scenarios such as customer service, information retrieval, and smart home control. These systems rely on natural language processing technology to parse user input and generate appropriate responses.

[0003] Currently, to improve the performance of dialogue systems, supervised learning based on large-scale datasets can be used. This involves training models with a large amount of manually labeled dialogue data, enabling them to better understand and respond to users. However, the manual labeling process for dialogue data is time-consuming, labor-intensive, and costly, and it is difficult to cover all possible dialogue scenarios, resulting in poor model performance. Therefore, a high-performance model training scheme is urgently needed. Summary of the Invention

[0004] In view of this, embodiments of this specification provide a task processing model training method. One or more embodiments of this specification also relate to a dialogue task processing method, a virtual character dialogue method, an information processing method based on a task processing model, a task processing model training device, a dialogue task processing device, a virtual character dialogue device, an information processing device based on a task processing model, a computing device, an electronic device, a computer-readable storage medium, and a computer program product, to address the technical deficiencies existing in the prior art.

[0005] According to a first aspect of the embodiments of this specification, a method for training a task processing model is provided, comprising:

[0006] Obtain sample dialogue parameters;

[0007] Input the sample dialogue parameters into the initial processing model to obtain sample dialogue data, and extract the unexpected dialogue data from the sample dialogue data.

[0008] Using the initial processing model, generate the expected dialogue data corresponding to the unexpected dialogue data;

[0009] The initial processing model is trained based on the unexpected dialogue data and the expected dialogue data to obtain the task processing model.

[0010] According to a second aspect of the embodiments of this specification, a dialogue task processing method is provided, comprising:

[0011] Obtain the pending dialogue data for the target dialogue task;

[0012] The dialogue data to be processed is input into the task processing model to obtain the target dialogue result. The task processing model is trained based on the task processing model training method.

[0013] According to a third aspect of the embodiments of this specification, a virtual character dialogue method is provided, comprising:

[0014] Receive virtual character dialogue parameters sent by the terminal device;

[0015] The virtual character's dialogue parameters are input into the task processing model to obtain the virtual character's dialogue results. The task processing model is trained based on the task processing model training method.

[0016] The results of the virtual character's dialogue are fed back to the terminal device.

[0017] According to a fourth aspect of the embodiments of this specification, an information processing method based on a task processing model is provided, applied to a task platform, comprising:

[0018] Receive model requests sent by terminal devices;

[0019] Based on the model request, the target task processing model is determined from multiple task processing models, wherein the multiple task processing models are trained based on the task processing model training method.

[0020] According to a fifth aspect of the embodiments of this specification, a task platform is provided, including a request interface and a response unit;

[0021] The request interface is used to receive model requests sent by terminal devices. The model request includes at least one of the following: the scene identifier of the target scene, the scene input data of the target scene, and the model specification parameters.

[0022] The response unit is used to determine the target task processing model from multiple task processing models based on the model request, wherein the multiple task processing models are trained based on the task processing model training method.

[0023] According to a sixth aspect of the embodiments of this specification, a method for training a task processing model is provided, comprising:

[0024] The first acquisition module is configured to acquire sample dialogue parameters;

[0025] The first input module is configured to input sample dialogue parameters into the initial processing model, obtain sample dialogue data, and extract unexpected dialogue data from the sample dialogue data.

[0026] The generation module is configured to use the initial processing model to generate the expected dialogue data corresponding to the unexpected dialogue data.

[0027] The first training module is configured to train the initial processing model based on the unexpected dialogue data and the expected dialogue data to obtain the task processing model.

[0028] According to a seventh aspect of the embodiments of this specification, a dialogue task processing method is provided, comprising:

[0029] The second acquisition module is configured to acquire the dialogue data to be processed for the target dialogue task;

[0030] The second input module is configured to input the dialogue data to be processed into the task processing model to obtain the target dialogue result. The task processing model is trained based on the task processing model training method.

[0031] According to an eighth aspect of the embodiments of this specification, a virtual character dialogue method is provided, comprising:

[0032] The first receiving module is configured to receive virtual character dialogue parameters sent by the terminal device;

[0033] The third input module is configured to input the virtual character dialogue parameters into the task processing model to obtain the virtual character dialogue results. The task processing model is trained based on the task processing model training method.

[0034] The feedback module is configured to send the results of the virtual character's dialogue back to the terminal device.

[0035] According to a ninth aspect of the embodiments of this specification, an information processing method based on a task processing model is provided, applied to a task platform, comprising:

[0036] The second receiving module is configured to receive model requests sent by the terminal device;

[0037] The determination module is configured to determine the target task processing model from multiple task processing models based on a model request, wherein the multiple task processing models are trained based on a task processing model training method.

[0038] According to a tenth aspect of the embodiments of this specification, a computing device is provided, comprising:

[0039] Memory and processor;

[0040] The memory is used to store computer programs / instructions, and the processor is used to execute the computer programs / instructions, which, when executed by the processor, implement the steps of the methods provided in the first, second, third, or fourth aspects described above.

[0041] According to an eleventh aspect of the embodiments of this specification, an electronic device is provided, comprising:

[0042] The memory and processor are connected via a bus;

[0043] The memory is used to store computer programs / instructions, and the processor is used to execute the computer programs / instructions, which, when executed by the processor, implement the steps of the methods provided in the first, second, third, or fourth aspects described above.

[0044] According to a twelfth aspect of the embodiments of this specification, a computer-readable storage medium is provided that stores a computer program / instructions that, when executed by a processor, implement the steps of the methods provided in the first, second, third, or fourth aspects described above.

[0045] According to a thirteenth aspect of the embodiments of this specification, a computer program product is provided, including a computer program / instructions that, when executed by a processor, implement the steps of the methods provided in the first, second, third, or fourth aspects described above.

[0046] This specification provides a task processing model training method according to one embodiment, comprising: acquiring sample dialogue parameters; inputting the sample dialogue parameters into an initial processing model to obtain sample dialogue data, and extracting unexpected dialogue data from the sample dialogue data; using the initial processing model to generate expected dialogue data corresponding to the unexpected dialogue data; and training the initial processing model based on the unexpected dialogue data and the expected dialogue data to obtain a task processing model. By automatically generating dialogue data and using it for model training, the reliance on a large amount of manually labeled dialogue data is reduced, lowering costs and workload. Since both the unexpected and expected dialogue data are generated by the model, the model can automatically adjust based on actual performance, achieving continuous model improvement. By comparing and learning from the unexpected and expected dialogue data, the model can more accurately understand dialogue intent and provide more appropriate and natural responses, improving model performance. Attached Figure Description

[0047] Figure 1 This is a flowchart illustrating a task processing model training method provided in one embodiment of this specification;

[0048] Figure 2 This is a flowchart illustrating the processing procedure of a task processing model training method provided in one embodiment of this specification.

[0049] Figure 3 This is a flowchart of a dialogue task processing method provided in one embodiment of this specification;

[0050] Figure 4 This is an architecture diagram of a dialogue task processing system provided in one embodiment of this specification;

[0051] Figure 5 This is a flowchart illustrating a virtual character dialogue method provided in one embodiment of this specification;

[0052] Figure 6 This is a flowchart illustrating an information processing method based on a task processing model, provided in one embodiment of this specification.

[0053] Figure 7 This is a schematic diagram of the structure of a task platform provided in one embodiment of this specification;

[0054] Figure 8 This is a schematic diagram of the structure of a task processing model training device provided in one embodiment of this specification;

[0055] Figure 9 This is a schematic diagram of the structure of a dialogue task processing device provided in one embodiment of this specification;

[0056] Figure 10 This is a schematic diagram of the structure of a virtual character dialogue device provided in one embodiment of this specification;

[0057] Figure 11 This is a schematic diagram of the structure of an information processing device based on a task processing model provided in one embodiment of this specification;

[0058] Figure 12 This is a structural block diagram of a computing device provided in one embodiment of this specification;

[0059] Figure 13 This is a structural block diagram of an electronic device provided in one embodiment of this specification. Detailed Implementation

[0060] Many specific details are set forth in the following description to provide a full understanding of this specification. However, this specification can be implemented in many other ways than those described herein, and those skilled in the art can make similar extensions without departing from the spirit of this specification. Therefore, this specification is not limited to the specific implementations disclosed below.

[0061] The terminology used in one or more embodiments of this specification is for the purpose of describing particular embodiments only and is not intended to be limiting of the one or more embodiments of this specification. The singular forms “a,” “described,” and “the” as used in one or more embodiments of this specification and the appended claims are also intended to include the plural forms unless the context clearly indicates otherwise. It should also be understood that the term “and / or” as used in one or more embodiments of this specification refers to and includes any or all possible combinations of one or more associated listed items.

[0062] It should be understood that although the terms first, second, etc., may be used to describe various information in one or more embodiments of this specification, such information should not be limited to these terms. These terms are only used to distinguish information of the same type from one another. For example, first may also be referred to as second without departing from the scope of one or more embodiments of this specification, and similarly, second may also be referred to as first. Depending on the context, the word "if" as used herein may be interpreted as "when," "when," or "in response to a determination."

[0063] Furthermore, it should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in one or more embodiments of this specification are all information and data authorized by the user or fully authorized by all parties. Moreover, the collection, use and processing of related data must comply with the relevant laws, regulations and standards of the relevant countries and regions, and corresponding operation entry points are provided for users to choose to authorize or refuse.

[0064] In one or more embodiments of this specification, a large model refers to a deep learning model with a large number of model parameters, typically containing hundreds of millions, tens of billions, hundreds of billions, trillions, or even tens of trillions of model parameters. A large model can also be called a foundational model (Foundation Model 1). It is pre-trained using large-scale unlabeled corpora to produce a pre-trained model with hundreds of millions of parameters. Such models can adapt to a wide range of downstream tasks and have good generalization ability. Examples include Large Language Models (LLMs) and Multi-modal Pre-training Models (MLMs).

[0065] In practical applications, large models only require a small number of samples to fine-tune the pre-trained model before they can be applied to different tasks. Large models can be widely used in fields such as Natural Language Processing (NLP) and Computer Vision. Specifically, they can be applied to computer vision tasks such as Visual Question Answering (VQA), Image Captioning (IC), and Image Generation, as well as NLP tasks such as text-based sentiment classification, text summarization, and machine translation. The main application scenarios for large models include digital assistants, intelligent robots, search, online education, office software, e-commerce, and intelligent design.

[0066] First, the terms and concepts used in one or more embodiments of this specification will be explained.

[0067] Alignment refers to the process of aligning the output of a large model with human preferences. These preferences can refer to the model's correctness, knowledge content, stylization, etc.

[0068] Direct Preference Optimization (DPO) is a method used in reinforcement learning and decision making to optimize policies by directly adjusting the agent's behavior based on preference feedback from humans or other evaluators. Unlike traditional reinforcement learning methods, DPO does not rely on explicit reward functions but instead uses preference information as guidance to improve the policy. The advantage of DPO is that it can bypass the difficulty of designing complex reward functions, especially in environments where rewards are difficult to quantify. Furthermore, it can capture more subtle or subjective preferences, which is crucial in certain applications such as personalized recommendation systems and human-computer collaborative tasks.

[0069] Segment-level direct preference optimization (SDPO) refers to the process of optimizing the initial processing model based on key dialogue segments from the expected dialogue data and unexpected dialogue segments from the unexpected dialogue data.

[0070] Supervised fine-tuning (SFT) is a method of further training a model based on a pre-trained model. In this method, the model is trained on a dataset containing pairs of inputs and desired outputs so that it learns how to generate responses closer to human-level performance. Supervised fine-tuning is often used to adapt a model to task-specific or domain-specific data, thereby improving its performance on those tasks.

[0071] This specification provides a task processing model training method, and also relates to a dialogue task processing method, a virtual character dialogue method, an information processing method based on a task processing model, a task processing model training device, a dialogue task processing device, a virtual character dialogue device, an information processing device based on a task processing model, a computing device, an electronic device, a computer-readable storage medium, and a computer program product, which will be described in detail in the following embodiments.

[0072] See Figure 1 , Figure 1 This specification illustrates a flowchart of a task processing model training method according to an embodiment, which specifically includes the following steps:

[0073] Step 102: Obtain sample dialogue parameters.

[0074] It should be noted that sample dialogue parameters refer to specific data elements used in a dialogue task processing system to describe one or more dialogue interactions between a user and the model. Sample dialogue parameters include at least one of the following: dialogue rounds, dialogue scenarios, dialogue roles, dialogue goals, dialogue sentiment, and historical dialogue data. A dialogue round refers to the number of back-and-forth exchanges during the entire dialogue process. Each user's statement and the subsequent model's response constitute a dialogue round. Dialogue rounds can be divided into single-round dialogues and multi-round dialogues. A single-round dialogue consists of one dialogue round, while a multi-round dialogue consists of multiple dialogue rounds connected sequentially to form continuous dialogue content. A dialogue scenario refers to the specific environment or background conditions in which the dialogue occurs, encompassing factors such as the topic and occasion of the dialogue. It can be a specific physical location (such as a restaurant or office) or an abstract task context (such as booking services or product consultations). A dialogue role refers to the identity or functional positioning of each party participating in the dialogue. Dialogue roles include not only attribute information but also the number of dialogue roles. Dialogue roles include, but are not limited to, users, customer service representatives, and virtual assistants, each with its unique responsibilities and communication style. Dialogue roles define the direction and boundaries of the conversation. For example, in customer service scenarios, the model should demonstrate professionalism and problem-solving skills, while in social chat, the focus might be more on friendliness and entertainment. The dialogue goal refers to the specific purpose or outcome desired through the dialogue. This could be solving a problem, completing a task, or obtaining information. Goals can be unilaterally set (e.g., a user seeking help) or mutually agreed upon (e.g., a buyer and seller negotiating transaction details). Dialogue sentiment refers to the emotional state of the dialogue roles during the conversation, such as happiness, anger, or confusion. Historical dialogue data is the collection of all interactions between the user and the model within a certain time period. Historical dialogue data can be understood as dialogue context data; this data captures past dialogue events and is an important resource for evaluating and improving dialogue task processing systems.

[0075] In practical applications, there are various ways to obtain sample dialogue parameters, and the specific method should be selected according to the actual situation. This specification does not impose any limitations on these methods in its embodiments. In one possible implementation, sample dialogue parameters can be received from a user's terminal device. In another possible implementation, sample dialogue parameters can be read from other data acquisition devices or databases.

[0076] Step 104: Input the sample dialogue parameters into the initial processing model to obtain sample dialogue data, and extract the unexpected dialogue data from the sample dialogue data.

[0077] It's important to clarify that the initial processing model refers to the original machine learning or deep learning model used at the beginning of the training process. The initial processing model can be an untrained original model or a model that has undergone preliminary training but whose performance needs further improvement. Examples of initial processing models include large models and large models in the dialogue domain. The initial processing model is used to parse sample dialogue parameters and generate sample dialogue data. Sample dialogue data refers to the set of dialogue instances generated by the initial processing model based on the sample dialogue parameters. Sample dialogue data can be one or multiple. Each sample dialogue data instance can be a single-turn dialogue instance or a multi-turn dialogue instance. Undesirable dialogue data refers to sample dialogue data that fails to achieve the expected results, i.e., sample dialogue data that does not meet the user's expectations or contains obvious errors. Undesirable dialogue data can be understood as negative examples or unfavorable dialogue data during model training. For example, suppose there are two sample dialogue data sets, sample dialogue data 1 and sample dialogue data 2. If sample dialogue data 1 meets the user's expectations, while sample dialogue data 2 does not, then sample dialogue data 2 is undesirable dialogue data.

[0078] In practical applications, there are multiple ways to input sample dialogue parameters into the initial processing model to obtain sample dialogue data. The specific method chosen depends on the actual situation, and this specification does not impose any limitations on this approach. In one possible implementation, the sample dialogue parameters can be directly input into the initial processing model to obtain sample dialogue data. In another possible implementation, to ensure that the task processing model is applicable to complex multi-turn interaction application scenarios (such as role-playing agents or intelligent virtual assistants), multi-turn dialogue prompts and sample dialogue parameters can be input into the initial processing model to obtain sample dialogue data. The multi-turn dialogue prompts are used to guide the initial processing model to conduct multi-turn dialogues based on the sample dialogue parameters.

[0079] Furthermore, there are various methods for extracting unexpected dialogue data from sample dialogue data, and the specific method chosen depends on the actual situation. This specification does not limit the methods used in this embodiment. In one possible implementation, unexpected dialogue keywords can be obtained, and the sample dialogue data can be matched with these keywords to identify the sample dialogue data containing the unexpected keywords as unexpected dialogue data. In another possible implementation, dialogue-level quality checks can be performed on each sample dialogue data set, and unexpected dialogue data can be extracted from the sample dialogue data based on the quality check results.

[0080] In one optional embodiment of this specification, the extraction of unwanted dialogue data from sample dialogue data may include the following steps:

[0081] Input the quality inspection prompts and sample dialogue data into the data processing model to obtain the second quality index;

[0082] Based on the second quality metric, unwanted dialogue data is filtered out from the sample dialogue data.

[0083] It's important to note that quality assessment prompts refer to information used to guide or trigger the data processing model to perform quality assessments on the sample dialogue data. Quality assessment checks whether the sample dialogue data achieves the dialogue objectives. Quality assessment prompts can include information such as sample dialogue parameters and quality assessment rules (e.g., rules for assessing dialogue fluency, accuracy, etc.). For example, a quality assessment prompt might be: "The dialogue objective of the sample dialogue data generated by the initial processing model is [dialogue objective]. Please provide a comprehensive analysis explaining the extent to which the initial processing model achieved these objectives, and detail the logic or thought process behind the conclusions. Additionally, please provide a second quality metric (integer score) from 0 to 10, where 0 represents the minimum degree of objective achievement, 10 represents complete objective achievement, and a higher score indicates greater progress made by the initial processing model in achieving the dialogue objectives." The data processing model refers to the algorithm or machine learning model used to process and analyze the input data (here, the quality assessment prompts and the sample dialogue data). The data processing model is capable of parsing the input data, performing specific tasks (e.g., calculating quality metrics), and outputting results. The data processing model can be a large model or a deep learning model trained based on training prompts, training dialogue data, and training quality metrics. The second quality metric is a set of indicators calculated by the data processing model based on quality detection prompts and sample dialogue data to measure dialogue quality. The second quality metric can reflect the performance of the sample dialogue data in specific dimensions, such as goal achievement, accuracy, relevance, and naturalness. The second quality metric can be a specific quality score, such as 8 points, or a quality level, such as high quality or low quality.

[0084] In practical applications, there are multiple ways to filter out unwanted dialogue data from sample dialogue data based on the second quality index. The specific method chosen depends on the actual situation, and this specification does not limit this approach. In one possible implementation, sample dialogue data with a second quality index lower than a first preset threshold can be identified as unwanted dialogue data. The first preset threshold is set according to the actual situation. In another possible implementation, the sample dialogue data can be sorted from largest to smallest based on the second quality index, and the last N sample dialogue data points are identified as unwanted dialogue data. Here, N is a positive integer and is set according to the actual situation.

[0085] For example, suppose the sample dialogue parameters include the following: Dialogue scenario: A and B are friends, and they are currently in a very cold house. It's snowing heavily outside. Dialogue characters: A and B. Dialogue goals: A's goal: to persuade B to share the blanket. B's goal: You have a blanket and want to use it alone. Historical dialogue data: A: It's so cold! Is there a chance I can use your blanket? B: Uh. I'm cold too. I think I might need this blanket more. Input the sample dialogue parameters into the initial processing model to obtain sample dialogue data. Taking the sample dialogue data as single-turn dialogue data as an example, sample dialogue data 1 is "A: Okay, sorry to bother you." Sample dialogue data 2 is "A: Then let's share this blanket." Input the quality detection prompt and sample dialogue data 1 into the data processing model, and the second quality index 1 is obtained as 2 points. Input the quality detection prompt and sample dialogue data 2 into the data processing model, and the second quality index 2 is obtained as 6 points. Then, the unexpected dialogue data is determined to be sample dialogue data 1. Taking multi-turn dialogue data as an example, using the same logic as single-turn dialogue data, we can determine the unexpected dialogue data as follows: "A: It's so cold! May I use your blanket? B: Uh. I'm cold too. I think I might need this blanket more. A: Then can we share this blanket? It can warm us both up. B: Sorry, I feel a little uncomfortable being too close to you. A: Okay, I guess I'll just have to try wearing all my clothes."

[0086] By applying the scheme of the embodiments in this specification, a second quality index for each sample dialogue data is generated based on a data processing model, thereby achieving an assessment of whether the sample dialogue data can achieve the dialogue goal, and thus obtaining highly accurate undesired dialogue data.

[0087] In one optional embodiment of this specification, the initial processing model is trained based on reference dialogue parameters and reference dialogue data corresponding to the reference dialogue parameters. That is, before inputting the sample dialogue parameters into the initial processing model to obtain the sample dialogue data, the following steps may also be included:

[0088] Retrieve reference dialogue parameters and the corresponding reference dialogue data;

[0089] Input the reference dialogue parameters into the original processing model to obtain the predicted dialogue data;

[0090] The original processing model is trained based on the reference dialogue data and the predicted dialogue data to obtain the initial processing model.

[0091] It's important to note that reference dialogue parameters refer to the specific data elements involved in the dialogue interaction during the generation of reference dialogue data. Reference dialogue data refers to the set of dialogue instances generated based on the reference dialogue parameters. There can be one or multiple reference dialogue instances. Each reference dialogue instance can be a single-turn or multi-turn dialogue instance. The original processing model refers to the initial dialogue processing algorithm or machine learning model before any optimization or adjustment. Predicted dialogue data refers to the dialogue output generated by the original processing model after the reference dialogue parameters are input. By comparing the predicted dialogue data with the reference dialogue data, the performance of the original processing model can be evaluated, and areas for improvement can be identified.

[0092] For example, the reference dialogue parameters include the following: Dialogue scenario: Two people are discussing their frustration with the work attitude and behavior of a third party. Dialogue roles: A and B. Dialogue goals: A's goal: To find an effective way to change the third party. B's goal: To guide A to express their feelings constructively and avoid escalating the conflict. Taking multi-turn dialogue data as an example, the reference dialogue data is: "A: Recently, C's behavior has really affected the team atmosphere, and I think we need to do something. B: Yes, it's really frustrating. How about we talk to him first to understand the reasons behind it? A: Good idea. We can find a relaxed time to talk to him instead of confronting him directly. B: Yes, for example, during lunch. Everyone can communicate more openly. A: Agreed. We can also discuss ways to improve together. B: Okay, I'll arrange it. Hopefully, this conversation will improve the situation."

[0093] In practical applications, the implementation method for "obtaining reference dialogue data and the reference dialogue parameters corresponding to the reference dialogue data" can refer to the implementation method for "obtaining sample dialogue parameters" described above, and will not be repeated in this embodiment. The process of training the original processing model based on the reference dialogue data and the predicted dialogue data to obtain the initial processing model is supervised fine-tuning. Specifically, the model prediction loss value can be calculated based on the reference dialogue data and the predicted dialogue data, and the model parameters of the original processing model can be adjusted based on the model prediction loss value until a preset stopping condition is reached to obtain the initial processing model. There are many functions for calculating the model prediction loss value, such as the cross-entropy loss function, the L1 norm loss function, the maximum loss function, the mean squared error loss function, and the logarithmic loss function. The specific function to be selected depends on the actual situation, and this embodiment does not impose any limitations on this. The preset stopping condition includes, but is not limited to, the model prediction loss value being less than or equal to a preset threshold, or the number of iterations reaching a preset number of iterations. The preset threshold and the preset number of iterations are selected based on the actual situation, and this embodiment does not impose any limitations on this.

[0094] By applying the solutions in the embodiments of this specification, and comparing the reference dialogue data (ideal output) with the predicted dialogue data (model-generated output), the shortcomings of the model in understanding and generating dialogue can be identified, thereby allowing the model parameters to be adjusted in a targeted manner, enabling the initial processing model to more accurately capture the user's intent and provide an appropriate response.

[0095] Step 106: Using the initial processing model, generate the expected dialogue data corresponding to the unexpected dialogue data.

[0096] It's important to note that expected dialogue data refers to ideal, predictable dialogue data tailored to the sample dialogue parameters. Expected dialogue data can be understood as positive or preferred dialogue data during model training. It can be single-turn or multi-turn dialogue data. While sample dialogue data other than the expected dialogue data can also be used as expected dialogue data, since this data may already be very good with limited room for further improvement, the expected dialogue data corresponding to the non-expected dialogue data can be generated through initial model processing. Compared to directly using sample dialogue data other than the non-expected dialogue data as expected dialogue data, generating expected dialogue data through a secondary model process allows for focused optimization of poorly performing (i.e., non-expected) dialogue instances, enabling targeted improvements and making the training process more efficient.

[0097] For example, when referencing the above unexpected dialogue data "A: Okay, sorry to bother you", the corresponding expected dialogue data could be "A: Can we share this blanket? It can keep us both warm". Using the above unexpected dialogue data as an example, "A: It's so cold! May I use your blanket? B: Uh. I'm cold too. I think I might need this blanket more. A: Then can we share this blanket? It can warm us both up. B: Sorry, I feel a little uncomfortable being too close to you. A: Okay, I guess I'll just have to try wearing all my clothes," the expected dialogue data could be: "A: It's so cold! My body is shivering. Even though I'm wearing all my clothes, the cold wind still seeps in like knives. B: It's really cold, the temperature is super low. A: I miss my mom so much, I'm so scared, I feel like I won't make it through the snow. B: Don't say that, we'll definitely get back safely. Here, let's share this blanket. A: I don't know how to thank you! B: Let's go home as soon as the snow stops."

[0098] In practical applications, there are various ways to generate expected dialogue data corresponding to unexpected dialogue data using the initial processing model. The specific method chosen depends on the actual situation, and the embodiments in this specification do not impose any limitations on this. In one possible implementation of this specification, the unexpected dialogue data can be directly input into the initial processing model to obtain the expected dialogue data output by the initial processing model. In another possible implementation of this specification, if the unexpected dialogue data is multi-turn dialogue data, the complete unexpected dialogue data may be quite long, exceeding the model input length of the initial processing model. Therefore, the dialogue segment to be optimized can be determined from the unexpected dialogue data, and the expected dialogue data corresponding to the unexpected dialogue data can be generated based on the historical dialogue segments of the dialogue segment to be optimized using the initial processing model.

[0099] In one optional embodiment of this specification, the unwanted dialogue data is multi-turn dialogue data; the above-mentioned generation of desired dialogue data corresponding to the unwanted dialogue data using the initial processing model may include the following steps:

[0100] Analyze unexpected dialogue data to obtain dialogue segments to be optimized, and extract historical dialogue segments of the dialogue segments to be optimized from the unexpected dialogue data.

[0101] Input historical dialogue fragments into the initial processing model to obtain the expected dialogue data corresponding to the unexpected dialogue data.

[0102] It should be noted that the dialogue segment to be optimized refers to the dialogue content of dialogue turns in the unexpected dialogue data where there are problems with the interaction or there is still room for improvement. The historical dialogue segment of the dialogue segment to be optimized refers to the dialogue content generated before the dialogue segment to be optimized. For example, taking the above unexpected dialogue data "A: It's so cold! May I use your blanket? B: Uh. I'm cold too. I think I might need this blanket more. A: Then can we share this blanket? It can warm us both up. B: Sorry, I feel a little uncomfortable being too close to you. A: Okay, I think I'll just try to wear all my clothes" as an example, assuming the dialogue segment to be optimized is "A: Okay, I think I'll just try to wear all my clothes", then the historical dialogue segment is "A: It's so cold! May I use your blanket? B: Uh. I'm cold too. I think I might need this blanket more. A: Then can we share this blanket? It can warm us both up. B: Sorry, I feel a little uncomfortable being too close to you".

[0103] In practical applications, there are various methods for parsing unexpected dialogue data to obtain dialogue segments to be optimized. The specific method chosen depends on the actual situation, and the embodiments in this specification do not impose any limitations on this. In one possible implementation of this specification, parsing rules for unexpected dialogue data can be obtained. These parsing rules are used to identify and extract problem points in the unexpected dialogue, such as rules based on keywords, sentence structure, or specific error patterns. Dialogue segments to be optimized are then extracted from the unexpected dialogue data through rule matching. In another possible implementation of this specification, a data processing model can be used to extract the dialogue segments to be optimized.

[0104] Furthermore, there are multiple ways to input historical dialogue fragments into the initial processing model to obtain the expected dialogue data corresponding to the unexpected dialogue data. The specific method chosen depends on the actual situation, and this specification does not impose any limitations on this approach. In one possible implementation, historical dialogue fragments can be directly input into the initial processing model to obtain the expected dialogue data corresponding to the unexpected dialogue data in one go. In another possible implementation, the initial processing model can be used to sample new interaction paths multiple times based on historical dialogue fragments, selecting the path with the highest score as the expected dialogue data.

[0105] By applying the scheme of the embodiments in this specification, the historical dialogue fragments of the dialogue segment to be optimized are input into the initial processing model to obtain the expected dialogue data corresponding to the unexpected dialogue data, thereby reducing the input length of the initial processing model and ensuring the stability of the expected dialogue data generation process.

[0106] In one optional embodiment of this specification, the above-described parsing of unwanted dialogue data to obtain the dialogue segment to be optimized may include the following steps:

[0107] Obtain the parsing rules for unexpected dialogue data, and construct parsing prompts based on the parsing rules;

[0108] The parsed prompts and unexpected dialogue data are input into the data processing model to extract the segments to be optimized, thus obtaining the dialogue segments to be optimized.

[0109] It's important to clarify that a data processing model refers to an algorithm or machine learning model used to process and analyze input data (here, based on parsing prompts, analyzing dialogue fragments in unexpected dialogue data and extracting the dialogue fragments to be optimized). A data processing model can parse input data, perform specific tasks (such as extracting the dialogue fragments to be optimized), and output results. A data processing model can be a large model or a deep learning model trained on optimized dialogue fragments, training prompts, and unexpected training dialogue data. Parsing rules are a set of predefined standards or patterns used to identify specific problem points or areas for improvement from unexpected dialogue data, i.e., the dialogue fragments to be optimized. These rules can be based on keywords, syntactic structure, semantic features, etc. For example, parsing rules include, but are not limited to, the following: Among all responses, the response to the dialogue fragment to be optimized is relatively critical to achieving the goal. The current response does not achieve the goal well enough, or there is room for improvement in achieving the goal better. There is room for improvement to enhance the relationship between the two parties in the dialogue, without hindering the achievement of the goal. Parsing prompts refer to model guidance information generated according to parsing rules, which can help the data processing model more accurately understand and process unexpected dialogue data. For example, the parsing prompt could be: Given the following dialogue content (displayed in JSON format), which includes the dialogue context, dialogue characters, dialogue goal, and specific dialogue content. {Dialogue History}. Please select the response that meets the parsing rules from all responses of the initial processing model according to the following conditions [parsing rules]. Please output the selected round index and the reason for selection in JSON format, as shown below: {"index":[index],"reason":[reason]}. The output pattern of the data processing model can be as follows: {"properties":{"index":{"description":index of the response selected from all responses of the initial processing model","title":"index","type":"integer"},"reason":{"description":"reason","type":"string"}},"required":["index","reason"]}.

[0110] The solution implemented in this specification uses a data processing model to label unwanted dialogue data and select dialogue segments to be optimized from the unwanted dialogue data, thereby improving the efficiency and accuracy of determining dialogue segments to be optimized.

[0111] In one optional embodiment of this specification, the above-mentioned inputting historical dialogue fragments into the initial processing model to obtain the expected dialogue data corresponding to the unexpected dialogue data may include the following steps:

[0112] The historical dialogue fragments were input into the initial processing model multiple times to obtain multiple candidate dialogue data, wherein the number of candidate dialogue data was the same as the number of times the model was input;

[0113] Quality checks are performed on multiple candidate dialogue data to obtain a first quality index, wherein the first quality index corresponds one-to-one with the candidate dialogue data;

[0114] Based on the first quality metric, the expected dialogue data corresponding to the undesired dialogue data is selected from multiple candidate dialogue data.

[0115] It should be noted that the number of candidate dialogue data is the same as the number of times the model is input. For example, the first time a historical dialogue fragment is input into the initial processing model, candidate dialogue data 1 is obtained; the second time, the historical dialogue fragment is input into the initial processing model, candidate dialogue data 2 is obtained. Quality detection is used to determine whether the candidate dialogue data can achieve the dialogue objective corresponding to the historical dialogue fragment. The primary quality metric reflects the performance of the candidate dialogue data on specific dimensions, such as objective achievement, accuracy, relevance, and naturalness. The primary quality metric can be a specific quality score, such as 8 points, or a quality level, such as high quality or low quality.

[0116] In practical applications, there are various ways to perform quality checks on multiple candidate dialogue data to obtain a first quality index. The specific method chosen depends on the actual situation, and the embodiments in this specification do not impose any limitations on this. One possible implementation of this specification involves using a predefined series of rules to perform quality checks on multiple candidate dialogue data to obtain a first quality index. These rules can be based on keywords, sentence structure, semantic features, etc. Another possible implementation of this specification involves inputting quality check prompts and any candidate dialogue data into a data processing model to obtain the first quality index of that candidate dialogue data.

[0117] Furthermore, based on the first quality index, there are multiple ways to filter out the desired dialogue data corresponding to the undesired dialogue data from multiple candidate dialogue data. The specific method chosen depends on the actual situation, and this specification does not limit this approach. In one possible implementation, candidate dialogue data whose first quality index is greater than a second preset threshold can be determined as desired dialogue data, where the second preset threshold is set according to the actual situation. In another possible implementation, candidate dialogue data can be sorted from largest to smallest based on the first quality index, and the top N candidate dialogue data are determined as desired dialogue data, where N is a positive integer and is set according to the actual situation.

[0118] The solution implemented in this specification uses a first quality index to filter out the desired dialogue data corresponding to the undesired dialogue data from multiple candidate dialogue data, thus ensuring the quality of the desired dialogue data.

[0119] Step 108: Train the initial processing model based on the unexpected dialogue data and the expected dialogue data to obtain the task processing model.

[0120] It should be noted that the task processing model is the initial processing model that has been trained. The task processing model is suitable for single-turn dialogue processing tasks, but can also be applied to multi-turn dialogue processing tasks. The process of training the initial processing model based on unexpected and expected dialogue data is a DPO-based training process.

[0121] In practical applications, there are various methods for training an initial processing model based on unexpected and expected dialogue data to obtain a task processing model. The specific method chosen depends on the actual situation, and this specification does not impose any limitations on this approach. In one possible implementation, the model preference loss can be directly calculated based on the unexpected and expected dialogue data. The model parameters of the initial processing model can then be adjusted based on this preference loss to obtain the task processing model. In another possible implementation, when both the unexpected and expected dialogue data are multi-turn dialogue data, which are typically quite long, to improve the efficiency of the training process, key dialogue segments in the expected dialogue data can be identified. Unexpected dialogue segments can then be extracted from the unexpected dialogue data based on these key segments. The initial processing model can then be trained using these key and unexpected dialogue segments to obtain the task processing model.

[0122] The solution described in this specification reduces reliance on large amounts of manually labeled dialogue data by automatically generating dialogue data for model training, thus lowering costs and workload. Since both unwanted and desired dialogue data are generated by the model, the model can automatically adjust based on actual performance, enabling continuous model improvement. By comparing and learning from unwanted and desired dialogue data, the model can more accurately understand dialogue intent and provide more appropriate and natural responses, thereby improving model performance.

[0123] In one optional embodiment of this specification, training the initial processing model based on the unexpected dialogue data and the expected dialogue data to obtain the task processing model may include the following steps:

[0124] Calculate the model preference loss based on the unexpected dialogue data and the expected dialogue data;

[0125] Based on the model preference loss, the model parameters of the initial processing model are adjusted to obtain the task processing model.

[0126] It should be noted that when calculating the model preference loss based on the unexpected and expected dialogue data, the model preference loss can be calculated according to the data types (single-turn dialogue data type or multi-turn dialogue data type) of the unexpected and expected dialogue data. Specifically, when the unexpected and expected dialogue data are single-turn dialogue data, the model preference loss can be calculated using the following formula (1). When both the unexpected and expected dialogue data are multi-turn dialogue data, the model preference loss can be calculated using the following formula (2):

[0127]

[0128]

[0129] Among them, L DPO Let E represent the model preference loss for a single-turn dialogue, e represent the expected value, and h represent the turn number for locating the dialogue segment (error) to be optimized. e y represents the historical dialogue segment to be optimized, and y represents the model's response. Indicates the desired dialogue data, Let D represent the unexpected dialogue data, D represent the set of model training data, σ represent the computation function, β represent the hyperparameters, and π represent the hyperparameters. θ This represents the initial processing model for iterative optimization, π. ref L represents a fixed, untrained initial processing model. SDPO τ represents the model preference loss for multi-turn dialogue data. w Indicates key dialogue segments, τ lLet represent the unexpected dialogue segment, k represent the dialogue several rounds after the round in which the dialogue segment to be optimized is located, and t represent the t-th round of dialogue after the round in which the dialogue segment to be optimized is located. This represents a key dialogue segment in round t. This represents an unexpected dialogue segment in round t. This represents a historical dialogue segment representing a key dialogue segment in round t. This represents a historical dialogue segment from the t-th round of undesired dialogue.

[0130] By applying the solutions in the embodiments of this specification and comparing and learning from unexpected and expected dialogue data, the model can more accurately understand the dialogue intent and provide more appropriate and natural responses, thereby improving model performance.

[0131] In one optional embodiment of this specification, both the unexpected dialogue data and the expected dialogue data are multi-turn dialogue data; the above-mentioned training of the initial processing model based on the unexpected dialogue data and the expected dialogue data to obtain the task processing model may include the following steps:

[0132] By comparing unexpected dialogue data and expected dialogue data, key dialogue segments in the expected dialogue data are identified, where the key dialogue segments make the quality of the expected dialogue data higher than that of the unexpected dialogue data.

[0133] Based on key dialogue fragments, extract unexpected dialogue fragments from unexpected dialogue data;

[0134] The initial processing model is trained based on key dialogue segments and unexpected dialogue segments to obtain the task processing model.

[0135] It's important to note that key dialogue segments refer to crucial dialogue intervals within the expected dialogue data. A key dialogue segment can be a single round of dialogue or multiple rounds of dialogue. Key dialogue segments can be understood as expected dialogue segments; by using key dialogue segments, given identical historical dialogue segments, the interaction trajectory of the expected dialogue data can be optimized compared to that of the unexpected dialogue data. The number of rounds in the unexpected dialogue segments is the same as the number of rounds in the key dialogue segments. For example, if the key dialogue segments are the third and fourth rounds of dialogue in the expected dialogue data, then the unkey dialogue segments are the third and fourth rounds of dialogue in the unexpected dialogue data.

[0136] In practical applications, there are multiple ways to identify key dialogue segments in the expected dialogue data by comparing unexpected and expected dialogue data. The specific method chosen depends on the actual situation, and this specification does not impose any limitations on these methods. In one possible implementation, dialogue segments in the expected dialogue data that show significant differences from the unexpected dialogue data can be directly identified as key dialogue segments. In another possible implementation, the data processing model can be trained on an initial processing model based on key and unexpected dialogue segments to obtain a task processing model.

[0137] Furthermore, the implementation method of "training the initial processing model based on key dialogue segments and unexpected dialogue segments to obtain the task processing model" can refer to the above implementation method of "training the initial processing model based on unexpected dialogue data and expected dialogue data to obtain the task processing model", and will not be repeated in the embodiments of this specification.

[0138] By applying the scheme of the embodiments in this specification, the initial processing model is trained based on key dialogue segments and unwanted dialogue segments to obtain a task processing model. While ensuring the quality of key dialogue segments and unwanted dialogue segments, the key dialogue segments and unwanted dialogue segments are made more concise, thereby improving the model training efficiency.

[0139] In one optional embodiment of this specification, the above-described comparison of unexpected dialogue data and expected dialogue data to determine key dialogue segments in the expected dialogue data may include the following steps:

[0140] The fragment extraction prompts, unexpected dialogue data, and expected dialogue data are input into the data processing model to extract key dialogue fragments, thereby obtaining key dialogue fragments from the expected dialogue data.

[0141] It's important to clarify that a data processing model refers to an algorithm or machine learning model used to process and analyze input data (here, based on fragment extraction prompts and unexpected dialogue data, analyzing each dialogue fragment in the expected dialogue data, and extracting key dialogue fragments from the expected dialogue data). A data processing model can parse input data, perform specific tasks (such as extracting key dialogue fragments from the expected dialogue data), and output results. A data processing model can be a large model or a deep learning model trained on key dialogue fragments, training prompts, expected training dialogue data, and unexpected training dialogue data. Fragment extraction prompts are used to guide the data processing model to extract key dialogue fragments from the expected dialogue data based on unexpected dialogue data. In short, a data processing model refers to an algorithm or machine learning model used to process and analyze input data (here, fragment extraction prompts, unexpected dialogue data, and expected dialogue data).

[0142] The fragment extraction prompts are constructed based on fragment extraction rules. Fragment extraction rules are a set of predefined criteria or patterns used to extract key dialogue fragments from desired dialogue data. These rules can be based on keywords, syntactic structure, semantic features, etc. For example, fragment extraction rules include, but are not limited to, the following: The interval starts at an index. The interval ends at the turn in which the initial processing model speaks. This interval is the reason why the dialogue is better than the original dialogue, achieving a higher goal completion rate or strengthening the relationship between participants. For example, the fragment extraction prompts could be: Given two dialogue data represented in JSON format, including the dialogue scenario, dialogue roles, dialogue goals, and dialogue content. Original dialogue: [Undesired dialogue data]. Better dialogue (a dialogue with higher goal completion or strengthened relationship between participants than the original dialogue): [Desired dialogue data]. Please select a closed interval from the better dialogue that meets the following criteria: Note that the interval should only include key content that affects goal completion or the relationship between the parties! The closed interval can contain one or more turns. Please output the selected closed interval and its reason for selection in the following JSON format: {"start_index":[starting index of the interval],"end_index":[ending index of the interval],"reason":[reason]}. The output model of the data processing model can be as follows: {"properties":{"start_index":{"description":"starting index of the interval","title":"start_index","type":"integer"},"end_index":{"description":"ending index of the interval","title":"end_index","type":"integer"},"reason":{"description":"reason","type":"string"}},"required":["start_index","end_index","reason"]}.

[0143] The scheme of the embodiments of this specification is applied so that both the expected dialogue data and the desired dialogue data are input into the data processing model. The model selects key dialogue segments from the desired dialogue data and then takes dialogue segments of the same round from the expected dialogue data as the expected dialogue segments. Based on the key dialogue segments and the expected dialogue segments, positive and negative sample pairs are constructed in the DPO process, which ensures the accuracy of the positive and negative sample pairs.

[0144] In one optional embodiment of this specification, since the unwanted dialogue data is selected from the sample dialogue data generated by the initial processing model, and the desired dialogue data is also generated by the initial processing model, there may be a problem that the desired dialogue data achieves the dialogue goal to a lesser extent than the unwanted dialogue data, leading to model training errors. Therefore, before model training, the quality of the unwanted dialogue data and the desired dialogue data can be checked separately. Model training is performed when the quality of the desired dialogue data is greater than that of the unwanted dialogue data. That is, before training the initial processing model based on the unwanted dialogue data and the desired dialogue data to obtain the task processing model, the following steps may also be included:

[0145] The third quality indicator is obtained by performing quality checks on the unexpected dialogue data, and the fourth quality indicator is obtained by performing quality checks on the expected dialogue data.

[0146] Training an initial processing model based on both unexpected and expected dialogue data to obtain a task processing model can include the following steps:

[0147] When the fourth quality metric is greater than the third quality metric, the initial processing model is trained based on the unexpected dialogue data and the expected dialogue data to obtain the task processing model.

[0148] It should be noted that quality checks are used to assess whether unexpected or expected dialogue data achieves the corresponding dialogue objective. The third quality metric reflects the performance of unexpected dialogue data across specific dimensions, such as objective achievement, accuracy, relevance, and naturalness. The fourth quality metric reflects the performance of expected dialogue data across specific dimensions, such as objective achievement, accuracy, relevance, and naturalness. Both the third and fourth quality metrics can be specific quality scores, such as 8 points, or quality levels, such as high quality or low quality.

[0149] In practical applications, the implementation method of "performing quality checks on non-expected dialogue data to obtain a third quality indicator and performing quality checks on expected dialogue data to obtain a fourth quality indicator" can refer to the implementation method of "performing quality checks on multiple candidate dialogue data to obtain a first quality indicator" described above. The embodiments in this specification will not be repeated here.

[0150] Furthermore, if the fourth quality metric is greater than the third quality metric, it indicates that the expected dialogue data achieves the dialogue goal to a greater extent than the unexpected dialogue data. Therefore, the initial processing model can be trained using both the expected and unexpected dialogue data to obtain the task processing model. If the fourth quality metric is less than the third quality metric, it indicates that the expected dialogue data achieves the dialogue goal to a lesser extent than the unexpected dialogue data. Therefore, the model cannot be trained based on the unexpected dialogue data. Thus, the unexpected dialogue data can be discarded, and the process can return to the steps of inputting sample dialogue parameters into the initial processing model to obtain sample dialogue data, and then extracting the unexpected dialogue data from the sample dialogue data.

[0151] By applying the scheme of the embodiments in this specification, the initial processing model is trained using unwanted dialogue data and desired dialogue data where the fourth quality index is greater than the third quality index, thereby obtaining the task processing model. This ensures the effectiveness of the model training data and improves the model training efficiency and the performance of the task processing model.

[0152] In modern artificial intelligence research and applications, the dialogue capability of language models has become one of the key factors in evaluating system performance. With the rapid development of large-scale model technology, deep learning-based chatbots have been widely used in many fields. Improving the dialogue continuity, contextual understanding, and multi-turn interaction capabilities of chatbots remains a challenging task. Traditional DPO algorithms are mainly optimized for single-turn dialogues, lacking a multi-turn adjustment mechanism in complex scenarios requiring multi-turn dialogues. This makes traditional DPO algorithms prone to gradually deviating from the ideal dialogue path after initial turn alignment. Therefore, this specification proposes a scheme to extend the DPO algorithm from single-turn constraints to multi-turn, and to train a task processing model based on multi-turn dialogue alignment data. See also Figure 2 , Figure 2 This specification shows a flowchart of a task processing model training method according to an embodiment. The task processing model training process can be divided into the following stages: initial processing model training, sample dialogue data generation, screening of unwanted dialogue data, determination of dialogue segments to be optimized, generation of desired dialogue data, quality detection and judgment, extraction of key dialogue segments, construction of model training data, and model training. The above stages will be described in detail below.

[0153] Initial processing model training: Obtain reference dialogue parameters and corresponding reference dialogue data; input the reference dialogue parameters into the original processing model to obtain predicted dialogue data; train the original processing model based on the reference dialogue data and predicted dialogue data to obtain the initial processing model.

[0154] Sample dialogue data generation: Obtain sample dialogue parameters; input the sample dialogue parameters into the initial processing model to generate multiple rounds of dialogue and obtain sample dialogue data.

[0155] Unexpected dialogue data filtering: Input the quality detection prompts and sample dialogue data into the data processing model to obtain the second quality index; based on the second quality index, filter out the unexpected dialogue data from the sample dialogue data.

[0156] Determining the dialogue segment to be optimized: Obtain the parsing rules for the unexpected dialogue data, and construct parsing prompts based on the parsing rules; input the parsing prompts and unexpected dialogue data into the data processing model to extract the dialogue segment to be optimized.

[0157] Expected dialogue data generation: Input historical dialogue fragments of the dialogue segment to be optimized into the initial processing model multiple times to obtain multiple candidate dialogue data, wherein the number of candidate dialogue data is the same as the number of times the model is input; perform quality detection on multiple candidate dialogue data to obtain a first quality index, wherein the first quality index corresponds one-to-one with the candidate dialogue data; based on the first quality index, filter out the expected dialogue data corresponding to the non-expected dialogue data from the multiple candidate dialogue data.

[0158] Quality inspection and judgment: Perform quality inspection on the unexpected dialogue data to obtain the third quality index, and perform quality inspection on the expected dialogue data to obtain the fourth quality index; determine whether the fourth quality index is greater than the third quality index. If yes, proceed to the key dialogue segment extraction stage; otherwise, discard the unexpected dialogue data.

[0159] Key dialogue fragment extraction: By comparing the unexpected dialogue data and the expected dialogue data, key dialogue fragments are identified in the expected dialogue data. These key dialogue fragments result in the expected dialogue data having a higher quality than the unexpected dialogue data.

[0160] Model training data construction: Based on key dialogue segments, extract unwanted dialogue segments from unwanted dialogue data; determine the key dialogue segments (positive examples) and unwanted dialogue segments (negative examples) as model training data.

[0161] Model training: Calculate the model preference loss based on key dialogue segments and unwanted dialogue segments; adjust the model parameters of the initial processing model based on the model preference loss to obtain the task processing model.

[0162] By applying the scheme of the embodiments in this specification, through multi-round model interaction, data with poor model responses are selected and their output is used as negative examples. Then, given the same question, the model output is sampled multiple times, and the better responses are selected as positive examples. The positive and negative examples are aligned with corresponding key intervals (containing multiple rounds) to form partial order pairs, which explicitly constrains the trajectory of the model after the incorrect round, increases the probability of the model generating positive example trajectories, and reduces the probability of the model generating negative example trajectories. The alignment range is expanded from a single round to multiple rounds, thereby significantly improving the performance of multi-round model interaction.

[0163] See Figure 3 , Figure 3 This specification shows a flowchart of a dialogue task processing method according to an embodiment, which specifically includes the following steps:

[0164] Step 302: Obtain the dialogue data to be processed for the target dialogue task.

[0165] Step 304: Input the dialogue data to be processed into the task processing model to obtain the target dialogue result. The task processing model is trained based on the task processing model training method.

[0166] It's important to clarify that the target dialogue task refers to the specific task or objective that the dialogue task processing system aims to accomplish. A target dialogue task can be answering a user's question, providing a service, or performing a specific operation, such as helping a user book a restaurant, checking the weather, or guiding a user through an online shopping process. Target dialogue tasks can be of different types, such as single-turn or multi-turn dialogue tasks. They can also be dialogue tasks in different scenarios, such as those in intelligent customer service, emotional dialogue, or virtual role-playing scenarios. The dialogue data to be processed refers to the specific dialogue instances that will be input into the task processing model for processing. This data includes, but is not limited to, the dialogue parameters of the target dialogue task, user input, and dialogue context information. The task processing model refers to a trained and optimized dialogue processing algorithm or machine learning model. This model is capable of understanding and generating high-quality dialogue content that meets the requirements of a specific dialogue task. The task processing model is trained on an initial processing model based on both unexpected and expected dialogue data. The expected dialogue data consists of the dialogue data corresponding to the unexpected dialogue data generated by the initial processing model, extracted from sample dialogue data. The sample dialogue data is obtained by the initial processing model based on the parameters of the sample dialogue data. The target dialogue result refers to the specific output generated by the task processing model based on the input dialogue data to be processed. The target dialogue result is typically a dialogue response that meets user needs and conforms to expected standards.

[0167] In practical applications, there are various ways to obtain the pending dialogue data of the target dialogue task, and the specific method should be selected according to the actual situation. This specification does not limit the methods used in this embodiment. In one possible implementation, the pending dialogue data of the target dialogue task can be received from a user via a terminal device. In another possible implementation, the pending dialogue data of the target dialogue task can be read from other data acquisition devices or a database.

[0168] By applying the solutions in the embodiments of this specification, since the task processing model is trained by comparative learning based on unexpected dialogue data and expected dialogue data, the task processing model can more accurately understand the dialogue data to be processed and generate more appropriate and more accurate target dialogue results.

[0169] Considering the large number of model parameters in the task processing model and the limited computing resources of the terminal device, the dialogue task processing method proposed in the embodiments of this specification can be applied to, for example... Figure 4 The dialogue task processing system shown is not limited to this. See also Figure 4 , Figure 4 This specification illustrates an architecture diagram of a dialogue task processing system according to an embodiment of the present specification. The dialogue task processing system may include a terminal device 402 and a server 404.

[0170] Terminal device 402 is used to send the pending dialogue data of the target dialogue task to server 404;

[0171] Server 404 is used to input the dialogue data to be processed into the task processing model to obtain the target dialogue result, wherein the task processing model is trained based on the task processing model training method; and to send the target dialogue result to the terminal device 402.

[0172] Terminal device 402 is also used to receive the target dialogue result sent by server 404.

[0173] like Figure 4 As shown, the task processing model is deployed in server 404. Server 404 can connect to one or more terminal devices 402 via a local area network (LAN), wide area network (WAN), Internet, or other types of data network. Terminal devices 402 may include, but are not limited to, smartphones, tablets, laptops, PDAs, personal computers, smart home devices, and in-vehicle devices. Terminal devices 402 can also interact with users through a graphical user interface to invoke the task processing model, thereby implementing the dialogue task processing method provided in the embodiments of this specification.

[0174] It is worth noting that the dialogue task processing method provided in the embodiments of this specification is generally executed by the server. However, in other embodiments of this specification, if the terminal device's operating resources can meet the deployment and operating conditions of the task processing model, the terminal device may also have similar functions to the server, thereby executing the dialogue task processing method provided in the embodiments of this specification. In other embodiments, the dialogue task processing method provided in the embodiments of this specification may also be jointly executed by the terminal device and the server.

[0175] The following is in conjunction with the appendix Figure 5 Taking the application of the dialogue task processing method provided in this specification in a virtual character dialogue scenario as an example, the dialogue task processing method will be further explained. Among other things, Figure 5 This specification shows a flowchart of a virtual character dialogue method according to an embodiment, which specifically includes the following steps:

[0176] Step 502: Receive the virtual character dialogue parameters sent by the terminal device.

[0177] Step 504: Input the virtual character dialogue parameters into the task processing model to obtain the virtual character dialogue results. The task processing model is trained based on the task processing model training method.

[0178] Step 506: Send the virtual character's dialogue results back to the terminal device.

[0179] It should be noted that virtual character dialogue parameters refer to the specific data elements describing the dialogue interaction between virtual characters. Virtual character dialogue parameters include at least one of the following: dialogue turn, dialogue scenario, dialogue character, dialogue goal, dialogue sentiment, and historical dialogue data. These parameters provide the task processing model with the information needed for the dialogue process, generating dialogue results that conform to the characteristics of a specific virtual character. The virtual character dialogue result refers to the specific output generated by the task processing model based on the input virtual character dialogue parameters. The virtual character dialogue result conforms to the virtual character's settings and meets the current dialogue requirements. Feedback refers to the process of returning the virtual character dialogue result generated by the task processing model to the terminal device, typically involving converting the result into a displayable format (such as text or voice) and sending it back to the user terminal via an appropriate communication protocol.

[0180] The solution implemented in this specification allows the task processing model to understand virtual character dialogue parameters more accurately, dynamically simulate the behavior of different virtual characters, and provide a rich variety of virtual character dialogue results, while maintaining the realism and coherence of the dialogue, since the task processing model is trained by comparing and contrasting unexpected dialogue data and expected dialogue data.

[0181] See Figure 6 , Figure 6 This specification illustrates a flowchart of an information processing method based on a task processing model, according to an embodiment of the present invention. The information processing method based on the task processing model is applied to a task platform and specifically includes the following steps:

[0182] Step 602: Receive the model request sent by the terminal device.

[0183] Step 604: Based on the model request, determine the target task processing model from multiple task processing models, wherein the multiple task processing models are trained based on the task processing model training method.

[0184] It should be noted that the target task processing model is a task processing model applicable to the target scenario. The model request includes at least one of the following: the scenario identifier of the target scenario, the scenario input data of the target scenario, and model specification parameters. There are multiple ways to determine the target task processing model from multiple task processing models based on the model request; the specific method chosen depends on the actual situation, and this specification does not impose any limitations on this method. In one possible implementation of this specification, the corresponding target task processing model can be searched from at least one task processing model included in the model library based on the model request. In another possible implementation, the target task processing model can be trained and obtained based on the model request. In yet another optional implementation, the target task processing model can be constructed based on the model request.

[0185] For example, based on the scene identifier of the target scene, at least one pre-trained task processing model can be searched from the model library. Then, based on the model specification parameters, an initial task processing model can be selected from the at least one task processing model. Finally, based on the scene input data of the target scene, the selected initial task processing model is trained to obtain a target task processing model suitable for user needs. The at least one task processing model can be based on... Figure 1 The task processing model shown was trained using the training method described in this specification, and will not be repeated in the embodiments.

[0186] The solutions implemented in the embodiments of this specification are adapted to user needs to obtain target task processing models, realize personalized model services, provide users with an efficient, flexible and easy-to-use model service method, and improve user experience.

[0187] In one optional embodiment of this specification, the model request includes a scene identifier of the target scene; the process of determining the target task processing model from multiple task processing models based on the model request may include the following steps:

[0188] Based on the scene identifier of the target scene, the target task processing model suitable for the target scene is searched from the model library. The model library stores multiple task processing models suitable for different task processing scenarios.

[0189] It should be noted that scene identifiers are unique or specific labels used to distinguish different task scenarios. The model library is a database for storing and managing various pre-trained deep learning models. Multiple task processing models adapted to different task scenarios cover different application scenarios and needs. The model library allows users to select appropriate models according to their needs, or directly call models for task processing through application programming interfaces.

[0190] Multiple task processing models adapted to different task scenarios are stored in the model library. Each model is optimized for a specific application environment. Any task processing model is based on... Figure 1 The training method shown is used to train the obtained model for the task processing model. For example, based on the scene identifier "virtual character dialogue" of the target scene, a target task processing model suitable for the virtual character dialogue scene can be found from the model library.

[0191] By applying the solutions in the embodiments of this specification, based on scenario requirements, the target task processing model adapted to the scenario is accurately found through scenario identification, making task processing more accurate and more scenario-appropriate, thereby improving user experience and task processing quality.

[0192] In one optional embodiment of this specification, the model request includes scene input data of the target scene; the above-mentioned determination of the target task processing model from multiple task processing models based on the model request may include the following steps:

[0193] From multiple task processing models, determine the initial task processing model that is suitable for the target scenario;

[0194] Based on the scene input data of the target scene, the initial task processing model is trained to obtain the target task processing model.

[0195] It should be noted that the scenarios adapted to by each task processing model differ. For example, task processing model 1 is suitable for scenarios 1 and 2, while task processing model 2 is suitable for scenarios 2 and 3. The initial task processing model refers to the model among multiple task processing models that is suitable for the target scenario. If the target scenario is scenario 1, then the initial task processing model is task processing model 1 adapted to scenario 1. The initial task processing model may not only be applicable to the target scenario but also to other scenarios, making it a general task processing model applicable to different scenarios. While the initial task processing model can be used for task processing, the results may not be ideal. In such cases, the initial task processing model can be optimized based on the scenario input data of the target scenario. For example, optimizing the initial task processing model based on the scenario input data of a virtual character dialogue scenario can yield a target task processing model suitable for virtual character dialogue scenarios. The scenario input data of the target scenario can be understood as the sample dialogue parameters of the sample tasks in the target scenario. The method for training the initial task processing model based on the scenario input data of the target scenario to obtain the target task processing model can be found in [reference needed]. Figure 1 The training method of the task processing model shown in this specification will not be described again in the embodiments.

[0196] By applying the solutions in the embodiments of this specification, based on scenario requirements, a general initial task processing model is further trained using scenario input data to obtain a target task processing model adapted to the scenario. This makes the target task processing model more closely fit the scenario, thereby improving user experience and task processing quality.

[0197] In one optional embodiment of this specification, the model request includes model specification parameters; the process of determining the target task processing model from multiple task processing models based on the model request may include the following steps:

[0198] Based on the model specification parameters, the corresponding target task processing model is searched from the model library, which stores multiple task processing models with different model specification parameters.

[0199] It's important to note that model specifications refer to the various parameters that define the model's structure and behavior. These parameters can be broadly categorized into two types: model parameters (learnable parameters) and hyperparameters. Model parameters are those automatically adjusted during model training via backpropagation, including but not limited to weight matrices and biases. For example, in a simple fully connected layer, the weight matrix is ​​a two-dimensional tensor connecting neurons in the input and output layers; the biases are one-dimensional vectors providing additional offset values ​​for each output neuron. Hyperparameters are parameters set before model training begins, controlling the model's learning process and architecture. Hyperparameters include, but are not limited to, the learning rate and the number of neurons per layer, chosen based on specific requirements.

[0200] By applying the solutions in the embodiments of this specification, based on the model specification parameters, the corresponding target task processing model can be accurately found, ensuring the efficient and stable operation of the target task processing model and improving the user experience.

[0201] In one optional embodiment of this specification, after determining the target task processing model from multiple task processing models based on the model request, the following steps may be further included:

[0202] Deploy the target task processing model and build a task processing interface based on the target task processing model so that the terminal device can schedule the target task processing model to execute the target dialogue task.

[0203] It should be noted that the task processing interface is an interactive programming interface for the terminal device to schedule the target task processing model to process the target task, and it is usually provided in the form of an application programming interface (API). Through the task processing interface, users can input task data for the target task, such as emotional speech generation data, to perform emotional speech generation.

[0204] In practical applications, there are various ways to deploy the target task processing model, and the specific method should be chosen based on the actual situation. This specification does not impose any limitations on this approach. One possible implementation of this specification is to deploy the target task processing model on cloud-side devices using infrastructure provided by a cloud service provider. Another possible implementation of this specification is to deploy the target task processing model on edge devices using a lightweight framework. For example, the target task processing model can be deployed on a distributed system, and a task processing interface can be built based on the target task processing model and provided to terminal devices, enabling the terminal devices to schedule the target task processing model to execute the target task.

[0205] By applying the solutions provided in the embodiments of this specification, deploying the target task processing model, and building a task processing interface based on the target task processing model, terminal devices can efficiently call the target task processing model, thereby improving the processing quality and response speed of the target task.

[0206] See Figure 7 , Figure 7 This specification shows a schematic diagram of the structure of a task platform 700 provided in one embodiment of the present specification. The task platform 700 includes a request interface 702 and a response unit 704.

[0207] Request interface 702 is used to receive a model request sent by a terminal device, wherein the model request includes at least one of the following: scene identifier of the target scene, scene input data of the target scene, and model specification parameters.

[0208] The response unit 704 is used to determine the target task processing model from multiple task processing models based on the model request, wherein the multiple task processing models are trained based on the task processing model training method.

[0209] In one optional embodiment of this specification, the task platform further includes a task processing interface, which is constructed based on the target task processing model.

[0210] The task processing interface is used for terminal devices to schedule and execute target dialogue tasks.

[0211] The above is an illustrative scheme of a task platform according to this embodiment. It should be noted that the technical solution of this task platform and the technical solution of the information processing method based on the task processing model described above belong to the same concept. For details not described in detail in the technical solution of the task platform, please refer to the description of the technical solution of the information processing method based on the task processing model described above.

[0212] Corresponding to the above-described embodiments of the task processing model training method, this specification also provides embodiments of the task processing model training apparatus. Figure 8 A schematic diagram of a task processing model training device provided in one embodiment of this specification is shown.

[0213] like Figure 8 As shown, the device includes:

[0214] The first acquisition module 802 is configured to acquire sample dialogue parameters;

[0215] The first input module 804 is configured to input sample dialogue parameters into the initial processing model, obtain sample dialogue data, and extract unexpected dialogue data from the sample dialogue data.

[0216] The generation module 806 is configured to use the initial processing model to generate the expected dialogue data corresponding to the unexpected dialogue data;

[0217] The first training module 808 is configured to train the initial processing model based on the unexpected dialogue data and the expected dialogue data to obtain the task processing model.

[0218] Optionally, the undesired dialogue data is multi-turn dialogue data; the generation module 806 is further configured to use the initial processing model to generate the desired dialogue data corresponding to the undesired dialogue data, which may include the following steps: parsing the undesired dialogue data to obtain the dialogue segment to be optimized, and extracting the historical dialogue segment of the dialogue segment to be optimized from the undesired dialogue data; inputting the historical dialogue segment into the initial processing model to obtain the desired dialogue data corresponding to the undesired dialogue data.

[0219] Optionally, the generation module 806 is further configured to input historical dialogue fragments into the initial processing model multiple times to obtain multiple candidate dialogue data, wherein the number of candidate dialogue data is the same as the number of times the model is input; to perform quality detection on the multiple candidate dialogue data to obtain a first quality index, wherein the first quality index corresponds one-to-one with the candidate dialogue data; and to filter out the desired dialogue data corresponding to the undesired dialogue data from the multiple candidate dialogue data according to the first quality index.

[0220] Optionally, the generation module 806 is further configured to obtain parsing rules for the unexpected dialogue data, and construct parsing prompt information based on the parsing rules; input the parsing prompt information and the unexpected dialogue data into the data processing model to extract the segment to be optimized, thereby obtaining the dialogue segment to be optimized.

[0221] Optionally, both the unexpected dialogue data and the expected dialogue data are multi-turn dialogue data; the first training module 808 is further configured to compare the unexpected dialogue data and the expected dialogue data to determine key dialogue segments in the expected dialogue data, wherein the key dialogue segments make the quality of the expected dialogue data higher than that of the unexpected dialogue data; extract unexpected dialogue segments from the unexpected dialogue data based on the key dialogue segments; and train the initial processing model based on the key dialogue segments and the unexpected dialogue segments to obtain the task processing model.

[0222] Optionally, the first training module 808 is further configured to input the fragment extraction prompt information, the unexpected dialogue data and the expected dialogue data into the data processing model to extract key dialogue fragments and obtain key dialogue fragments from the expected dialogue data.

[0223] Optionally, the first training module 808 is further configured to calculate the model preference loss based on the unexpected dialogue data and the expected dialogue data; and adjust the model parameters of the initial processing model based on the model preference loss to obtain the task processing model.

[0224] Optionally, the first input module 804 is configured to input quality detection prompts and sample dialogue data into the data processing model to obtain a second quality index; and to filter out unwanted dialogue data from the sample dialogue data based on the second quality index.

[0225] Optionally, the device further includes: a detection module configured to perform quality detection on the unexpected dialogue data to obtain a third quality index, and to perform quality detection on the expected dialogue data to obtain a fourth quality index; and a first training module 808 further configured to train an initial processing model based on the unexpected dialogue data and the expected dialogue data to obtain a task processing model when the fourth quality index is greater than the third quality index.

[0226] Optionally, the device further includes: a second training module configured to acquire reference dialogue parameters and reference dialogue data corresponding to the reference dialogue parameters; input the reference dialogue parameters into the original processing model to obtain predicted dialogue data; and train the original processing model based on the reference dialogue data and the predicted dialogue data to obtain an initial processing model.

[0227] The solution described in this specification reduces reliance on large amounts of manually labeled dialogue data by automatically generating dialogue data for model training, thus lowering costs and workload. Since both unwanted and desired dialogue data are generated by the model, the model can automatically adjust based on actual performance, enabling continuous model improvement. By comparing and learning from unwanted and desired dialogue data, the model can more accurately understand dialogue intent and provide more appropriate and natural responses, thereby improving model performance.

[0228] The above is a schematic scheme of a task processing model training device according to this embodiment. It should be noted that the technical solution of this task processing model training device and the technical solution of the task processing model training method described above belong to the same concept. For details not described in detail in the technical solution of the task processing model training device, please refer to the description of the technical solution of the task processing model training method described above.

[0229] Corresponding to the above-described embodiments of the dialogue task processing method, this specification also provides embodiments of the dialogue task processing apparatus. Figure 9 A schematic diagram of a dialogue task processing apparatus according to one embodiment of this specification is shown. Figure 9 As shown, the device includes:

[0230] The second acquisition module 902 is configured to acquire the pending dialogue data of the target dialogue task;

[0231] The second input module 904 is configured to input the dialogue data to be processed into the task processing model to obtain the target dialogue result, wherein the task processing model is trained based on the task processing model training method.

[0232] By applying the solutions in the embodiments of this specification, since the task processing model is trained by comparative learning based on unexpected dialogue data and expected dialogue data, the task processing model can more accurately understand the dialogue data to be processed and generate more appropriate and more accurate target dialogue results.

[0233] The above is an illustrative scheme of a dialogue task processing device according to this embodiment. It should be noted that the technical solution of this dialogue task processing device and the technical solution of the above-described dialogue task processing method belong to the same concept. For details not described in detail in the technical solution of the dialogue task processing device, please refer to the description of the technical solution of the above-described dialogue task processing method.

[0234] Corresponding to the above embodiments of the virtual character dialogue method, this specification also provides embodiments of the virtual character dialogue device. Figure 10 A schematic diagram of a virtual character dialogue device according to one embodiment of this specification is shown. Figure 10 As shown, the device includes:

[0235] The first receiving module 1002 is configured to receive virtual character dialogue parameters sent by the terminal device;

[0236] The third input module 1004 is configured to input the virtual character dialogue parameters into the task processing model to obtain the virtual character dialogue results, wherein the task processing model is trained based on the task processing model training method.

[0237] Feedback module 1006 is configured to send the results of the virtual character's dialogue to the terminal device.

[0238] The solution implemented in this specification allows the task processing model to understand virtual character dialogue parameters more accurately, dynamically simulate the behavior of different virtual characters, and provide a rich variety of virtual character dialogue results, while maintaining the realism and coherence of the dialogue, since the task processing model is trained by comparing and contrasting unexpected dialogue data and expected dialogue data.

[0239] The above is an illustrative scheme of a virtual character dialogue device according to this embodiment. It should be noted that the technical solution of this virtual character dialogue device and the technical solution of the virtual character dialogue method described above belong to the same concept. For details not described in detail in the technical solution of the virtual character dialogue device, please refer to the description of the technical solution of the virtual character dialogue method described above.

[0240] Corresponding to the above-described embodiments of the information processing method based on the task processing model, this specification also provides embodiments of the information processing apparatus based on the task processing model. Figure 11 A schematic diagram of the structure of an information processing device based on a task processing model provided in one embodiment of this specification is shown. Figure 11 As shown, the device is applied to a mission platform and includes:

[0241] The second receiving module 1102 is configured to receive model requests sent by the terminal device;

[0242] The determination module 1104 is configured to determine the target task processing model from multiple task processing models based on a model request, wherein the multiple task processing models are trained based on a task processing model training method.

[0243] Optionally, the model request includes a scene identifier of the target scene; the determination module 1104 is further configured to search for a target task processing model suitable for the target scene from the model library based on the scene identifier of the target scene, wherein the model library stores multiple task processing models suitable for different task processing scenes.

[0244] Optionally, the model request includes scene input data of the target scene; the determination module 1104 is further configured to determine an initial task processing model suitable for the target scene from multiple task processing models; and to train the initial task processing model based on the scene input data of the target scene to obtain the target task processing model.

[0245] Optionally, the model request includes model specification parameters; the determination module 1104 is further configured to search for the corresponding target task processing model from the model library based on the model specification parameters, wherein the model library stores multiple task processing models with different model specification parameters.

[0246] Optionally, the device further includes: a deployment module configured to deploy a target task processing model and, based on the target task processing model, construct a task processing interface to enable the terminal device to schedule the target task processing model to execute a target dialogue task.

[0247] The solutions implemented in the embodiments of this specification are adapted to user needs to obtain target task processing models, realize personalized model services, provide users with an efficient, flexible and easy-to-use model service method, and improve user experience.

[0248] The above is an illustrative scheme of an information processing device based on a task processing model according to this embodiment. It should be noted that the technical solution of this information processing device based on a task processing model belongs to the same concept as the technical solution of the information processing method based on a task processing model described above. For details not described in detail in the technical solution of the information processing device based on a task processing model, please refer to the description of the technical solution of the information processing method based on a task processing model described above.

[0249] Figure 12 A structural block diagram of a computing device 1200 provided in one embodiment of this specification is shown.

[0250] The computing device 1200 includes:

[0251] Memory 1210 and processor 1220;

[0252] The memory 1210 is used to store computer programs / instructions, and the processor 1220 is used to execute the computer programs / instructions. When the computer programs / instructions are executed by the processor 1220, they implement the steps of the above-mentioned task processing model training method, dialogue task processing method, virtual character dialogue method, or information processing method based on task processing model.

[0253] In one or more embodiments of this specification, the computing device can be understood as an integrated smart terminal, including but not limited to a server, desktop computer, personal computer (PC), all-in-one model machine, mobile phone, tablet computer or other portable smart terminal, etc., and the computing device may have the model described in the above embodiments of this application pre-installed.

[0254] Specifically, this computing device can pre-install various types of models, including but not limited to models in natural language processing, visual processing, speech processing, code processing, and multimodal task processing, thus providing diverse model selection. In different product forms, this computing device can support one or more model usage methods, including but not limited to model training, model invocation, model fine-tuning, model deployment, model inference, and application. In some product forms, this computing device also supports model management, including but not limited to multi-type model management (supporting the management of discriminative, generative, and other model types), model version control (supporting the control of different model versions), and model evaluation (evaluating model performance and effectiveness based on model evaluation tools). In other product forms, this computing device can also create applications based on models, providing application programming interface (API) invocation capabilities. Models can be invoked into created applications through the API interface, and application management tools are provided for application management and monitoring.

[0255] Furthermore, the computing device may also include data management (supporting the creation and management of model tuning datasets), a training center (providing abundant training resources to help users learn and master artificial intelligence technology), and basic control capabilities (providing enterprise-level basic control capabilities to ensure the security and efficient operation of the system). Through the above functions, it provides a comprehensive and integrated device for artificial intelligence development, training, deployment, and application.

[0256] Figure 13 A structural block diagram of an electronic device 1300 provided according to one embodiment of this specification is shown.

[0257] The memory 1310 and the processor 1320 are connected via a bus 1330;

[0258] The memory 1310 is used to store computer programs / instructions, and the processor 1320 is used to execute the computer programs / instructions. When the computer programs / instructions are executed by the processor 1320, they implement the steps of the above-mentioned task processing model training method, dialogue task processing method, virtual character dialogue method, or information processing method based on task processing model.

[0259] Specifically, the components of the electronic device 1300 include, but are not limited to, a memory 1310 and a processor 1320. The processor 1320 and the memory 1310 can be connected via a bus 1330.

[0260] Electronic device 1300 may also include access device 1340, which enables electronic device 1300 to communicate with database 1350 storing data via one or more networks 1360. Examples of such networks include Public Switched Telephone Network (PSTN), Local Area Network (LAN), Wide Area Network (WAN), Personal Area Network (PAN), or combinations of communication networks such as the Internet. Access device 1340 may include one or more of any type of wired or wireless network interface (e.g., a network interface card (NIC)), such as an IEEE 802.11 wireless local area network (WLAN) interface, a Wi-MAX (World Interoperability for Microwave Access) interface, an Ethernet interface, a Universal Serial Bus (USB) interface, a cellular network interface, a Bluetooth interface, a Near Field Communication (NFC) interface, and so on.

[0261] In one embodiment of this specification, the above-described components of the electronic device 1300 and Figure 13 Other components, not shown, can also be connected to each other, for example, via a bus. It should be understood that... Figure 13 The block diagram of the electronic device shown is for illustrative purposes only and is not intended to limit the scope of this specification. Those skilled in the art can add or replace other components as needed.

[0262] Electronic device 1300 can be any type of stationary or mobile electronic device, including mobile computers or mobile computing devices (e.g., tablet computers, personal digital assistants, laptop computers, notebook computers, netbooks, etc.), mobile phones (e.g., smartphones), wearable electronic devices (e.g., smartwatches, smart glasses, etc.) or other types of mobile devices, or stationary electronic devices such as desktop computers or PCs. Electronic device 1300 can also be a mobile or stationary electronic device.

[0263] The above is an illustrative scheme of an electronic device according to this embodiment. It should be noted that the technical solution of this electronic device belongs to the same concept as the technical solutions of the task processing model training method, dialogue task processing method, virtual character dialogue method, and information processing method based on task processing model described above. For details not described in detail in the technical solution of the electronic device, please refer to the description of the technical solutions of the task processing model training method, dialogue task processing method, virtual character dialogue method, or information processing method based on task processing model described above.

[0264] An embodiment of this specification also provides a computer-readable storage medium storing a computer program / instructions that, when executed by a processor, implement the steps of the task processing model training method, dialogue task processing method, virtual character dialogue method, or information processing method based on the task processing model described above.

[0265] The above is an illustrative scheme of a computer-readable storage medium according to this embodiment. It should be noted that the technical solution of this storage medium belongs to the same concept as the technical solutions of the task processing model training method, dialogue task processing method, virtual character dialogue method, and information processing method based on task processing models described above. Details not described in detail in the technical solution of the storage medium can be found in the descriptions of the technical solutions of the task processing model training method, dialogue task processing method, virtual character dialogue method, or information processing method based on task processing models.

[0266] An embodiment of this specification also provides a computer program product, including a computer program / instructions that, when executed by a processor, implement the steps of the above-described task processing model training method, dialogue task processing method, virtual character dialogue method, or information processing method based on a task processing model.

[0267] The above is an illustrative scheme of a computer program product according to this embodiment. It should be noted that the technical solution of this computer program product belongs to the same concept as the technical solutions of the task processing model training method, dialogue task processing method, virtual character dialogue method, and information processing method based on task processing model described above. For details not described in detail in the technical solution of the computer program product, please refer to the description of the technical solutions of the task processing model training method, dialogue task processing method, virtual character dialogue method, or information processing method based on task processing model described above.

[0268] The foregoing has described specific embodiments of this specification. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recited in the claims may be performed in a different order than that shown in the embodiments and may still achieve the desired result. Furthermore, the processes depicted in the drawings do not necessarily require the specific or sequential order shown to achieve the desired result. In some embodiments, multitasking and parallel processing are possible or may be advantageous.

[0269] The computer instructions include computer program code, which may be in the form of source code, object code, executable file, or certain intermediate forms. The computer-readable medium may include: any entity or device capable of carrying the computer program code, recording media, USB flash drive, portable hard drive, magnetic disk, optical disk, computer memory, read-only memory (ROM), random access memory (RAM), electrical carrier signals, telecommunication signals, and software distribution media, etc. It should be noted that the content included in the computer-readable medium may be appropriately added or removed according to the requirements of patent practice. For example, in some regions, according to patent practice, computer-readable media may not include electrical carrier signals and telecommunication signals.

[0270] It should be noted that, for the sake of simplicity, the foregoing method embodiments are all described as a series of actions. However, those skilled in the art should understand that the embodiments in this specification are not limited to the described order of actions, because according to the embodiments in this specification, some steps can be performed in other orders or simultaneously. Furthermore, those skilled in the art should also understand that the embodiments described in this specification are all preferred embodiments, and the actions and modules involved are not necessarily essential to the embodiments in this specification.

[0271] In the above embodiments, the descriptions of each embodiment have different focuses. For parts not described in detail in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0272] The preferred embodiments disclosed above are merely illustrative of this specification. The optional embodiments do not exhaustively describe all details, nor do they limit the invention to the specific implementations described. Clearly, many modifications and variations can be made based on the embodiments described herein. These embodiments are selected and specifically described in this specification to better explain the principles and practical applications of the embodiments, thereby enabling those skilled in the art to better understand and utilize this specification. This specification is limited only by the claims and their full scope and equivalents.

Claims

1. A method for training a task processing model, comprising: Obtain sample dialogue parameters; The sample dialogue parameters are input into the initial processing model to obtain sample dialogue data, and unwanted dialogue data is extracted from the sample dialogue data. Using the initial processing model, generate the desired dialogue data corresponding to the undesired dialogue data; The initial processing model is trained based on the unexpected dialogue data and the expected dialogue data to obtain the task processing model.

2. The method according to claim 1, wherein the unwanted dialogue data is multi-turn dialogue data; The step of generating the desired dialogue data corresponding to the undesired dialogue data using the initial processing model includes: The unexpected dialogue data is parsed to obtain the dialogue segment to be optimized, and the historical dialogue segments of the dialogue segment to be optimized are extracted from the unexpected dialogue data. The historical dialogue fragments are input into the initial processing model to obtain the expected dialogue data corresponding to the unexpected dialogue data.

3. The method according to claim 2, wherein inputting the historical dialogue fragment into the initial processing model to obtain the expected dialogue data corresponding to the unexpected dialogue data includes: The historical dialogue fragments are input into the initial processing model multiple times to obtain multiple candidate dialogue data, wherein the number of candidate dialogue data is the same as the number of times the model is input; The multiple candidate dialogue data are subjected to quality detection to obtain a first quality index, wherein the first quality index corresponds one-to-one with the candidate dialogue data. Based on the first quality index, the desired dialogue data corresponding to the undesired dialogue data is selected from the plurality of candidate dialogue data.

4. The method according to claim 2, wherein parsing the unwanted dialogue data to obtain the dialogue fragment to be optimized includes: Obtain the parsing rules for the unexpected dialogue data, and construct parsing prompt information based on the parsing rules; The parsed prompt information and the unexpected dialogue data are input into the data processing model to extract the segment to be optimized, thereby obtaining the dialogue segment to be optimized.

5. The method according to claim 1, wherein both the unwanted dialogue data and the desired dialogue data are multi-turn dialogue data; The step of training the initial processing model based on the unwanted dialogue data and the desired dialogue data to obtain the task processing model includes: By comparing the unwanted dialogue data and the desired dialogue data, key dialogue segments in the desired dialogue data are identified, wherein the key dialogue segments make the quality of the desired dialogue data higher than that of the unwanted dialogue data. Based on the key dialogue fragments, extract the unexpected dialogue fragments from the unexpected dialogue data; The initial processing model is trained based on the key dialogue segments and the unexpected dialogue segments to obtain the task processing model.

6. The method according to claim 5, wherein comparing the unwanted dialogue data and the desired dialogue data to determine key dialogue segments in the desired dialogue data comprises: The fragment extraction prompts, the unexpected dialogue data, and the expected dialogue data are input into a data processing model to extract key dialogue fragments, thereby obtaining key dialogue fragments from the expected dialogue data.

7. The method according to claim 1, wherein extracting unexpected dialogue data from the sample dialogue data comprises: The quality inspection prompts and the sample dialogue data are input into the data processing model to obtain the second quality index. Based on the second quality index, unwanted dialogue data is filtered out from the sample dialogue data.

8. The method according to claim 1, further comprising, before training the initial processing model based on the unexpected dialogue data and the expected dialogue data to obtain the task processing model: The unexpected dialogue data is subjected to quality inspection to obtain a third quality index, and the expected dialogue data is subjected to quality inspection to obtain a fourth quality index. The step of training the initial processing model based on the unwanted dialogue data and the desired dialogue data to obtain the task processing model includes: If the fourth quality metric is greater than the third quality metric, the initial processing model is trained based on the unexpected dialogue data and the expected dialogue data to obtain a task processing model.

9. The method according to claim 1, wherein training the initial processing model based on the unexpected dialogue data and the expected dialogue data to obtain a task processing model comprises: Calculate the model preference loss based on the unwanted dialogue data and the desired dialogue data; Based on the model preference loss, the model parameters of the initial processing model are adjusted to obtain the task processing model.

10. The method according to any one of claims 1 to 9, wherein before inputting the sample dialogue parameters into the initial processing model to obtain the sample dialogue data, the method further comprises: Obtain the reference dialogue parameters and the reference dialogue data corresponding to the reference dialogue parameters; The reference dialogue parameters are input into the original processing model to obtain the predicted dialogue data; The original processing model is trained based on the reference dialogue data and the predicted dialogue data to obtain an initial processing model.

11. A dialogue task processing method, comprising: Obtain the pending dialogue data for the target dialogue task; The dialogue data to be processed is input into the task processing model to obtain the target dialogue result, wherein the task processing model is trained based on the training method described in any one of claims 1 to 10.

12. A method for virtual character dialogue, comprising: Receive virtual character dialogue parameters sent by the terminal device; The virtual character dialogue parameters are input into the task processing model to obtain the virtual character dialogue results, wherein the task processing model is trained based on the training method described in any one of claims 1 to 10; The results of the virtual character's dialogue are fed back to the terminal device.

13. An information processing method based on a task processing model, applied to a task platform, comprising: Receive model requests sent by terminal devices; Based on the model request, a target task processing model is determined from a plurality of task processing models, wherein the plurality of task processing models are trained based on the training method described in any one of claims 1 to 10.

14. The method according to claim 13, wherein the model request includes a scene identifier of the target scene; The step of determining the target task processing model from multiple task processing models based on the model request includes: Based on the scene identifier of the target scene, a target task processing model suitable for the target scene is searched from the model library, wherein the model library stores multiple task processing models suitable for different task processing scenarios.

15. The method of claim 13, wherein the model request includes scene input data of the target scene; The step of determining the target task processing model from multiple task processing models based on the model request includes: From multiple task processing models, determine an initial task processing model that is suitable for the target scenario; Based on the scene input data of the target scene, the initial task processing model is trained to obtain the target task processing model.

16. The method of claim 13, wherein the model request includes model specification parameters; The step of determining the target task processing model from multiple task processing models based on the model request includes: Based on the model specification parameters, the corresponding target task processing model is searched from the model library, wherein the model library stores multiple task processing models with different model specification parameters.

17. The method according to any one of claims 13 to 16, wherein after determining the target task processing model from multiple task processing models based on the model request, the method further comprises: Deploy the target task processing model and, based on the target task processing model, construct a task processing interface so that the terminal device can schedule the target task processing model to execute the target dialogue task.

18. A task platform, comprising a request interface and a response unit; The request interface is used to receive model requests sent by the terminal device, wherein... The model request includes at least one of the following: the scene identifier of the target scene, the scene input data of the target scene, and the model specification parameters. The response unit is configured to determine a target task processing model from multiple task processing models based on the model request, wherein the multiple task processing models are trained based on the training method described in any one of claims 1 to 10.

19. The task platform according to claim 18, further comprising a task processing interface, wherein the task processing interface is constructed based on the target task processing model; The task processing interface is used for the terminal device to schedule and execute target dialogue tasks.

20. A computing device, comprising: Memory and processor; The memory is used to store computer programs / instructions, and the processor is used to execute the computer programs / instructions, which, when executed by the processor, implement the steps of the method according to any one of claims 1 to 17.

21. An electronic device, comprising: A memory and a processor, the memory and the processor being connected via a bus; The memory is used to store computer programs / instructions, and the processor is used to execute the computer programs / instructions, which, when executed by the processor, implement the steps of the method according to any one of claims 1 to 17.

22. A computer-readable storage medium storing a computer program / instructions that, when executed by a processor, implement the steps of the method according to any one of claims 1 to 17.

23. A computer program product comprising a computer program / instructions that, when executed by a processor, implement the steps of the method according to any one of claims 1 to 17.

Images