Model training method and device, conversation processing method and device, storage medium and program product

Abstract

The embodiment of the invention provides a model training method and device, a conversation processing method and device, a storage medium and a program product. In the model training method, the target driving model can output the context guidance information according to the dialogue message so as to activate or regulate the generation capability of the reply model, so that data-driven and scene-adaptive dynamic prompt generation is realized, and the generation quality and the output stability of the reply model are improved. And the parameter quantity of the target driving model is smaller than that of the reply model, so that the target driving model can be operated at a lower cost and optimized at a lower calculation cost in an edge calculation or private deployment scene, and the dual requirements of a user on data security and calculation power saving in a target dialogue scene are met. Wherein the context reconstruction capability of the target driving model can be continuously optimized according to the quality of the reply result output by the reply model, so that the reply model is indirectly guided to continuously enhance the task processing capability in the vertical field.

Description

Technical Field

[0001] This application relates to the field of artificial intelligence technology, and in particular to a model training, dialogue processing method, device, storage medium, and program product. Background Technology

[0002] With the development of Artificial Intelligence (AI) technology, Large Language Models (LLMs) have demonstrated excellent general-purpose capabilities in natural language understanding and generation tasks. LLM-related applications are rapidly expanding into various scenarios, such as general-purpose intelligent agent systems, vertical industry knowledge question answering, and enterprise-level service automation. However, when targeting specific customers, the practical deployment of LLM-related applications still faces a series of core challenges. Although current mainstream large language models possess strong generalization and reasoning abilities, their behavior in practical applications is highly dependent on input prompts. In current applications, prompts are mostly designed manually, resulting in inconsistent prompt quality and thus limiting the full potential of LLMs. Therefore, a new solution is needed. Summary of the Invention

[0003] This application provides a model training, dialogue processing method, device, storage medium, and program product to optimize the quality of prompt words, thereby improving the task processing capability of the target language model.

[0004] This application provides a dialogue processing method, including: acquiring a dialogue message sample; invoking a target-driven model to reconstruct the context of the dialogue message sample to obtain context guidance information; inputting the context guidance information and the dialogue message sample into a response model, so that the response model generates a response result sample of the dialogue message sample under the guidance of the context guidance information; the number of parameters of the response model is greater than the number of parameters of the target-driven model; and optimizing the target-driven model according to the quality score of the response result sample.

[0005] Optionally, before invoking the target-driven model, the method further includes: identifying the target dialogue scenario corresponding to the dialogue message sample; determining, from at least one driving model that collaborates with the response model, the driving model corresponding to the target dialogue scenario as the target driving model; wherein, different driving models correspond to different dialogue scenarios in the at least one driving model.

[0006] Optionally, the target-driven model corresponding to the target dialogue scenario is invoked to reconstruct the context of the dialogue message sample and obtain context guidance information, including: in the target-driven model, task planning is performed based on the dialogue message sample to obtain the target processing logic corresponding to the dialogue message sample, and the context guidance information is output based on the target processing logic.

[0007] Optionally, in the target-driven model, task planning is performed based on the dialogue message sample to obtain the target processing logic corresponding to the dialogue message sample, including: under the constraints of the task processing standard corresponding to the target dialogue scenario, target decomposition is performed on the dialogue message sample to obtain a sub-task sequence; and the target processing logic is generated based on the sub-task sequence.

[0008] Optionally, under the constraints of the task processing standard corresponding to the target dialogue scenario, the dialogue message sample is decomposed to obtain a subtask sequence, including: determining the subtask type according to the task processing standard corresponding to the target dialogue scenario, and decomposing the dialogue message sample according to the subtask type to obtain the subtask sequence; or, decomposing the dialogue message sample to obtain an initial subtask sequence, and optimizing the initial subtask sequence according to the task processing standard corresponding to the target dialogue scenario to obtain the subtask sequence.

[0009] Optionally, the target-driven model is optimized based on the quality score of the response result sample, including: calling the reward model, using the target evaluation method corresponding to the target dialogue scenario to score the response result sample, and obtaining the quality score of the response result sample; and optimizing the target-driven model based on the quality score.

[0010] Optionally, before calling the reward model and scoring the response result sample of the dialogue message sample using the target evaluation method corresponding to the target dialogue scenario, the method further includes: obtaining the scenario response difficulty corresponding to the dialogue message sample; selecting an evaluation method that matches the scenario response difficulty from at least one evaluation method as the target evaluation method, wherein the at least one evaluation method includes: a point-based evaluation method and / or a pair-based evaluation method, wherein the scenario response difficulty matched by the point-based evaluation method is less than the scenario response difficulty corresponding to the pair-based evaluation method.

[0011] Optionally, obtaining the scenario response difficulty corresponding to the dialogue message sample includes: obtaining the scenario evaluation criteria of the target dialogue scenario provided by the user; and using at least one difficulty analysis dimension to perform response difficulty analysis on the scenario evaluation criteria to obtain the scenario response difficulty corresponding to the dialogue message sample.

[0012] Optionally, the reward model is invoked to score the response result sample of the dialogue message sample using the target evaluation method corresponding to the target dialogue scenario, thereby obtaining a quality score for the response result sample. This includes: if the target evaluation method is a peer evaluation method, obtaining a reference response result sample corresponding to the dialogue message sample; forming a sample pair from the dialogue message sample, the response result sample, and the reference response result sample; inputting the sample pair into the reward model, so that the reward model calculates the reward scores of the response result sample and the reference response result sample given the dialogue message sample; and binarizing the reward score of the response result sample based on the reward score of the reference response result sample to obtain a quality score for the response result sample.

[0013] This application also provides a dialogue processing method, including: acquiring a target dialogue message; invoking a target-driven model to reconstruct the context of the target dialogue message to obtain contextual guidance information; the target-driven model is trained using the model training method described in this application; inputting the contextual guidance information and the target dialogue message into a response model, so that the response model generates a response result for the target dialogue message under the guidance of the contextual guidance information; the number of parameters of the response model is greater than the number of parameters of the target-driven model.

[0014] Optionally, before invoking the target-driven model, the method further includes: identifying the target dialogue scenario corresponding to the target dialogue message; determining, from at least one driving model that collaborates with the response model, the driving model corresponding to the target dialogue scenario as the target driving model; wherein, different driving models correspond to different dialogue scenarios in the at least one driving model.

[0015] Optionally, the at least one driving model includes at least one of the following: a driving model corresponding to a medical dialogue scenario, a driving model corresponding to a financial dialogue scenario, a driving model corresponding to a government dialogue scenario, and a driving model corresponding to an industrial scheduling scenario.

[0016] This application also provides an electronic device, including: a memory and a processor; the memory is used to store one or more computer instructions; the processor is used to execute the one or more computer instructions to perform the steps in the method provided in this application.

[0017] This application also provides a computer-readable storage medium storing a computer program, which, when executed by a processor, can implement the steps in the method provided in this application.

[0018] This application also provides a computer program product, including: a computer program / instructions, which, when executed by a processor, can implement the steps in the method provided in this application.

[0019] In this embodiment, a target-driven model capable of performing context reconstruction and a response model for content generation are employed in collaborative operation. The target-driven model outputs contextual guidance information based on dialogue messages. This contextual guidance information serves as the core carrier for activating or regulating the generation capabilities of the response model, replacing traditional methods of manually writing or statically setting prompts. This achieves data-driven, scenario-adaptive dynamic prompt generation, which is beneficial for improving the generation quality and output stability of the response model. The target-driven model has fewer parameters than the response model, making it easier to run and optimize the target-driven model at a lower cost in edge computing or private deployment scenarios, thus meeting the dual needs of users in the target dialogue scenario for data security and computational efficiency. Furthermore, the context reconstruction capability of the target-driven model can be continuously optimized based on the quality of the response results output by the response model, thereby indirectly guiding the response model to continuously enhance its task processing capabilities in vertical domains. Attached Figure Description

[0020] The accompanying drawings, which are included to provide a further understanding of this application and form part of this application, illustrate exemplary embodiments and are used to explain this application, but do not constitute an undue limitation of this application. In the drawings: Figure 1 A schematic flowchart of a model training method provided for an exemplary embodiment of this application; Figure 2 A schematic diagram illustrating the collaborative operation and joint training of the driving model and the response model provided for an exemplary embodiment of this application; Figure 3 A flowchart illustrating a dialogue processing method provided in an exemplary embodiment of this application; Figure 4 A schematic diagram of the structure of an electronic device provided for an exemplary embodiment of this application. Detailed Implementation

[0021] To make the objectives, technical solutions, and advantages of this application clearer, the technical solutions of this application will be clearly and completely described below in conjunction with specific embodiments and corresponding drawings. Obviously, the described embodiments are only a part of the embodiments of this application, and not all of them. Based on the embodiments in this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0022] The terminology used in the embodiments of this application is for the purpose of describing particular embodiments only and is not intended to limit the application. The singular forms “a,” “the,” and “the” used in the embodiments of this application and the appended claims are also intended to include the plural forms unless the context clearly indicates otherwise. “Multiple” generally includes at least two, but does not exclude the inclusion of at least one. “A plurality” generally includes at least two, but does not exclude the inclusion of at least one.

[0023] It should be understood that the term "and / or" used in this article is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, and B existing alone. Additionally, the character " / " in this article generally indicates that the preceding and following related objects have an "or" relationship.

[0024] It should also be noted that the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a product or system that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a product or system. Without further limitation, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the product or system that includes said element.

[0025] In LLM application scenarios, the prompts input to the LLM are mostly designed manually, resulting in inconsistent prompt quality and thus limiting the capabilities of the LLM.

[0026] To address the aforementioned technical problems, some embodiments of this application provide a solution, the core of which lies in: when deploying a generative dialogue system, following the principle of modular design, the overall architecture of the dialogue system is decomposed into a collaborative driving model and a response model, thereby decoupling the pre-processing stage from the generative response stage. The driving model acts as a pre-processing cognitive controller, primarily used to perform context engineering based on the input dialogue messages to generate high-level contextual guidance information. The driving model is used to reconstruct the context of the input dialogue messages, ensuring that the reconstructed contextual guidance information accurately and efficiently guides the response model to generate response results. The response model is used to accurately generate high-quality response results corresponding to the dialogue messages based on the contextual guidance information output by the driving model.

[0027] The dialogue system provided in this application can provide localized dialogue capabilities for highly sensitive data industries, such as government service agencies, financial institutions, and medical institutions, to meet their needs in terms of data security, compliance, and controllability. In the above scenarios, the driving model in the dialogue system can be deployed on the user's edge computing device or private device, such as in the user's local data center or private cloud environment, to ensure that dialogue-related data remains within the internal network environment. In particular, this deployment method is also applicable to regulatory-sensitive industries (such as the judicial and auditing industries), ensuring that the driving model is auditable, traceable, and meets specified compliance requirements. The response model can be deployed in a cloud environment to save the user's computing power costs.

[0028] In the embodiments of this application, the driving model is a model that can be post-trained. Post-training refers to a training method that, after the main training phase of the model, such as after pre-training or fine-tuning, further processes or lightweight optimizations are performed on the model for specific application scenarios, hardware platforms, or performance requirements to improve its inference efficiency, deployment capabilities, or performance on specific tasks.

[0029] In this embodiment, the number of parameters in the driving model is less than that in the response model. Fewer parameters mean fewer resources are needed for deployment and optimization. Therefore, in this embodiment, post-training of the driving model can be achieved with fewer resources. Furthermore, in the case of localized deployment of the driving model, optimization and iteration can be performed in the user's local or private environment according to the user's scenario-specific customization needs, thereby meeting the user's scenario-specific customization and optimization requirements for the driving model at a lower cost. In some embodiments, the number of parameters in the driving model can meet the needs of iteration on a single-card processor, such as a single-card GPU (Graphics Processing Unit), supporting rapid updates on a weekly or even daily basis, thus adapting to scenarios with frequent changes in service rules.

[0030] The driving model and response model provided in this application embodiment can be deep learning models with relatively large parameter sizes. This application embodiment does not limit the number of model parameters supported by the deep learning model used, aiming to meet actual needs. The number of parameters of the deep learning model used by the driving model is less than the number of parameters of the deep learning model used by the response model. The order of magnitude corresponding to the number of parameters of the driving model can be less than the order of magnitude corresponding to the amount of data of the response model; or, the number of parameters of the driving model and the number of parameters of the response model are at the same order of magnitude, but the amount of data of the driving model is less. This embodiment does not impose any restrictions.

[0031] The deep learning model involved in this application embodiment can be an artificial intelligence-based language model (LM) or a multimodal model (MM). A language model is a large-scale trained natural language processing (NLP) model. Built on deep learning technology and trained on a large training dataset, it learns language patterns from a vast amount of text, thus demonstrating powerful performance in handling natural language tasks such as language understanding, reasoning, writing, translation, and programming. Given a text (i.e., context), the language model attempts to predict the most likely next word. In some embodiments, the language model may employ an advanced neural network architecture, such as the Transformer architecture, to build its structure. This architecture enables the language model to capture complex patterns in the text and handle long-distance dependencies. During training, these parameters are continuously adjusted and optimized based on the difference between the language model's predictions and actual results to improve the prediction accuracy. After sufficient training, the language model possesses powerful generative capabilities, producing coherent and context-appropriate text content based on given prompts.

[0032] Multimodal models are artificial intelligence models capable of simultaneously understanding, processing, and generating multiple types of data (such as text, images, audio, and video), and establishing connections and interactions between different modalities. Multimodal models can employ advanced neural network architectures, such as the Transformer architecture, to construct a unified model structure. This structure encodes data from different modalities (such as text, images, and audio) into unified vector sequence representations and models complex semantic relationships within and between modalities through self-attention and cross-modal attention mechanisms, effectively capturing cross-modal alignment relationships and contextual dependencies. During training, the model continuously optimizes parameters through multi-task objectives (such as text-image contrast learning, mask reconstruction, and generative prediction), minimizing the difference between the model output and the true label, thereby learning the shared semantic space between modalities. After sufficient training, multimodal models possess powerful cross-modal understanding and generation capabilities, able to generate relevant images based on given text prompts, or answer questions and generate descriptions based on image content, achieving semantically consistent and context-coherent multimodal interactions.

[0033] In some embodiments of this application, a general-purpose language model with small parameters, pre-trained on a large general dataset, can be used as the base model for the response model. This base model can then be lightweightly post-trained for specific dialogue scenarios, hardware platforms, or performance requirements to improve its inference efficiency, deployment capabilities, or performance on specific tasks. Alternatively, a general-purpose large language model with large parameters, pre-trained on a large general dataset, can be used as the response model. The parameters of the response model can be frozen, preventing adjustments to the driving model from backpropagating to the response model. This reduces the risk of catastrophic forgetting due to post-training, while simultaneously indirectly enhancing the response model's ability to handle vertical domain tasks through post-training of the driving model.

[0034] The training method for the response model will be illustrated below with reference to the accompanying drawings.

[0035] Figure 1 A schematic flowchart of a model training method provided for an exemplary embodiment of this application is shown below. Figure 1 As shown, the method includes: Step 101: Obtain a sample of the dialogue message.

[0036] Step 102: Invoke the target-driven model to reconstruct the context of the dialogue message sample and obtain context guidance information.

[0037] Step 103: Input the context guidance information and the dialogue message sample into the response model, so that the response model generates a response result sample of the dialogue message sample under the guidance of the context guidance information; the number of parameters of the response model is greater than the number of parameters of the target-driven model.

[0038] Step 104: Optimize the target-driven model based on the quality scores of the response result samples.

[0039] This embodiment is used to train a goal-driven model in a dialogue system. The goal-driven model can be any driving model that works in conjunction with the response model.

[0040] In some optional embodiments, the target-driven model can be a general-purpose driving model applicable to multiple dialogue scenarios. This general-purpose driving model can reconstruct the context of dialogue message samples in different dialogue scenarios and learn the ability to reconstruct the context of dialogue message samples in different dialogue scenarios.

[0041] In some alternative embodiments, the target-driven model can be a vertical domain model applicable to a target dialogue scenario, where the target dialogue scenario refers to the dialogue scenario to which the dialogue message sample belongs. In this implementation, the dialogue system may include a response model and at least one driving model that collaborates with the response model, wherein different driving models correspond to different dialogue scenarios. The target-driven model can be adapted to the target dialogue scenario to learn the ability to reconstruct the context of dialogue message samples in the target dialogue scenario, enabling the reconstructed contextual guidance information to accurately and efficiently guide the response model to generate response results. Based on this implementation, the response model can be reused in multiple vertical domains, reducing the risk of an explosive operational burden caused by a "one domain, one response model" approach.

[0042] In this embodiment, a dialogue scenario refers to a specific industry or field that the dialogue system targets. For example, the dialogue system can be applied to scenarios such as medical dialogue, financial dialogue, government dialogue, and industrial scheduling dialogue. Different dialogue scenarios target different user groups, and the dialogue messages in different scenarios are initiated around different topics or knowledge areas. This embodiment can train driving models corresponding to different dialogue scenarios based on dialogue message samples from different dialogue scenarios.

[0043] In step 101, the dialogue message sample can be a question, request, or instruction, with the purpose of obtaining information, resolving problems, or completing a task. Dialogue message samples are typically presented in natural language and can be factual, operational / instructional, comparative, or explanatory questions. In locally deployed dialogue systems targeting industries with highly sensitive data, user-provided dialogue messages often relate to internal enterprise / organization policies, processes, or data, and the dialogue system relies on the enterprise / organization's proprietary knowledge base to respond to these dialogue messages.

[0044] In some optional embodiments, after obtaining a dialogue message sample, the target dialogue scenario corresponding to the dialogue message sample can be identified, and the driving model corresponding to the target dialogue scenario can be determined as the target driving model from at least one driving model that collaborates with the response model. Optionally, the target dialogue scenario corresponding to the dialogue message sample can be identified through keyword matching or rule matching. For example, when the dialogue message sample includes "medicine," the target dialogue scenario can be determined to be a medical dialogue scenario; when the dialogue message sample includes "loan interest rate," the target dialogue scenario can be determined to be a financial dialogue scenario. Optionally, the target dialogue scenario corresponding to the dialogue message sample can also be identified based on the metadata context of the dialogue message sample. For example, when the metadata context includes the user identity of the questioner in the dialogue message sample, if the questioner's user identity is a patient seeking medical treatment, the target dialogue scenario is determined to be a medical dialogue scenario; if the questioner's user identity is a bank customer, the target dialogue scenario is determined to be a financial dialogue scenario. As another example, when the metadata context includes the entry source of the dialogue message sample, if the entry source is a hospital's electronic medical consultation platform, the target dialogue scenario is determined to be a medical dialogue scenario; if the entry source is a bank client, the target dialogue scenario is determined to be a financial dialogue scenario.

[0045] In step 102, the target-driven model can be invoked to reconstruct the context of the dialogue message sample and obtain context guidance information.

[0046] Context reconstruction refers to the process of reconstructing the contextual data required for reasoning based on the characteristics of the dialogue scenario, with the goal of improving reasoning effectiveness. This involves dynamically constructing a context optimization mechanism that matches the prompts provided by the target dialogue scenario. Contextual data reconstruction may include at least one of the following operations: dynamic perception, filtering, structuring organization, content enhancement, and task planning. Based on context reconstruction, the scenario-specific features of the target dialogue scenario can be injected into the input context of the response model, thereby guiding the model to generate professional and standardized outputs that conform to the scenario. The scenario features of the target dialogue scenario may include at least one of the following: task processing standards, role setting features, knowledge constraint features, compliance requirement features, response style features, and risk control strategies.

[0047] In some optional embodiments, the context reconstruction operation may include: dynamically assembling and rewriting the context data related to the dialogue message sample under the constraints of scene features. The context data may include at least one of: language context, user context, environmental context, and task context. Language context may include at least one of historical dialogue data and external knowledge information. User context may include at least one of user preference information, attribute information, and user dialogue intent. Environmental context may include information describing scene features. Task context may include at least one of task rule data and task constraint data. For example, different context templates can be preset according to the scene features of different dialogue scenarios. The goal-driven model can select the corresponding context template according to the target dialogue scenario corresponding to the dialogue message sample, and assemble or rewrite the context data related to the dialogue message sample according to the context template to obtain context guidance information.

[0048] In some alternative embodiments, the context reconstruction operation may include: performing task planning for the response task of the dialogue message sample under the constraints of scene features. Task planning refers to the process of automatically decomposing the complex or ambiguous task objectives implicit in the dialogue message sample into a series of logically clear, executable, sequentially related, or conditionally dependent subtasks. Based on the context reconstruction operation, a semantically coherent, informationally complete, behaviorally executable, and task-constrained dynamic context can be constructed to guide the subsequent response model in generating accurate, reliable, and operable responses.

[0049] Compared to traditional methods of manually writing prompts or using static prompts, this step utilizes a goal-driven model to generate contextual guidance information. This allows the guidance information to integrate one or more elements from user intent, task planning results, domain knowledge, task constraints, historical states, and output specifications. Consequently, ambiguous or complex dialogue messages are transformed into information more easily understood by the response model, guiding it to perform accurate, compliant, and efficient reasoning in specific tasks, thus improving the quality of response model generation and output stability.

[0050] In step 103, the context guidance information and the dialogue message sample can be input into the response model so that the response model can generate a response result sample of the dialogue message sample under the guidance of the context guidance information.

[0051] In some optional embodiments, contextual guidance information can be injected into the reasoning process of the response model using contextual injection. This contextual guidance information can be injected into the prompts of the response model along with one or more other contextual information required for the response model to perform reasoning. The following will combine... Figure 2 An example is provided.

[0052] In some embodiments, the goal-driven model and the response model can be integrated into the AI ​​Agent platform and can replace traditional modules of the AI ​​Agent platform as standard plug-ins. For example, an agent encapsulating a goal-driven model can replace the router module in the AI ​​Agent platform; similarly, an agent encapsulating a response model can replace the execution module in the AI ​​Agent platform, such as a code execution agent (CodeAgent) and a web browsing / interaction agent (BrowserUseAgent). In such embodiments, such as Figure 2 As shown, contextual guidance information, agent profile information, task objectives, retrieved "contextual learning examples", sub-agent action information, step information, and format specifications can be injected into the prompts of the response model to form structured, high-quality prompts.

[0053] The agent profile information describes the static attributes of the agent encapsulating the response model in the dialogue system, such as its role, capabilities, permissions, and background knowledge, to avoid role confusion. The task objective provides direction for the entire dialogue processing task, guiding subtask decomposition and decision-making. "Contextual learning examples" refer to demonstration samples of similar tasks obtained from historical successful cases by retrieving data from the knowledge base, providing task paradigms to guide the response model in imitating correct behavior. Sub-agent action information records the actions, output results, or state updates performed by other collaborating sub-agents, enabling state synchronization and collaborative reasoning among multiple agents. Step information records the stage information in the current task's step-by-step process, and format information describes the hard requirements for the output format.

[0054] After inputting contextual guidance information into the response model, the model can parse this information, construct a reasoning context, and generate binding text within that context to produce a preliminary response. The response model can then verify the preliminary response for factual consistency, format compliance, and security policies, and output a sample of the verified response.

[0055] After obtaining the response sample, in step 104, the target-driven model can be optimized based on the quality score of the response sample. The response sample is generated by the response model based on the contextual guidance information of the target-driven model. The quality of the response sample can reflect the quality of the contextual guidance information to a certain extent. Therefore, optimizing the target-driven model based on the quality score of the response sample can guide the target-driven model to achieve autonomous exploration, feedback learning, and continuous evolution. This allows for better injection of scene information by optimizing the context reconstruction capability of the target-driven model without causing catastrophic forgetting in the response model, thereby enhancing the overall dialogue system's ability to handle vertical domain issues.

[0056] In this embodiment, a target-driven model capable of performing context reconstruction and a response model for content generation work together. The target-driven model outputs contextual guidance information based on dialogue messages. This contextual guidance information serves as the core carrier for activating or regulating the generation capabilities of the response model, replacing traditional methods of manually writing or statically setting prompts. This achieves data-driven, scenario-adaptive dynamic prompt generation, which is beneficial for improving the generation quality and output stability of the response model. The target-driven model has fewer parameters than the response model, making it easier to run and optimize the target-driven model at a lower cost in edge computing or private deployment scenarios, thus meeting the dual needs of users in the target dialogue scenario for data security and computational efficiency. Furthermore, the context reconstruction capability of the target-driven model can be continuously optimized based on the quality of the response results output by the response model, thereby indirectly guiding the response model to continuously enhance its task processing capabilities in the vertical domain. Therefore, when facing dialogue scenarios with changing requirements, training the target-driven model according to new requirements in the dialogue scenario can effectively guide and enhance the response model in a scenario-specific way, without retraining the response model, thereby reducing the user's cost of using and maintaining the response model.

[0057] It should also be noted that in this modular design framework, the goal-driven model and the response model are decoupled, and the goal-driven model is a lightweight model with a small number of parameters, enabling it to support rapid retraining or scenario replacement. This allows for the rapid training of multiple driver models with different versions adapted to the same scenario, or multiple driver models adapted to different dialogue scenarios, improving the scalability and scenario migration capabilities of the response model and the overall dialogue system, effectively reducing the overall operational complexity and version management burden of the dialogue system.

[0058] In some optional embodiments, one way to call the target-driven model corresponding to the target dialogue scenario to reconstruct the context of the dialogue message sample and obtain context guidance information may include: in the target-driven model, performing task planning based on the dialogue message sample to obtain the target processing logic corresponding to the dialogue message sample, and outputting the context guidance information based on the target processing logic.

[0059] Optionally, the goal-driven model performs task planning operations, which may include: goal decomposition, execution order and dependency modeling, and tool / resource binding. The goal decomposition operation breaks down the implicit task goal in the dialogue message sample into multiple specific sub-tasks. For example, "analyzing problem A" is broken down into multiple sub-tasks such as "acquiring data B, performing operation C, and performing operation D." The execution order and dependency modeling operation clarifies the sequence, parallelism, or conditional branches between sub-tasks. For example, when data B meets the first condition, operation C is executed; when data B meets the second condition, operation D is executed. The tool / resource binding operation specifies the required capabilities or external interfaces for each sub-task. For example, a database query tool is bound to the sub-task "acquiring data B," a data statistics tool is bound to the sub-task "performing operation C," and a tool for generating data reports is bound to the sub-task "performing operation D."

[0060] It's important to note that different dialogue scenarios employ different task processing standards. These standards constrain the task processing methods and output content of the dialogue system to meet industry norms. For example, the government services industry emphasizes policy authority, requiring the dialogue system to cite authoritative official documents and policy documents, and to quote the original text rather than subjective summaries or speculations when reasoning about the answer. The financial industry emphasizes regulatory compliance, requiring the dialogue system to limit the data it cites to exchange announcements, listed company financial reports, and compliant financial databases. When reasoning about the answer, if product returns or market judgments are involved, standardized risk warnings must be added. The healthcare industry emphasizes life safety, requiring the dialogue system not to diagnose diseases, prescribe medications, or recommend specific treatment plans, but only to provide factual information from reliable sources such as drug instructions and authoritative clinical guidelines.

[0061] Based on this, in some optional embodiments, the driving model corresponding to different dialogue scenarios can perform context reconstruction operations differently according to the task criteria of different dialogue scenarios. The following will continue to illustrate this using any target dialogue scenario as an example.

[0062] Optionally, the goal-driven model in the target dialogue scenario can decompose the dialogue message sample into a sub-task sequence under the constraints of the task processing standard corresponding to the target dialogue scenario.

[0063] In some optional implementations, the goal-driven model can use the task processing standards corresponding to the target dialogue scenario as prior rules. Based on these standards, it determines the sub-task types and decomposes the dialogue message sample according to the sub-task types to obtain the sub-task sequence. For example, if the target dialogue scenario is a medical scenario and the dialogue message sample is: "Can patients with hypertension take ibuprofen?", the goal-driven model, when decomposing the dialogue message sample, considers that task processing annotations in medical dialogue scenarios do not recommend specific treatment plans. Therefore, it will not generate sub-tasks like "recommended medication," but instead breaks them down into compliant sub-tasks such as "searching for contraindications in drug instructions" and "finding clinical guidelines regarding the interaction between ibuprofen and hypertension." Based on this, the types and boundaries of sub-tasks can be preemptively shaped during the task goal decomposition process.

[0064] In other alternative implementations, the goal-driven model can use the task processing standards corresponding to the target dialogue scenario as a posteriori rules. The goal-driven model can decompose the dialogue message sample to obtain an initial sub-task sequence, and then optimize this initial sub-task sequence according to the task processing standards corresponding to the target dialogue scenario. For example, optionally, after decomposing the dialogue message sample to obtain initial sub-tasks, the goal-driven model can dynamically modify the decomposed initial sub-tasks according to the task processing standards corresponding to the target dialogue scenario to rewrite non-compliant initial sub-tasks into compliant forms; or, the goal-driven model can filter out initial sub-tasks that do not conform to the task processing standards corresponding to the target dialogue scenario; or, the goal-driven model can inject mandatory sub-tasks corresponding to the target dialogue scenario into the already decomposed initial sub-task sequence, such as mandating the addition of a "risk warning requiring regulatory compliance" sub-task in a financial dialogue scenario, and so on.

[0065] Based on this implementation method, the goal-driven model can embed task processing standards for the target dialogue scenario into the task planning stage, thereby guiding or constraining the task planning process to plan safe, compliant, and professional processing logic. This, in turn, enables the dialogue system to flexibly adapt to different dialogue scenarios, improve the accuracy of responses, and reduce the probability of compliance risks or factual errors in the responses.

[0066] After performing the above operations, the goal-driven model can obtain a set of subtask sequences to drive the response model. Based on the subtask sequence, the goal-driven model can generate the target processing logic corresponding to the dialogue message sample.

[0067] After obtaining the target processing logic based on the above implementation methods, the target-driven model can output contextual guidance information according to the target processing logic. In some optional embodiments, the target-driven model can directly output the target processing logic as contextual guidance information. In other optional embodiments, the target-driven model can perform knowledge retrieval-Augmented Generation (RAG) operations based on the dialogue message sample to obtain the knowledge information needed to respond to the dialogue message sample. For example, in some scenarios, the dialogue message sample depends on the user's (e.g., an enterprise's) internal knowledge documents. The target-driven model can retrieve some relevant knowledge documents from these knowledge documents based on the dialogue message sample as knowledge information. The target-driven model can then structure and organize the knowledge information and the target processing logic before outputting it as contextual guidance information.

[0068] After obtaining the contextual guidance information output by the goal-driven model, the response model can be invoked based on the contextual guidance information and the dialogue message sample. The response model is used to generate a response result sample for the dialogue message sample under the guidance of the contextual guidance information. In some optional embodiments, the contextual guidance information includes the goal processing logic corresponding to the dialogue message sample. This goal processing logic defines the sequence of subtasks required to complete the task goal corresponding to the dialogue message sample. The sequence of subtasks includes the subtasks to be executed, the execution order dependencies between the subtasks, etc. The response model can identify the sequence of subtasks in the goal processing logic and execute each subtask sequentially according to the execution order between the subtasks.

[0069] Based on this implementation method, the goal-driven model injects task criteria from the target dialogue scenario into the contextual guidance information, effectively guiding the response model to output dialogue under the task criteria. On the one hand, high-quality contextual guidance information dynamically guides the response model to unleash its greater potential; on the other hand, the task criteria injected into the contextual guidance information enable the adaptation of dialogue output capabilities to the target dialogue scenario.

[0070] In this embodiment, the goal-driven model is jointly trained based on its own input data and the output of the response model. During training, the goal-driven model learns the ability to generate high-level guidance information adapted to the target dialogue scenario, enabling this high-level guidance information to effectively guide the response model, ensuring that the response model's output meets the requirements of the target dialogue scenario. The training method of the goal-driven model will be described below by example.

[0071] It is worth noting that, in some optional embodiments, when the response model works in conjunction with driving models corresponding to multiple different dialogue scenarios, a unified reward model can be deployed for multiple different dialogue scenarios. This reward model can provide scalable and dynamically customized quality evaluation services for different dialogue scenarios based on the scenario evaluation criteria of each scenario. Optionally, the reward model can be deployed on a cloud server and can be used to evaluate response result samples from a full set of samples corresponding to different dialogue scenarios. Optionally, the reward model can be a discriminative model or a generative model; this embodiment does not impose any limitations. Specifically, the discriminative reward model outputs a quality score based on the input dialogue message and response result to indicate the quality of the response. The generative reward model outputs natural language feedback based on the input dialogue message and response result, such as "The answer is factually incorrect" or "It is suggested to be more concise," to explain the evaluation reasoning. The feedback output by the generative reward model can be further converted into a quantified quality score.

[0072] In some optional embodiments, after obtaining the response result sample output by the response model, a reward model can be invoked to score the response result sample using the target evaluation method corresponding to the target dialogue scenario, thereby obtaining a quality score for the response result sample. Based on this quality score, the target-driven model is then optimized. The reward model is used to evaluate the quality of the responses generated by the response model and outputs a quality score, which guides the target-driven model to optimize through reinforcement learning.

[0073] In some optional embodiments, before invoking the reward model and scoring the response sample of the dialogue message sample using the target evaluation method corresponding to the target dialogue scenario, the scenario response difficulty corresponding to the dialogue message sample can be obtained, and an evaluation method adapted to the scenario response difficulty can be selected as the target evaluation method from at least one evaluation method. Optionally, the at least one evaluation method may include: point-wise evaluation method and / or pair-wise evaluation method. Point-wise evaluation method is used to give a relatively accurate quantitative score to a single answer of the response model, such as 1 to 5 points, or a real number from -2 to +3. Pair-wise evaluation method is used to compare the merits of a single answer output by the response model with a reference answer. The scenario response difficulty adapted to point-wise evaluation method is lower than the scenario response difficulty corresponding to pair-wise evaluation method.

[0074] The scenario response difficulty is used to describe the technical complexity and uncertainty level of generating high-quality, compliant, verifiable, and acceptable response samples for the dialogue messages in that scenario. Optionally, the scenario response difficulty can be divided into different difficulty levels, such as high difficulty, medium difficulty, and low difficulty, with different difficulty levels resulting in different scenario response difficulties.

[0075] In some embodiments, a fixed difficulty level can be preset according to the industry and field type corresponding to the target dialogue scenario. For example, when applied to fields such as medical, financial, and legal, the scenario response difficulty level can be set to high difficulty; when applied to fields such as government services and educational tutoring, the scenario response difficulty level can be set to medium difficulty; when set to general encyclopedia, entertainment recommendations, or life knowledge, the scenario response difficulty level can be set to low difficulty.

[0076] In some alternative embodiments, a scenario evaluation standard for the target dialogue scenario provided by the user of the dialogue system can be obtained, and at least one difficulty analysis dimension can be used to analyze the response difficulty of the scenario evaluation standard to obtain the scenario response difficulty corresponding to the dialogue message sample. The scenario evaluation standard is used to define a set of specifications applicable to the dialogue scenario, which is used to evaluate whether the response results generated by the response model meet the quality requirements of the dialogue scenario. Optionally, the at least one difficulty analysis dimension includes at least one of the following: clarity of the scenario evaluation standard, knowledge certainty, security risk level, and reasoning complexity.

[0077] The clarity of the evaluation criteria refers to whether there are clear, operational, and consensus-based standards for judging the accuracy or excellence of an answer. For example, if an answer is objective or can be automatically evaluated based on existing data, the clarity of the evaluation criteria is high, and the difficulty level of the scenario response is low. If an answer is subjective, relies on expert experience to judge its quality, or requires balancing multiple dimensions to evaluate its quality, the clarity of the evaluation criteria is low, and the difficulty level of the scenario response is high. Knowledge certainty refers to whether the knowledge supporting the answer is stable, authoritative, uncontroversial, and traceable. If the knowledge supporting the answer is stable and authoritative, such as the answer originating from mathematical formulas or legal provisions, the difficulty level of the scenario response is low. Conversely, if the knowledge supporting the answer is dynamic or controversial, such as limited clinical evidence for a certain drug, the difficulty level of the scenario response is high. The safety risk level refers to the severity of the negative consequences that may result from generating incorrect, inappropriate, or out-of-bounds answers, including legal, health, financial, and reputational risks. If the consequences of an incorrect answer are minor, such as recommending the wrong movie or novel, the scenario's difficulty level is low. If an incorrect / inappropriate answer could lead to serious consequences, such as giving incorrect medication or investment advice, the scenario's difficulty level is high. Reasoning complexity refers to the depth of cognitive steps and logical structure complexity required to arrive at an answer from a question. If an answer can be obtained after a retrieval, the required reasoning depth is low, and the scenario's difficulty level is low. If multiple-hop reasoning, conditional judgments, or uncertain reasoning are required, the scenario's difficulty level is high.

[0078] Based on this, in some optional embodiments, calculable or evaluable metrics can be designed for each difficulty analysis dimension. The following will provide illustrative examples for each difficulty analysis dimension.

[0079] Optionally, when using clarity to perform difficulty analysis on scene evaluation criteria, a natural language processing model can be used to determine whether the scene evaluation criteria contain ambiguous words, such as “good,” “reasonable,” “appropriate,” or subjective expressions, and output a clarity score (such as a score of 0–1) or a level (such as high, medium, or low).

[0080] Optionally, when using knowledge certainty to perform difficulty analysis on scenario evaluation criteria, the system can check whether the knowledge required to answer the question exists in an authoritative knowledge base based on the scenario evaluation criteria, and output the knowledge certainty level (e.g., high, medium or low) or confidence score based on the judgment result.

[0081] Optionally, when using security risk levels to analyze the difficulty of scenario evaluation criteria, the scenario evaluation criteria text can be input into a pre-trained sensitive content classifier to detect whether it is explicitly or implicitly associated with high-risk topics such as medical, legal, and financial matters; rule matching can be used to determine whether the scenario evaluation criteria contain strongly constrained wording. If so, it indicates a low fault tolerance rate and serious risk consequences; then, the security risk level (e.g., high, medium, or low) or risk label can be output by combining the risk domain type and the constraint strength of the wording.

[0082] Optionally, when using reasoning complexity to analyze the difficulty of scenario evaluation criteria, NLP techniques or template matching can be used to extract conditional statements (e.g., "if...then..."), multiple constraints (e.g., "must satisfy both A and B"), and exception clauses from the scenario evaluation criteria, and to count their logical nesting levels and the number of constraint couplings. Simultaneously, keywords (e.g., "step-by-step explanation," "based on") can be used to determine whether the criterion requires the model to explicitly present the reasoning process. Based on the above analysis, the minimum number of reasoning steps or logical dependency depth required to meet the criterion can be further estimated, and combined with the strength of the explicit reasoning requirement, a comprehensive determination of the reasoning complexity level (e.g., high, medium, or low) can be made, or a quantitative score can be output.

[0083] Then, the evaluation results of each difficulty analysis dimension can be fused to obtain the final scenario response difficulty. Optionally, the fusion method can include: weighting the evaluation results of multiple difficulty analysis dimensions according to rules. For example, if the evaluation result of the security risk level is "high," the evaluation results of other difficulty analysis dimensions can be ignored, and the overall scenario response difficulty can be directly set to "high." Optionally, the fusion method can include: weighted averaging of the evaluation results of multiple difficulty analysis dimensions. For example, different weights can be assigned to each difficulty evaluation dimension, such as a weight of 0.4 for security risk level, 0.2 for clarity, 0.2 for knowledge certainty, and 0.2 for reasoning complexity; then, a weighted average is performed based on the evaluation results of multiple difficulty analysis dimensions and their weights to obtain the comprehensive difficulty as the scenario response difficulty. Optionally, a difficulty classification model can also be trained using a decision tree or classifier, inputting the evaluation results of each difficulty evaluation dimension, and the difficulty classification model can directly output the scenario response difficulty, which will not be elaborated further.

[0084] Based on this implementation method, the reward model can determine the evaluation method for the response results samples generated by the response model according to accurate scenario evaluation criteria or rules, rather than relying on fuzzy human preferences. This allows it to provide more professional evaluation results that meet the requirements of the dialogue system's users. Secondly, the reward model can specifically evaluate the quality of response results samples in the target dialogue scenario according to the scenario evaluation criteria corresponding to the target dialogue scenario. This helps guide the goal-driven model to learn the scenario features of the target dialogue scenario, thereby enabling the goal-driven model to adapt to the context reconstruction task corresponding to a specific dialogue scenario.

[0085] Optionally, if the target evaluation method is a positive evaluation method, then a reference response result sample corresponding to the dialogue message sample can be obtained. Optionally, multiple contextual guidance information samples generated by the target-driven model based on the dialogue message sample can be obtained, such as... Figure 2 The context guidance information SG1, SG1…SGn, where n is a positive integer, is shown. These multiple context guidance information samples and the dialogue message sample are input into the response model, so that the response model generates multiple response result samples for the dialogue message sample under the guidance of these multiple context guidance information samples, such as… Figure 2 The response results shown are GL1, LG1…LGn. A ranking model can be used to sort these multiple response result samples, obtaining a ranking result. Based on this ranking result, a reference response result sample can be selected from these multiple response result samples. The reference response result sample can be the middle-ranked sample among the multiple response result samples. The reward model can use this sample as a quality anchor to evaluate the quality scores of other response result samples, obtaining the individual quality scores of each of the multiple response result samples, such as… Figure 2The response results shown are R1, R1…Rn.

[0086] Taking any dialogue message sample as an example, the dialogue message sample, the response result sample, and the reference response result sample can be combined into a sample pair. For example, if the dialogue message sample is labeled Q, the response result sample is A1, and the reference response result sample is A2, then (Q, A1, A2) is formed. This sample pair is input into the reward model so that, given the dialogue message sample, the reward model calculates the reward scores for the response result sample and the reference response result sample. Based on the reward score of the reference response result sample, the reward score of the response result sample is binarized to obtain the quality score of the response result sample. Optionally, if the reward score of the response result sample is greater than the reward score of the reference response result sample, the reward score of the response result sample can be updated to 1; if the reward score of the response result sample is less than or equal to the reward score of the reference response result sample, the reward score of the response result sample can be updated to 0. In this implementation, using the reference response result sample as a benchmark and binarizing the reward score of the response result sample allows for a more accurate assessment of the advantages or disadvantages of the response result sample relative to the reference response result sample, thereby more accurately evaluating the quality of the response result sample.

[0087] Based on this implementation, when evaluating the quality of the response model's generation, a scenario evaluation standard provided by the user of the dialogue system is introduced. This allows the quality evaluation results to characterize the degree to which the overall output of the dialogue system adapts to the quality requirements of the target dialogue scenario. Multiple iterative optimizations of the target-driven model based on these quality evaluation results can parameterize the quality requirements and scenario features of the target dialogue scenario into the target-driven model, enhancing its ability to guide the response model to adapt to the target dialogue scenario, thereby improving the overall dialogue system's adaptability to the target dialogue scenario.

[0088] It should also be noted that in some application scenarios, such as government services, the cost of obtaining high-quality labeled corpus samples is high, while the cost of obtaining scenario problems and scenario evaluation criteria is low. Based on this, with the help of a reward model, the goal-driven model can be guided to autonomously explore, learn from feedback, and continuously evolve in real-world scenarios at low cost, based on the scenario evaluation criteria provided by the user. This achieves dynamic optimization and self-improvement of capabilities without requiring a large number of labeled corpus samples.

[0089] Figure 3 This is a flowchart illustrating a model training method provided in an exemplary embodiment of this application, which is used to train a target-driven model in a dialogue system. Figure 3 As shown, the model training method may include the following steps: Step 301: Obtain the target dialogue message.

[0090] Step 302: Invoke the target-driven model to reconstruct the context of the target dialogue message and obtain context guidance information.

[0091] Step 303: Input the context guidance information and the target dialogue message into the response model so that the response model generates a response result to the target dialogue message under the guidance of the context guidance information; the number of parameters of the response model is greater than the number of parameters of the target-driven model.

[0092] This embodiment can be executed by a dialogue system that responds to received dialogue messages. The dialogue system includes at least a goal-driven model and a response model. The goal-driven model can be deployed on the edge computing device or private device of the user of the dialogue system to ensure that dialogue-related data remains within the intranet environment. The response model can be deployed in a cloud environment to save the user's computing power costs.

[0093] The goal-driven model performs context engineering based on the input dialogue message to generate high-level contextual guidance information. The response model generates a response corresponding to the dialogue message based on the contextual guidance information output by the goal-driven model. The training method for the goal-driven model can be found in the description of the foregoing embodiments, and will not be repeated here.

[0094] In step 301, the target dialogue message can be input by the user into the dialogue system, or it can be any dialogue message provided by the upstream system or service of the dialogue system; this embodiment does not impose any restrictions. The target dialogue message can be a question, a request, or an instruction. Target dialogue messages are usually presented in natural language and can be factual questions, operational / instructional questions, comparative questions, or explanatory questions. In locally deployed dialogue systems targeting industries with highly sensitive data, the dialogue messages provided by users usually involve internal enterprise / organization policies, processes, or data, and the dialogue system relies on the enterprise / organization's private knowledge base to respond to these dialogue messages.

[0095] In step 302, the dialogue system can invoke the target-driven model to reconstruct the context of the target dialogue message and obtain context guidance information.

[0096] In some alternative embodiments, the goal-driven model can be a general model applicable to a variety of dialogue scenarios.

[0097] In some alternative embodiments, the target-driven model can be a vertical domain model applicable to the target dialogue scenario, where the target dialogue scenario refers to the dialogue scenario to which the target dialogue message belongs. In this embodiment, the dialogue system may include a response model and at least one driving model that collaborates with the response model, wherein different driving models correspond to different dialogue scenarios. The target-driven model is adapted to the target dialogue scenario to reconstruct the context of the dialogue messages in the target dialogue scenario, and to enable the reconstructed context guidance information to accurately and efficiently guide the response model to generate response results. Optionally, the at least one driving model includes at least one of the following: a driving model corresponding to a medical dialogue scenario, a driving model corresponding to a financial dialogue scenario, a driving model corresponding to a government affairs dialogue scenario, and a driving model corresponding to an industrial scheduling scenario. The dialogue system can flexibly select the driving model that collaborates with the response model in the dialogue message processing task according to the dialogue scenario to which the dialogue message belongs, so that the same response model can adapt to different dialogue scenarios.

[0098] Context reconstruction refers to the process of reconstructing the contextual data required for reasoning based on the characteristics of the dialogue scenario, with the goal of improving reasoning effectiveness. This involves dynamically constructing a context optimization mechanism that matches the prompts provided by the target dialogue scenario. Contextual data reconstruction may include at least one of the following operations: dynamic perception, filtering, structuring organization, content enhancement, and task planning. Based on context reconstruction, the scenario-specific features of the target dialogue scenario can be injected into the input context of the response model, thereby guiding the model to generate professional and standardized outputs that conform to the scenario. The scenario features of the target dialogue scenario may include at least one of the following: task processing standards, role setting features, knowledge constraint features, compliance requirement features, response style features, and risk control strategies.

[0099] In some optional embodiments, the context reconstruction operation may include dynamically assembling and rewriting context data related to the target dialogue message under the constraints of scene features. The data related to the target dialogue message may include at least one of the following: historical dialogue data, external knowledge information, task rule data, and task constraint data. For example, different context templates can be preset according to the scene features of different dialogue scenarios. The target-driven model corresponding to the target dialogue scenario can select the corresponding context template based on the target dialogue scenario corresponding to the target dialogue message, and assemble or rewrite the context data related to the target dialogue message according to the context template to obtain context guidance information.

[0100] In some alternative embodiments, the context reconstruction operation may include: performing task planning for the response task of the target dialogue message under the constraints of scene features. Task planning refers to the process of automatically decomposing the complex or ambiguous task objectives implicit in the target dialogue message into a series of logically clear, executable, sequentially related, or conditionally dependent subtasks. Based on the context reconstruction operation, a semantically coherent, informationally complete, behaviorally executable, and task-constrained dynamic context can be constructed to guide the subsequent response model in generating accurate, reliable, and operable responses. The context reconstruction operation based on the goal-driven model enables the contextual guidance information to integrate one or more of the following: user intent, task planning results, domain knowledge, task constraints, historical states, and output specifications. This transforms ambiguous or complex dialogue messages into information more easily understood by the response model, thereby guiding the response model to perform accurate, compliant, and efficient reasoning in a specific task.

[0101] In step 303, the contextual guidance information and the target dialogue message can be input into the response model, so that the response model generates a response to the target dialogue message under the guidance of the contextual guidance information. The response model can parse the contextual guidance information, construct an inference context, and perform constraint text generation within the inference context to generate a preliminary response. The response model can verify the factual consistency, format compliance, and security policy of the preliminary response, and output a verified response result. The dialogue system can output the response result to the user in the form of a dialogue.

[0102] In this embodiment, a target-driven model capable of performing context reconstruction and a response model for content generation are employed in collaborative operation. The target-driven model can output contextual guidance information based on dialogue messages. This contextual guidance information serves as the core carrier for activating or regulating the generation capability of the response model, replacing the traditional method of manually writing or statically setting prompts. This achieves data-driven, scenario-adaptive dynamic prompt generation, which is beneficial for improving the generation quality and output stability of the response model. The target-driven model has fewer parameters than the response model, making it easier to run and optimize the target-driven model at a lower cost in edge computing or private deployment scenarios, thus meeting the dual needs of users in the target dialogue scenario for data security and computational efficiency.

[0103] It should be noted that the execution subject of each step of the method provided in the above embodiments can be the same device, or the method can be executed by different devices. For example, the execution subject of steps 101 to 104 can be device A; or the execution subject of steps 101 and 102 can be device A, and the execution subject of step 103 can be device B; and so on.

[0104] Furthermore, some processes described in the above embodiments and accompanying drawings include multiple operations appearing in a specific order. However, it should be clearly understood that these operations may not be executed in the order they appear herein, or they may be executed in parallel. The operation numbers, such as 101, 102, etc., are merely used to distinguish different operations and do not represent any execution order. Additionally, these processes may include more or fewer operations, and these operations may be executed sequentially or in parallel. It should be noted that the descriptions such as "first" and "second" in this document are used to distinguish different messages, devices, modules, etc., and do not represent a sequential order, nor do they limit "first" and "second" to different types.

[0105] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties. Furthermore, the collection, use and processing of the relevant data must comply with the relevant laws, regulations and standards of the relevant countries and regions, and corresponding operation entry points are provided for users to choose to authorize or refuse.

[0106] Figure 4 This illustration shows a structural diagram of an electronic device provided in an exemplary embodiment of this application. For example... Figure 4 As shown, the electronic device includes: a memory 401, a processor 402, and a communication component 403.

[0107] Memory 401 is used to store computer programs and can be configured to store various other data to support operation on the electronic device. Examples of this data include instructions for any application or method used to operate on the electronic device, data structures, contact data, phone book data, messages, pictures, videos, etc.

[0108] Processor 402, coupled to memory 401, is used to execute computer programs in memory 401 for: acquiring dialogue message samples; invoking a target-driven model to reconstruct the context of the dialogue message samples to obtain context guidance information; inputting the context guidance information and the dialogue message samples into a response model, so that the response model generates response result samples of the dialogue message samples under the guidance of the context guidance information; the number of parameters of the response model is greater than the number of parameters of the target-driven model; and optimizing the target-driven model based on the quality score of the response result samples.

[0109] Optionally, before invoking the target driving model, the processor 402 is further configured to: identify the target dialogue scenario corresponding to the dialogue message sample; determine the driving model corresponding to the target dialogue scenario from at least one driving model that collaborates with the response model as the target driving model; wherein different driving models correspond to different dialogue scenarios in the at least one driving model.

[0110] Optionally, the processor 402 calls the target-driven model corresponding to the target dialogue scenario to reconstruct the context of the dialogue message sample and obtain context guidance information, including: in the target-driven model, performing task planning based on the dialogue message sample to obtain the target processing logic corresponding to the dialogue message sample, and outputting the context guidance information based on the target processing logic.

[0111] Optionally, in the target-driven model, the processor 402 performs task planning based on the dialogue message sample to obtain the target processing logic corresponding to the dialogue message sample, including: under the constraints of the task processing standard corresponding to the target dialogue scenario, performing target decomposition on the dialogue message sample to obtain a sub-task sequence; and generating the target processing logic based on the sub-task sequence.

[0112] Optionally, under the constraints of the task processing standard corresponding to the target dialogue scenario, the processor 402 performs target decomposition on the dialogue message sample to obtain a subtask sequence, including: determining the subtask type according to the task processing standard corresponding to the target dialogue scenario, and performing target decomposition on the dialogue message sample according to the subtask type to obtain the subtask sequence; or, performing target decomposition on the dialogue message sample to obtain an initial subtask sequence, and optimizing the initial subtask sequence according to the task processing standard corresponding to the target dialogue scenario to obtain the subtask sequence.

[0113] Optionally, the processor 402 optimizes the target-driven model based on the quality score of the response result sample, including: calling the reward model, using the target evaluation method corresponding to the target dialogue scenario to score the response result sample, and obtaining the quality score of the response result sample; and optimizing the target-driven model based on the quality score.

[0114] Optionally, before the processor 402 calls the reward model and scores the response result sample of the dialogue message sample using the target evaluation method corresponding to the target dialogue scenario, the processor 402 further includes: obtaining the scenario response difficulty corresponding to the dialogue message sample; selecting an evaluation method that matches the scenario response difficulty from at least one evaluation method as the target evaluation method, wherein the at least one evaluation method includes: a point-based evaluation method and / or a pair-based evaluation method, wherein the scenario response difficulty matched by the point-based evaluation method is less than the scenario response difficulty corresponding to the pair-based evaluation method.

[0115] Optionally, the processor 402 obtains the scenario response difficulty corresponding to the dialogue message sample, including: obtaining the scenario evaluation criteria of the target dialogue scenario provided by the user; and using at least one difficulty analysis dimension to perform response difficulty analysis on the scenario evaluation criteria to obtain the scenario response difficulty corresponding to the dialogue message sample.

[0116] Optionally, the processor 402 invokes a reward model to score the response result sample of the dialogue message sample using the target evaluation method corresponding to the target dialogue scenario, thereby obtaining a quality score for the response result sample. This includes: if the target evaluation method is a peer evaluation method, obtaining a reference response result sample corresponding to the dialogue message sample; forming a sample pair from the dialogue message sample, the response result sample, and the reference response result sample; inputting the sample pair into the reward model so that, given the dialogue message sample, the reward model calculates the reward scores of the response result sample and the reference response result sample; and binarizing the reward score of the response result sample based on the reward score of the reference response result sample to obtain a quality score for the response result sample.

[0117] Figure 4 The illustrated electronic device can also be used to execute a dialogue processing method. The processor 402 is coupled to the memory 401 and is used to execute a computer program in the memory 401 for: acquiring a target dialogue message; invoking a target-driven model to reconstruct the context of the target dialogue message and obtain context guidance information; the target-driven model is trained using the model training method described in the embodiments of this application; inputting the context guidance information and the target dialogue message into a response model so that the response model generates a response result for the target dialogue message under the guidance of the context guidance information; the number of parameters of the response model is greater than the number of parameters of the target-driven model.

[0118] Optionally, before invoking the target driving model, the processor 402 is further configured to: identify the target dialogue scenario corresponding to the target dialogue message; determine the driving model corresponding to the target dialogue scenario as the target driving model from at least one driving model that collaborates with the response model; wherein different driving models correspond to different dialogue scenarios in the at least one driving model.

[0119] Optionally, the at least one driving model includes at least one of the following: a driving model corresponding to a medical dialogue scenario, a driving model corresponding to a financial dialogue scenario, a driving model corresponding to a government dialogue scenario, and a driving model corresponding to an industrial scheduling scenario.

[0120] Furthermore, such as Figure 4 As shown, the electronic device also includes other components such as a power supply component 404, a display component 405, and an audio component 406. Figure 4 The diagram only shows some components and does not mean that the electronic device includes only these components. Figure 4 The components shown. Figure 4 In this embodiment, the components within the dashed boxes are optional, not mandatory, and their specific requirements depend on the product form of the electronic device. The electronic device in this embodiment can be a terminal device such as a desktop computer, laptop computer, smartphone, or IoT device, or a server-side device such as a conventional server, cloud server, or server array. If the electronic device in this embodiment is a terminal device such as a desktop computer, laptop computer, or smartphone, it may include... Figure 4 The components within the dashed box; if the electronic device in this embodiment is implemented as a conventional server, cloud server, or server array, it may be omitted. Figure 4 The component within the dashed box.

[0121] The memory 401 can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random-access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic storage, flash memory, magnetic disk, or optical disk.

[0122] The communication component 403 is configured to facilitate wired or wireless communication between the device containing the communication component and other devices. The device containing the communication component can access wireless networks based on communication standards, such as 2G (e.g., Global System for Mobile Communications (GSM)), 3G (e.g., Wideband Code Division Multiple Access (WCDMA), 4G (e.g., Long Term Evolution (LTE)), 4G+ (e.g., LTE-Advanced (LTE-A)), or 5G (5th Generation Mobile Communication Technology), or combinations thereof. In one exemplary embodiment, the communication component receives broadcast signals or broadcast-related information from an external broadcast management system via a broadcast channel.

[0123] The power supply component 404 is used to provide power to various components of the device in which the power supply component is located. The power supply component may include a power management system, one or more power supplies, and other components associated with generating, managing, and distributing power to the device in which the power supply component is located.

[0124] The display component includes a screen, which may include a liquid crystal display (LCD) and a touch panel (TP). If the screen includes a touch panel, the screen can be implemented as a touchscreen to receive input signals from the user. The touch panel includes one or more touch sensors to sense touches, swipes, and gestures on the touch panel. The touch sensors can sense not only the boundaries of the touch or swipe action but also the duration and pressure associated with the touch or swipe operation.

[0125] An audio component may be configured to output and / or input audio signals. For example, the audio component includes a microphone (MIC) configured to receive external audio signals when the device containing the audio component is in an operating mode, such as call mode, recording mode, or voice recognition mode. The received audio signals may be further stored in memory or transmitted via a communication component. In some embodiments, the audio component also includes a speaker for outputting audio signals.

[0126] In this embodiment, a target-driven model capable of performing context reconstruction and a response model for content generation work together. The target-driven model can output contextual guidance information based on dialogue messages. This contextual guidance information serves as the core carrier for activating or regulating the generation capability of the response model, replacing the traditional method of manually writing or statically setting prompts. This achieves data-driven, scenario-adaptive dynamic prompt generation, which is beneficial for improving the generation quality and output stability of the response model. The target-driven model has fewer parameters than the response model, making it easier to run and optimize the target-driven model at a lower cost in edge computing or private deployment scenarios, thus meeting the dual needs of users in the target dialogue scenario for data security and computational efficiency. Furthermore, the context reconstruction capability of the target-driven model can be continuously optimized based on the quality of the response results output by the response model, thereby indirectly guiding the response model to continuously enhance its task processing capabilities in the vertical domain.

[0127] Accordingly, embodiments of this application also provide a computer-readable storage medium storing a computer program, which, when executed by a processor, enables the processor to implement the steps in the above-described method embodiments. The computer-readable storage medium includes volatile or non-volatile components, or a combination thereof, and can be removable or non-removable. Examples of computer-readable storage media include, but are not limited to, phase-change random access memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random-access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), flash memory or other memory technologies, CD-ROM, Digital Video Disc (DVD) or other optical storage, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other non-transmission medium. Accordingly, this application also provides a computer program product, which includes a computer program or instructions that, when executed by a processor, cause the processor to implement the steps in the above method embodiments. It should be understood that each step or combination of steps in the above method flow can be implemented by the computer program or instructions. Furthermore, these computer programs or instructions can be applied to the processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device, enabling the processor of the general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing device to function as an apparatus for implementing the corresponding functions in the above method embodiments.

[0128] It should also be noted that the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, product, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, product, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, product, or apparatus that includes said element.

[0129] The above description is merely an embodiment of this application and is not intended to limit the scope of this application. Various modifications and variations can be made to this application by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the scope of the claims of this application.

Claims

1. A model training method, characterized in that, include: Obtain a sample of the conversation message; The target-driven model is invoked to reconstruct the context of the dialogue message sample, thereby obtaining context guidance information. The contextual guidance information and the dialogue message sample are input into the response model, so that the response model generates a response result sample of the dialogue message sample under the guidance of the contextual guidance information; the number of parameters of the response model is greater than the number of parameters of the target-driven model; The target-driven model is optimized based on the quality scores of the response results samples.

2. The method according to claim 1, characterized in that, Before invoking the target-driven model, the following is also included: Identify the target dialogue scenario corresponding to the dialogue message sample; From at least one driving model that collaborates with the response model, a driving model corresponding to the target dialogue scenario is determined as the target driving model; among the at least one driving model, different driving models correspond to different dialogue scenarios.

3. The method according to claim 1, characterized in that, The target-driven model corresponding to the target dialogue scenario is invoked to reconstruct the context of the dialogue message sample, obtaining context guidance information, including: In the target-driven model, task planning is performed based on the dialogue message sample to obtain the target processing logic corresponding to the dialogue message sample, and the context guidance information is output based on the target processing logic.

4. The method according to claim 3, characterized in that, In the target-driven model, task planning is performed based on the dialogue message sample to obtain the target processing logic corresponding to the dialogue message sample, including: Under the constraints of the task processing standard corresponding to the target dialogue scenario, the dialogue message sample is decomposed to obtain a sub-task sequence. The target processing logic is generated based on the sub-task sequence.

5. The method according to claim 4, characterized in that, Under the constraints of the task processing criteria corresponding to the target dialogue scenario, the dialogue message sample is decomposed to obtain a sub-task sequence, including: Based on the task processing criteria corresponding to the target dialogue scenario, the sub-task types are determined, and the dialogue message samples are decomposed according to the sub-task types to obtain the sub-task sequence; or... The dialogue message sample is decomposed to obtain an initial subtask sequence, and the initial subtask sequence is optimized according to the task processing standard corresponding to the target dialogue scenario to obtain the subtask sequence.

6. The method according to claim 2, characterized in that, Based on the quality scores of the response result samples, the target-driven model is optimized, including: The reward model is invoked, and the target evaluation method corresponding to the target dialogue scenario is used to score the response result sample to obtain the quality score of the response result sample; The target-driven model is optimized based on the quality score.

7. The method according to claim 6, characterized in that, Before calling the reward model and scoring the response results of the dialogue message sample using the target evaluation method corresponding to the target dialogue scenario, the process further includes: Obtain the scenario response difficulty corresponding to the dialogue message sample; From at least one evaluation method, an evaluation method that is compatible with the difficulty of the scenario response is selected as the target evaluation method. The at least one evaluation method includes: point-based evaluation method and / or pair-based evaluation method. The scenario response difficulty that the point-based evaluation method is compatible with is less than the scenario response difficulty corresponding to the pair-based evaluation method.

8. The method according to claim 7, characterized in that, Obtaining the scenario response difficulty corresponding to the dialogue message sample includes: Obtain the scene evaluation criteria for the target dialogue scenario provided by the user; By employing at least one difficulty analysis dimension, the response difficulty of the scenario evaluation criteria is analyzed to obtain the scenario response difficulty corresponding to the dialogue message sample.

9. The method according to claim 6, characterized in that, The reward model is invoked to evaluate the response samples of the dialogue message samples using the target evaluation method corresponding to the target dialogue scenario, thereby obtaining a quality score for the response samples, including: If the target evaluation method is a positive evaluation method, then obtain the reference response result sample corresponding to the dialogue message sample; The dialogue message sample, the response result sample, and the reference response result sample are combined into a sample pair; The sample pairs are input into the reward model so that, given the dialogue message sample, the reward model calculates the reward scores for the response result sample and the reference response result sample. Based on the reward score of the reference response result sample, the reward score of the response result sample is binarized to obtain the quality score of the response result sample.

10. A dialogue processing method, characterized in that, include: Retrieve target dialogue messages; The target-driven model is invoked to reconstruct the context of the target dialogue message, thereby obtaining context guidance information. The target-driven model is trained using the method described in any one of claims 1-9; The contextual guidance information and the target dialogue message are input into the response model so that the response model generates a response result to the target dialogue message under the guidance of the contextual guidance information; the number of parameters of the response model is greater than the number of parameters of the target-driven model.

11. The method according to claim 10, characterized in that, Before invoking the target-driven model, the following is also included: Identify the target dialogue scenario corresponding to the target dialogue message; From at least one driving model that collaborates with the response model, a driving model corresponding to the target dialogue scenario is determined as the target driving model; among the at least one driving model, different driving models correspond to different dialogue scenarios.

12. The method according to claim 11, characterized in that, The at least one driving model includes at least one of the following: a driving model corresponding to a medical dialogue scenario, a driving model corresponding to a financial dialogue scenario, a driving model corresponding to a government dialogue scenario, and a driving model corresponding to an industrial scheduling scenario.

13. An electronic device, characterized in that, include: Memory and processor; The memory is used to store one or more computer instructions; The processor is configured to execute one or more computer instructions for performing the steps of the method according to any one of claims 1-12.

14. A computer-readable storage medium storing a computer program, characterized in that, When a computer program is executed by a processor, it is able to perform the steps of the method described in any one of claims 1-12.

15. A computer program product, characterized in that, include: A computer program / instruction that, when executed by a processor, enables the implementation of the steps in the method according to any one of claims 1-12.

Images