Model training methods, dialogue processing methods, and dialogue systems

Abstract

This application provides a model training method, a dialogue processing method, and a dialogue system. The model training method includes: retrieving at least one reference knowledge from a retrieval knowledge base using a target retrieval decision model based on dialogue data samples; generating a dialogue response for the dialogue data samples based on the reference knowledge and the dialogue data samples; and training the target retrieval decision model based on the referencing behavior of the reference knowledge during the generation of the dialogue response. This method trains the target retrieval decision model based on the referencing behavior of the reference knowledge during the generation of the dialogue response, thereby improving the efficiency of model training compared to related technologies that rely on standard answers.

Description

Technical Field

[0001] This application relates to the field of cloud computing, and in particular to a model training method, a dialogue processing method, and a dialogue system. Background Technology

[0002] In Artificial Intelligence (AI) question answering, it is necessary to output structured and highly credible answers to the questions posed by users. For example, in the Retrieval-Augmented Generation (RAG) framework, the search agent can retrieve text information related to the user's question from a knowledge base, and the generator model uses a Large Language Model (LLM) to generate the final answer based on the retrieved text information.

[0003] In related technologies, the training of Search Agents relies on manually labeled standard answers, resulting in low training efficiency. Summary of the Invention

[0004] This application provides a model training method, a dialogue processing method, and a dialogue system to address the problem of low model training efficiency.

[0005] Firstly, this application provides a model training method, the method comprising:

[0006] Based on dialogue data samples, a target retrieval decision model is used to recall at least one reference knowledge from the retrieval knowledge base;

[0007] Based on the reference knowledge and the dialogue data sample, generate a dialogue response for the dialogue data sample;

[0008] The target retrieval decision model is trained based on the referencing behavior of the reference knowledge when generating the dialogue response.

[0009] In one possible implementation, the referencing behavior is marked by a reference marker contained in the dialogue response; the reference marker is used to indicate the target knowledge referenced when generating the dialogue response; the target knowledge comes from the at least one reference knowledge.

[0010] The step of training the target retrieval decision model based on the referencing behavior of the reference knowledge when generating the dialogue response includes:

[0011] Parse the reference markers in the dialogue response and extract the target knowledge referenced by the reference markers;

[0012] The target retrieval decision model is trained based on the target knowledge and the reference knowledge.

[0013] In one possible implementation, training the target retrieval decision model based on the target knowledge and the reference knowledge includes:

[0014] Based on the target knowledge and the reference knowledge, a first reward value is obtained; the first reward value is used to characterize the reference signal-to-noise ratio of the reference knowledge; the reference signal-to-noise ratio is the proportion of reference knowledge referenced by the dialogue response in the reference knowledge.

[0015] The target retrieval decision model is trained based on the first reward value.

[0016] In one possible implementation, obtaining the first reward value based on the target knowledge and the reference knowledge includes:

[0017] Based on the target knowledge, determine the number of reference knowledge referenced by the dialogue response in the reference knowledge;

[0018] The first reward value is obtained based on the number of reference knowledge cited and the number of reference knowledge.

[0019] In one possible implementation, obtaining the first reward value based on the target knowledge and the reference knowledge includes:

[0020] Based on the target knowledge, determine the referenced knowledge texts that are referenced by the dialogue response in the reference knowledge;

[0021] Based on the cited reference knowledge text, and the reference knowledge, the first reward value is obtained.

[0022] In one possible implementation, the method further includes:

[0023] Based on the dialogue data sample and the dialogue response, a second reward value is obtained; the second reward value is used to characterize the generation quality of the dialogue response.

[0024] The step of training the target retrieval decision model based on the first reward value includes:

[0025] Based on the first reward value and the second reward value, obtain the total reward value;

[0026] The target retrieval decision model is trained based on the total reward value.

[0027] In one possible implementation, obtaining the total reward value based on the first reward value and the second reward value includes:

[0028] Based on the training objective, or the training progress of the objective retrieval decision model, determine the first weight corresponding to the first reward value, and the second weight corresponding to the second reward value;

[0029] The total reward value is determined based on the product of the first reward value and the first weight, and the product of the second reward value and the second weight.

[0030] In one possible implementation, generating a dialogue response based on the reference knowledge and the dialogue data sample includes:

[0031] Based on the reference knowledge and the dialogue data sample, prompt words are generated; the prompt words are used to instruct the generation model to mark the referenced target knowledge in the dialogue response by reference tags when generating the dialogue response based on the reference knowledge and the dialogue data sample.

[0032] Based on the prompt words, a generative model is used to generate a dialogue response for the dialogue data sample.

[0033] Secondly, this application provides a dialogue processing method, the method comprising:

[0034] Obtain user conversation data;

[0035] Based on the dialogue data, at least one reference knowledge is retrieved from the retrieval knowledge base using the target retrieval decision model trained by the method described in any of the first aspects.

[0036] Based on the reference knowledge and the dialogue data, a dialogue response to the dialogue data is generated.

[0037] Thirdly, this application provides a dialogue processing method, the method comprising:

[0038] In response to input operations applied to the user interface, the user-input dialogue data is displayed on the user interface.

[0039] In response to a dialogue operation on the user interface, the system uses a target retrieval decision model trained as described in any of the first aspects to retrieve at least one reference knowledge from a retrieval knowledge base, and generates a dialogue response based on the reference knowledge and the dialogue data, and displays the dialogue response on the user interface.

[0040] Fourthly, this application provides a dialogue system, which includes: a client and a server;

[0041] The client is used to receive dialogue data input by the user;

[0042] The server is configured to, based on the dialogue data, use a target retrieval decision model trained by any one of the methods described in the first to third aspects to recall at least one reference knowledge from the retrieval knowledge base, and generate a dialogue response based on the reference knowledge and the dialogue data.

[0043] In one possible implementation, the server includes: a search agent, a search engine, and a generative model;

[0044] The search agent is used to retrieve a list of candidate knowledge from the retrieval knowledge base based on dialogue data through the search engine, and to obtain at least one reference knowledge from the list of candidate knowledge using a target retrieval decision model trained by any one of the methods in the first to third aspects.

[0045] The generative model is used to generate a dialogue response based on the reference knowledge and the dialogue data.

[0046] Fifthly, this application provides an electronic device, comprising:

[0047] At least one processor; and a memory communicatively connected to said at least one processor;

[0048] The memory stores instructions executable by the at least one processor to cause the electronic device to perform the method as described in any one of the first to third aspects.

[0049] In a sixth aspect, this application provides a computer-readable storage medium storing computer-executable instructions that, when executed by a processor, implement the method as described in any one of the first to third aspects.

[0050] In a seventh aspect, this application provides a computer program product, including a computer program that, when executed by a processor, implements the method as described in any one of the first to third aspects.

[0051] The model training method, dialogue processing method, and dialogue system of this application retrieve at least one reference knowledge from a retrieval knowledge base based on dialogue data samples using a target retrieval decision model; generate a dialogue response for the dialogue data samples based on the reference knowledge and the dialogue data samples; and train the target retrieval decision model based on the referencing behavior of the reference knowledge during the generation of the dialogue response. The method of this application trains the target retrieval decision model based on the referencing behavior of the reference knowledge during the generation of the dialogue response. During the training process, the target retrieval decision model is trained by relying on the referencing behavior of the reference knowledge in the dialogue response. Compared with the training methods in related technologies that rely on standard answers, this method eliminates the need for manual annotation, thereby improving the efficiency of model training. Attached Figure Description

[0052] The accompanying drawings, which are included to provide a further understanding of this application and form part of this application, illustrate exemplary embodiments and are used to explain this application, but do not constitute an undue limitation of this application. In the drawings:

[0053] Figure 1 This is a flowchart illustrating the model training methods in related technologies;

[0054] Figure 2 Flowchart of the model training method provided for an exemplary embodiment of this application Figure 1 ;

[0055] Figure 3 Flowchart of the model training method provided for an exemplary embodiment of this application Figure 2 ;

[0056] Figure 4 Flowchart of the model training method provided for an exemplary embodiment of this application Figure 3 ;

[0057] Figure 5 A flowchart illustrating a first dialogue processing method provided for an exemplary embodiment of this application;

[0058] Figure 6 A flowchart illustrating a second dialogue processing method provided for an exemplary embodiment of this application;

[0059] Figure 7 A schematic diagram of the structure of a dialogue system provided as an exemplary embodiment of this application;

[0060] Figure 8 A schematic diagram of the structure of the model training apparatus provided in an exemplary embodiment of this application;

[0061] Figure 9A schematic diagram of the structure of a first dialogue processing apparatus provided for an exemplary embodiment of this application;

[0062] Figure 10 A schematic diagram of the structure of a second dialogue processing apparatus provided as an exemplary embodiment of this application;

[0063] Figure 11 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Detailed Implementation

[0064] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties. Furthermore, the collection, use and processing of the relevant data must comply with relevant laws, regulations and standards, and corresponding operation entry points are provided for users to choose to authorize or refuse.

[0065] To make the objectives, technical solutions, and advantages of this application clearer, the technical solutions of this application will be clearly and completely described below in conjunction with specific embodiments and corresponding drawings. Obviously, the described embodiments are only a part of the embodiments of this application, and not all of them. Based on the embodiments in this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0066] Below, in order to facilitate understanding of the technical solution of this application, the concepts involved in this application will be explained first.

[0067] 1. RAG Framework: This is an AI-powered question-answering framework that combines retrieval and generation. First, it retrieves relevant reference knowledge related to the user's question. Then, using LLM (Local Level Management), it obtains the corresponding answer to the user's question based on the reference knowledge and the user's question.

[0068] 2. Reinforcement Learning (RL): is the process by which an agent learns optimal behavioral strategies by interacting with the environment and receiving rewards or punishments.

[0069] In the RAG framework's search agent, the Search Agent interacts with the search engine to retrieve relevant reference knowledge from the knowledge base related to the user's dialogue data. This reference knowledge is then processed, such as filtered, sorted, and / or truncated. The Search Agent transmits the processed reference knowledge to the Generator, which, based on the reference knowledge and dialogue data, uses LLM to generate the final answer.

[0070] Below, in conjunction with Figure 1 The training process of the Search Agent in related technologies is explained.

[0071] Figure 1 This is a flowchart illustrating the model training methods in related technologies. Please refer to [link / reference]. Figure 1 The Search Agent, based on dialogue data samples, inputs an extracted keyword list into a search engine. The search engine then retrieves documents related to the dialogue data samples from a knowledge base, processes these documents, and transmits the processed documents to the Generator. The Generator, based on the documents and dialogue data samples, generates dialogue responses using an LLM (Local Language Management) framework. The Search Agent is then trained using the dialogue responses output by the Generator, referencing standard answers corresponding to manually annotated dialogue data samples.

[0072] Therefore, the training of the Search Agent relies on manually labeled standard answers, resulting in a strong dependence on standard answers during training. However, manually labeling standard answers is inefficient, leading to low training efficiency for the Search Agent.

[0073] To address the aforementioned issues, this application proposes a model training method. Based on the generative model's referencing behavior of reference knowledge during dialogue response generation, a retrieval decision model for the Search Agent is trained, thus achieving Search Agent training. The method in this application relies on the behavior of referencing reference knowledge in the generated answers to train the Search Agent model. Compared to related technologies that rely on training based on standard answers, this method eliminates the need for manual annotation, thereby improving model training efficiency.

[0074] This application applies to the model training phase of an AI question-answering system. For example, in an AI question-answering system, the Search Agent is responsible for acquiring reference knowledge such as reference documents and text blocks, and the Generator generates answers based on this reference knowledge, relying on an LLM (Local Modeling Language). The AI ​​question-answering system can receive dialogue data input by the user and output corresponding answers. In the training of this AI question-answering system, the model training method of this application embodiment can be used for step-by-step training to obtain a trained Search Agent. Based on the trained Search Agent, the AI ​​question-answering system can output better questions and answers to user questions. Below, the solution in the embodiments of this application is described using the training module used to train the AI ​​question-answering system as the execution entity. The training module can execute the solution in the embodiments of this application.

[0075] The technical solutions shown in this application will now be described in detail through specific embodiments. It should be noted that the following embodiments may exist independently or in combination with each other; for identical or similar content, the description will not be repeated in different embodiments.

[0076] Figure 2 Flowchart of the model training method provided for an exemplary embodiment of this application Figure 1 Please see. Figure 1 The methods may include:

[0077] S201. Based on the dialogue data sample, use the target retrieval decision model to recall at least one reference knowledge from the retrieval knowledge base.

[0078] Dialogue data samples are input samples used to train a target retrieval decision model, representing dialogues input by the user. For example, dialogue data samples can be retrieved historical user dialogues.

[0079] Target retrieval decision models are trained models used to determine search term generation and document filtering strategies. For example, a reinforcement learning-based search agent has the ability to generate search keywords and filter documents.

[0080] A retrieval knowledge base is a database that stores the content of external documents. For example, a retrieval knowledge base includes documents such as academic papers and industry reports.

[0081] Reference knowledge is the content of documents retrieved from a knowledge base. For example, reference knowledge can be a document, or a portion of text from a document.

[0082] For example, the training module can retrieve at least one reference knowledge related to the dialogue data sample from the retrieval knowledge base using the Search Agent in the AI ​​question-answering system. The Search Agent can interact with the search engine to retrieve text blocks with high relevance from the retrieval knowledge base based on the text relevance between the text blocks in the retrieval knowledge base and the dialogue data sample, thus obtaining at least one reference knowledge. Taking multiple reference knowledge as an example, the training module can sort the text blocks from highest to lowest relevance and select the top few text blocks, i.e., the multiple text blocks with high relevance, as multiple reference knowledge.

[0083] For example, the search engine retrieves at least one text block from the knowledge base based on the cosine similarity between the vector of a text block in the knowledge base and the vector of the dialogue data sample. A higher cosine similarity indicates a higher relevance. At least one reference knowledge is obtained by acquiring at least one text block whose cosine similarity is greater than a preset threshold.

[0084] For example, the training module can filter multiple text blocks obtained from the retrieved knowledge base to obtain at least one reference knowledge. For example, the training module can employ a semantic analysis model from related technologies, such as a Transformer-based cross-encoder, to jointly encode the dialogue data sample with each of the multiple text blocks, capturing interactive semantic information. An attention mechanism is used to evaluate the semantic fit between the text blocks and the dialogue data sample, and at least one text block with a high semantic fit is retained to obtain at least one reference knowledge.

[0085] S202. Based on reference knowledge and dialogue data samples, generate dialogue responses from the dialogue data samples.

[0086] The dialogue response is the answer obtained from the dialogue data sample based on reference knowledge and dialogue data sample.

[0087] For example, the training module generates dialogue responses with reference knowledge numbers using a Generator in the AI ​​question-answering system. The Generator can generate dialogue responses based on reference knowledge and dialogue data samples using LLM (Local Level Management). When generating the dialogue response, text that references reference knowledge is annotated to indicate the reference knowledge number. For example, the Generator numbers the reference knowledge, with each number corresponding to a specific reference knowledge. If a sentence in the Generator's generated text is marked with a superscript "...", the Generator can further enhance the dialogue response. [1] The '' indicates that the text references the knowledge source numbered 1.

[0088] For example, the Generator, based on the format of citing documents in natural language, indicates the referenced document and outputs a dialog response. For instance, the dialog response might include: "Based on 'Data…' in Report X." In this approach, the Generator indicates in natural language that "Data…" refers to the referenced knowledge, and the corresponding document is Report XX.

[0089] S203. Based on the referencing behavior of reference knowledge during the generation of dialogue response, the target retrieval decision model is trained.

[0090] Citing behavior refers to the act of referencing sources in a dialogue response, that is, referencing some or all of the knowledge in the referenced knowledge. Citing behavior is used to provide training signals for the target retrieval decision model. For example, in a dialogue response, the document number of the referenced knowledge is labeled for a portion of the text.

[0091] For example, the training module can determine a reward value based on a preset reward value calculation rule, taking into account the ratio of the number of cited references to the number of retrieved references. A higher ratio of cited references indicates higher quality retrieved references. The reward value calculation rule can be set based on the principle that a higher ratio of cited references results in a higher reward value. The training module can feed the reward value back to the Search Agent in the AI ​​question-answering system to train the parameters influencing decision-making within the Search Agent, thus achieving reinforcement learning for the Search Agent.

[0092] For example, the AI ​​question-answering system can determine a reward value based on a preset reward value calculation rule, taking into account the proportion of the text volume of cited references relative to the text volume of retrieved references. A higher proportion of cited references indicates higher quality of the retrieved references. The reward value calculation rule can be set based on the principle that a higher proportion of cited references results in a higher reward value. The training module can feed the reward value back to the Search Agent in the AI ​​question-answering system to optimize the parameters influencing decision-making within the Search Agent, thereby achieving reinforcement learning for the Search Agent.

[0093] The model training method in this embodiment of the application, based on dialogue data samples, uses a target retrieval decision model to recall at least one reference knowledge from a retrieval knowledge base; based on the reference knowledge and dialogue data samples, it generates a dialogue response for the dialogue data samples; and based on the referencing behavior of the reference knowledge during the generation of the dialogue response, it trains the target retrieval decision model. The method in this embodiment of the application trains the target retrieval decision model based on the referencing behavior of the reference knowledge during the generation of the dialogue response, thus achieving the training of the target retrieval decision model. During the training process, it relies on the referencing behavior of the reference knowledge in the dialogue response to train the target retrieval decision model. Compared with related technologies that rely on standard answers for training, it does not require manual annotation, thereby improving the efficiency of model training.

[0094] The following describes a specific implementation of how the training module trains the target retrieval decision model based on the referencing behavior of reference knowledge when generating dialogue responses.

[0095] Optionally, referencing behavior is indicated by reference markers contained in the dialogue response. Reference markers are used to indicate the target knowledge referenced when generating the dialogue response; the target knowledge comes from at least one reference knowledge.

[0096] For example, a quotation mark can be a superscript. For instance, if text is marked with the superscript " [1] The '' indicates that the source of the text is the reference knowledge with reference number 1.

[0097] For example, a citation marker can be parentheses. For instance, if text is marked with parentheses “(1)”, it indicates that the text is citing reference knowledge with citation number 1. As another example, if text is marked with parentheses “(X report)”, it indicates that the text is citing reference knowledge named “X report”.

[0098] Accordingly, the training module trains the target retrieval decision model based on the referencing behavior of reference knowledge when generating dialogue responses. It can parse the reference markers in the dialogue responses and extract the target knowledge referenced by the reference markers. Based on the target knowledge and the reference knowledge, the target retrieval decision model is trained.

[0099] Target knowledge is the reference knowledge cited by the dialogue response. For example, if text is marked with the superscript "..." [1] If the reference number is 1, then the target knowledge is the reference knowledge.

[0100] For example, after the Generator generates a dialogue response, the training module first parses the citation numbers marked in the Generator's dialogue response to obtain the reference indicators corresponding to the citation numbers, thus obtaining the target knowledge. Subsequently, the training module compares the target knowledge with the reference knowledge, calculates the quantity and / or text volume proportion of the target knowledge within the reference knowledge, and obtains a reward value based on this comparison result, where the comparison result is the quantity and / or text volume proportion. The training module feeds back the reward value to the Search Agent, i.e., the target retrieval decision model, to train the target retrieval decision model. For example, the Search Agent can adjust its decision parameters based on its own learning mechanism.

[0101] In the above embodiments, referencing behavior is marked with reference tags contained in the dialogue response. The quality of the dialogue response can be quantified by the target knowledge referenced by the reference tags, and a reward value can be obtained. The Search Agent is trained based on the reward value.

[0102] It should be understood that the method by which the target retrieval decision model trains itself based on the feedback parameters can refer to the training methods of models in related technologies, and this application does not impose any restrictions on it.

[0103] The method in this application, by parsing referencing behavior and extracting target knowledge, trains a target retrieval decision model based on the target knowledge and reference knowledge, thereby achieving precise optimization of target retrieval decisions. The quality of dialogue responses is quantified through referencing behavior, enabling the training of the target retrieval decision model. This method, relying on the referencing of reference knowledge in dialogue responses to train the target retrieval decision model, eliminates the need for manual annotation compared to related technologies that rely on standard answers. This not only reduces training costs but also improves model training efficiency.

[0104] Regarding the trained model, the target retrieval decision model obtained by the model training method in this application embodiment is based on reinforcement learning training and has the ability to handle complex logic and open-ended questions. Compared with the target retrieval decision models trained by relying on standard answers in related technologies, which have the problem of poor ability to handle open-ended questions, the target retrieval decision model trained in this application embodiment without relying on standard answers is based on a multi-angle, multi-hop retrieval strategy learned. In open-ended question-answering scenarios, it can provide better reference knowledge and improve the quality and credibility of the dialogue response output by the AI ​​question-answering system.

[0105] Optionally, during the training process of the target retrieval decision model based on target knowledge and reference knowledge, the training module may obtain a first reward value based on the target knowledge and reference knowledge, and train the target retrieval decision model based on the first reward value.

[0106] The first reward value is used to characterize the reference signal-to-noise ratio (SNR) of the reference knowledge. The reference SNR is the proportion of reference knowledge cited by the dialogue response. For example, the first reward value can be obtained based on the ratio of the number of target knowledge items to the number of reference knowledge items.

[0107] For example, the training module calculates a first reward value by comparing target knowledge with reference knowledge. For instance, the reference knowledge contains 10 documents, of which 3 are cited, representing 3 target knowledge items. Based on the ratio of the number of target knowledge items to the number of reference knowledge items, the first reward value is 0.3. This first reward value is fed back to the target retrieval decision model through Proximal Policy Optimization (PPO) to adjust its parameters for training. In PPO training, the probability ratio of taking the same action under the old and new policies is first calculated. By pruning this probability ratio, large policy updates are avoided, preventing training instability and achieving stable training of the target retrieval decision model based on the first reward value. It should be understood that the training method based on PPO can refer to specific implementations of PPO in related technologies, and this embodiment will not elaborate further.

[0108] Optionally, the training module may determine the number of referenced knowledge items cited by the dialogue response in the reference knowledge based on the target knowledge; and obtain a first reward value based on the number of referenced knowledge items and the total number of reference knowledge items.

[0109] The number of referenced knowledge items is the number of items in the reference knowledge that are referenced by the dialogue response. For example, if the reference knowledge includes 10 recalled documents, and 3 of them are referenced, then the number of referenced knowledge items that are referenced by the dialogue response is 3.

[0110] For example, the training module can obtain a first reward value based on the number of reference knowledge documents cited and the ratio of the number of reference knowledge documents. Taking 10 reference knowledge documents and 3 documents citing the reference knowledge documents as an example, the ratio of 3 to 10 is calculated, resulting in a first reward value of 0.3.

[0111] By calculating the number of referenced knowledge points cited by the dialogue response and obtaining a first reward value based on the number of referenced knowledge points and the total number of reference knowledge points, the quality of retrieved reference knowledge is accurately quantified. Training the target retrieval decision model based on the first reward value can improve the accuracy of model training.

[0112] For example, the training module can determine the referenced reference knowledge text cited by the dialogue response in the reference knowledge based on the target knowledge; and obtain a first reward value based on the referenced reference knowledge text and the reference knowledge.

[0113] Referenced knowledge text is the text of referenced knowledge cited in reference knowledge. For example, a dialog response might include the text "Data..." [1] Taking paragraph annotation as an example, the text "Data..." in the dialogue response comes from the reference knowledge with reference number 1, and the text of the reference knowledge with reference number 1 is the text of the referenced reference knowledge.

[0114] For example, the training module can obtain a first reward value based on the text length of the referenced knowledge text and the ratio of the text length of the referenced knowledge text. The text length can be represented by the number of characters. Taking the text length of the referenced knowledge text as `text_selected` and the text length of the referenced knowledge text as `text_cited` as an example, the ratio of `text_cited` to `text_selected` is calculated to obtain the first reward value `text_cited / text_selected`.

[0115] By calculating the referenced reference knowledge text cited by the dialogue response in the reference knowledge, and obtaining the first reward value based on the referenced reference knowledge text and the text of the reference knowledge, the quality of the retrieval reference indication is accurately quantified. Training the target retrieval decision model based on the first reward value can improve the accuracy of model training.

[0116] The method in this embodiment obtains a first reward value based on target knowledge and reference knowledge, thereby achieving quantitative feedback on the retrieved reference knowledge. The target retrieval decision model is trained based on the first reward value. The method in this embodiment relies on target knowledge and reference knowledge to quantify the quality of the retrieved reference knowledge in order to train the target retrieval decision model and improve its adaptive evolutionary capability under unlabeled data.

[0117] Figure 3 Flowchart of the model training method provided for an exemplary embodiment of this application Figure 2 Please see. Figure 3 It can include:

[0118] S301. Parse the reference markers in the dialogue response and extract the referenced target knowledge.

[0119] For example, the dialogue response includes "data..." [1] ... [2] Taking paragraph annotation as an example, the training module can extract reference number 1 and reference number 2 from the dialogue response, obtain the reference knowledge of reference number 1 and reference knowledge of reference number 2, and obtain the target knowledge.

[0120] S302. Based on the target knowledge and the reference knowledge, obtain the first reward value.

[0121] For example, the training module can obtain the text length of the reference knowledge with reference number 1 and the text length of the reference knowledge with reference number 2, and add the text length of the reference knowledge with reference number 1 and the text length of the reference knowledge with reference number 2 to obtain the text of the referenced reference knowledge that is referenced by the dialogue response, i.e., the text length of the target knowledge.

[0122] For example, the training module can calculate the ratio of the text length of the target knowledge to the text length of the reference knowledge to obtain a first reward value.

[0123] S303. Based on the first reward value, train the target retrieval decision model.

[0124] For example, the training module can feed back the first reward value to the target retrieval decision model to train the target retrieval decision model based on reinforcement learning.

[0125] It should be noted that, in Figure 3 The various model training steps (S301-S303) shown in the embodiments can be implemented with reference to the same or similar steps in the above embodiments. Figure 3 The various model training steps shown in the embodiments do not constitute a specific limitation on the model training method. In other embodiments of this application, the model training method may include... Figure 3 The embodiments may include more or fewer steps. For example, a model training method may include... Figure 3 Some steps in the embodiments, or, Figure 3 Some steps in the embodiments can be replaced by steps with the same function, or Figure 3 Some steps in the embodiments can be broken down into multiple steps, etc.

[0126] Optionally, the training module can also obtain a second reward value based on dialogue data samples and dialogue responses.

[0127] The second reward value is used to characterize the quality of the generated dialogue response. For example, the logical coherence of the dialogue response is scored using a quality assessment model to obtain the second reward value.

[0128] Accordingly, the training module can also obtain a total reward value based on the first reward value and the second reward value. The target retrieval decision model is then trained based on the total reward value.

[0129] For example, the training module can input dialogue data samples and dialogue responses into a natural language inference model based on related technologies. The natural language inference model evaluates whether the dialogue response contains dialogue data samples to obtain a quality score corresponding to the dialogue response. The training module can use this quality score as a second reward value and calculate a total reward value based on the weighted sum of the first and second reward values.

[0130] It should be understood that a natural language inference model is a large language model trained in related technologies. It can analyze the logical coherence of dialogue responses and the relevance of dialogue responses to dialogue data samples based on semantic analysis, thereby evaluating the quality of the dialogue responses. This application does not impose any limitations on the natural language inference model; large language models in related technologies can be referenced.

[0131] For example, the total reward value is obtained based on the first reward value and the second reward value. It can be as follows:

[0132]

[0133] in, This indicates reference knowledge. This indicates the referenced knowledge, or target knowledge, used in the dialogue response. This represents the first reward value. This represents the second reward value, used to characterize the quality of the generated dialogue response A. This indicates the weight corresponding to the first reward value. This indicates the weight corresponding to the second reward value.

[0134] The method in this application embodiment uses the quality of the generated dialogue response as a reward signal to achieve quantitative feedback on the retrieved reference knowledge, thereby improving the accuracy of the feedback. Based on the total reward value including the first reward value and the second reward value, the target retrieval decision model is trained, thereby improving the training accuracy of the target retrieval decision model.

[0135] Optionally, the training module may determine the first weight corresponding to the first reward value and the second weight corresponding to the second reward value based on the training objective or the training progress of the objective retrieval decision model; and determine the total reward value based on the product of the first reward value and the first weight and the product of the second reward value and the second weight.

[0136] The training objective prioritizes either the signal-to-noise ratio of referenced knowledge or the quality of generated dialogue responses.

[0137] Training progress refers to the progress of training the target retrieval decision model, and can be determined based on the ratio of the number of input dialogue data samples to the total number of dialogue data samples. For example, if the total number of dialogue data samples is 10,000 and the number of input dialogue data samples is 5,000, the current training progress is 50%.

[0138] For example, the training objective prioritizes the reference signal-to-noise ratio of the reference knowledge, i.e., increasing the first weight and decreasing the second weight. The training objective also prioritizes the generation quality of the dialogue response, i.e., increasing the second weight and decreasing the first weight. The total reward value is determined based on the product of the adjusted first reward value and the adjusted first weight, and the product of the second reward value and the adjusted second weight.

[0139] For example, in the initial training phase, the training model can assign a higher weight to the first reward value to quickly increase the citation ratio. In the later training phase, the weight of the second reward value is increased to train the generation quality. For instance, if the training progress is less than or equal to 50%, the first weight is adjusted to 0.7, the second weight is adjusted to 0.3, and the total reward value is determined based on the product of the first reward value and 0.7, and the product of the second reward value and 0.3, and the target retrieval decision model is trained. If the training progress is greater than 50%, the first weight is adjusted to 0.3, the second weight is adjusted to 0.7, and the total reward value is determined based on the product of the first reward value and 0.3, and the product of the second reward value and 0.7, and the target retrieval decision model is trained.

[0140] The method in this application embodiment dynamically adjusts the first weight and the second weight according to the training objective or the model training progress. By dynamically adjusting the first weight and the second weight, it can adapt to various training scenarios and achieve adaptability of the training method.

[0141] Optionally, during the process of generating dialogue responses for dialogue data samples based on reference knowledge and dialogue data samples, the training module can generate prompt words based on the reference knowledge and dialogue data samples. Based on the prompt words, the generative model then generates dialogue responses for the dialogue data samples.

[0142] Cue words are used to instruct the generative model to mark the referenced target knowledge in the dialogue response by referencing tags when generating dialogue responses based on reference knowledge and dialogue data samples.

[0143] For example, the reference knowledge and dialogue data samples obtained by the Search Agent, as well as the natural language text that instructs the generation of a dialogue response based on the reference knowledge and dialogue data samples, and requires the referenced target knowledge to be marked in the dialogue response by reference tags, are combined into prompt words. The prompt words are input into the Generator, and the Generator generates a dialogue response based on the dialogue data samples through LLM, with the referenced target knowledge marked in the dialogue response.

[0144] The method in this application guides the referencing behavior of the generation model through prompt words, so as to facilitate the subsequent training of the target retrieval decision model based on the referencing behavior.

[0145] Figure 4 Flowchart of the model training method provided for an exemplary embodiment of this application Figure 3 Please see. Figure 4 It can include:

[0146] S401. Receive dialogue data samples, generate a list of extracted keywords through the search agent, and input them into the search engine.

[0147] For example, the Search Agent bases its data on dialogue data samples. Based on its decision to generate a keyword list, a keyword list {q1, q2, ...} is generated.

[0148] It should be understood that the decisions made by the search agent in generating the keyword list can refer to the decisions adopted when constructing the SearchAgent in related technologies. For example, the decisions made by the search agent in generating the keyword list include: segmenting the dialogue data sample into words based on word segmentation methods to obtain the keyword list.

[0149] S402. Using a search engine based on a keyword list, retrieve the initial list from the retrieval knowledge base and return it to the search agent.

[0150] For example, the search engine retrieves an initial list from the retrieval knowledge base based on the keyword list {q1, q2, ...}. Return to the Search Agent. Initial list. Includes at least one document.

[0151] S403. The initial list is processed by a search agent to obtain at least one reference knowledge, which is then input into the generative model.

[0152] Optionally, the search agent processes the initial list, and the processing operation may include at least one of filtering, sorting, and truncation.

[0153] For example, the Search Agent filters some or all documents from the initial list based on its internal decision-making methods. For instance, based on the relevance of the dialogue data sample to the documents in the initial list, it filters and retains documents with higher relevance, thus obtaining reference knowledge. .

[0154] Optionally, the Search Agent can sort the documents in the initial list using its internal decision-making methods. For example, it can sort the documents from highest to lowest relevance based on the relevance of the dialogue data sample to the documents in the initial list.

[0155] Optionally, the Search Agent may truncate portions of each document in the initial list based on its internal decision-making methods to retain more relevant content.

[0156] It should be understood that the search agent filters, sorts, and / or truncates the initial list to obtain at least one reference knowledge, which can refer to the decisions adopted when constructing a search agent in related technologies, which will not be elaborated in this application.

[0157] S404. Generate dialogue responses from dialogue data samples using a generative model based on reference knowledge and dialogue data samples.

[0158] For example, the Generator, via LLM, is based on Dialogue data samples This generates a dialogue response A that serves as a sample of dialogue data. Taking the use of superscript tags to represent references as an example, dialogue response A uses superscript tags to mark the referenced target knowledge. .

[0159] S405. Parse the reference markers in the dialogue response and extract the referenced target knowledge.

[0160] For example, the training module parses the indexes in the dialogue response and extracts the referenced target knowledge. .

[0161] S406. Based on the target knowledge and the reference knowledge, obtain the total reward value.

[0162] For example, a first reward value is obtained based on target knowledge and reference knowledge. A second reward value is obtained based on dialogue data samples and dialogue responses. A total reward value R is determined based on the product of the first reward value and a first weight, and the product of the second reward value and a second weight.

[0163]

[0164] The higher the level of reference knowledge cited in the dialogue response, the higher the first reward value. The higher the value, the higher the total reward value R. The higher the quality of the dialogue response, the higher the secondary reward value. The higher the value, the higher the total reward value R.

[0165] S407. Train the target retrieval decision model based on the total reward value.

[0166] For example, the training module can update the parameters influencing decision-making in the Search Agent based on reinforcement learning methods such as PPO or Group Relative Policy Optimization (GRPO), according to the total reward value R. Reinforcement learning methods such as PPO or GRPO optimize the updated policy, for example, by adjusting the update step size. Specific implementation steps for methods based on PPO or GRPO can be found in related technologies, and will not be elaborated upon in this embodiment.

[0167] It should be noted that, in Figure 4 The various model training steps (S401-S407) shown in the embodiments can be implemented with reference to the same or similar steps in the above embodiments. Figure 4 The various model training steps shown in the embodiments do not constitute a specific limitation on the model training method. In other embodiments of this application, the model training method may include... Figure 4 The embodiments may include more or fewer steps. For example, a model training method may include... Figure 4 Some steps in the embodiments, or, Figure 4 Some steps in the embodiments can be replaced by steps with the same function, or Figure 4 Some steps in the embodiments can be broken down into multiple steps, etc.

[0168] The model training method in this embodiment of the application, based on dialogue data samples, uses a target retrieval decision model to recall at least one reference knowledge from a retrieval knowledge base; based on the reference knowledge and dialogue data samples, it generates a dialogue response for the dialogue data samples; and based on the referencing behavior of the reference knowledge during the generation of the dialogue response, it trains the target retrieval decision model. The method in this embodiment of the application trains the target retrieval decision model based on the referencing behavior of the reference knowledge during the generation of the dialogue response, thus achieving the training of the target retrieval decision model. During the training process, it relies on the referencing behavior of the reference knowledge in the dialogue response to train the target retrieval decision model. Compared with the training methods in related technologies that rely on standard answers, it does not require manual annotation, thereby improving the efficiency of model training. Since no manual annotation is required, it can continuously self-reinforce using user interaction. As the amount of interaction increases, the target retrieval decision model gradually conforms to the preferences of the generation model in generating dialogue responses. The method in this embodiment of the application implements end-to-end strategy training of the target retrieval decision model, including full-link training from extracting the keyword list and processing the initial list, achieving complete gradient backpropagation from the downstream generation end to the upstream retrieval end.

[0169] This application also provides a dialogue processing method. Figure 5 A flowchart illustrating a first dialogue processing method provided for an exemplary embodiment of this application. Please refer to... Figure 5 It can include:

[0170] S501, Obtain user's conversation data.

[0171] S502. Based on the dialogue data, a target retrieval decision model trained using a model training method is used to recall at least one reference knowledge from the retrieval knowledge base.

[0172] The target retrieval decision model can be trained using model training methods, and any of the above-mentioned model training methods can be referred to in the embodiments.

[0173] S503. Based on reference knowledge and dialogue data, generate a dialogue response from the dialogue data.

[0174] The dialogue processing method provided in this application embodiment can refer to the embodiment of the above model training method. Its implementation principle and beneficial effects are similar, and will not be repeated here.

[0175] The dialogue processing method in this application embodiment uses a target retrieval decision model trained using a model training method. Compared with target retrieval decision models trained on standard answers in related technologies, which have a poor ability to handle open-ended questions, the target retrieval decision model trained in this application embodiment can provide better reference knowledge in open-ended question-and-answer scenarios without relying on standard answers, thereby improving the quality and credibility of dialogue responses.

[0176] This application also provides a dialogue processing method. Figure 6 A flowchart illustrating a second dialogue processing method provided for an exemplary embodiment of this application. Please refer to... Figure 6 It can include:

[0177] S601, responds to input operations applied to the operation interface and displays the dialogue data entered by the user on the operation interface.

[0178] S602. In response to the dialogue operation on the operation interface, a target retrieval decision model trained by the model prediction method is used to recall at least one reference knowledge from the retrieval knowledge base, and based on the reference knowledge and dialogue data, a dialogue response is generated and displayed on the operation interface.

[0179] The training of the target retrieval decision model using model training methods can refer to any of the above-mentioned model training method implementation examples.

[0180] The dialogue processing method of this application embodiment responds to input operations on the operation interface by displaying the user-input dialogue data on the operation interface; responding to dialogue operations on the operation interface by employing a target retrieval decision model to recall at least one reference knowledge from a retrieval knowledge base; and generating a dialogue response based on the reference knowledge and dialogue data, and displaying the dialogue response on the operation interface. Since the target retrieval decision model is trained using a model training method, compared to target retrieval decision models trained on standard answers in related technologies, it suffers from poor ability to handle open-ended questions. This application embodiment, without relying on standard answers for training, allows the trained model to provide better reference knowledge in open-ended question-and-answer scenarios, thereby improving the quality and credibility of the dialogue response.

[0181] This application also provides a dialogue system, which includes a client and a server.

[0182] The client is used to receive dialogue data input by the user;

[0183] The server-side component is used to retrieve at least one reference knowledge from the retrieval knowledge base based on the target retrieval decision model trained using the model training method described above, and to generate a dialogue response based on the reference knowledge and the dialogue data.

[0184] Optionally, the server-side includes: a search agent, a search engine, and a generative model.

[0185] The search agent is used to retrieve a list of candidate knowledge from a retrieval knowledge base based on dialogue data through a search engine, and to obtain at least one reference knowledge from the candidate knowledge list by using a target retrieval decision model trained by a model training method.

[0186] Generative models are used to generate dialogue responses based on reference knowledge and dialogue data.

[0187] Figure 7 A schematic diagram of the structure of a dialogue system provided for an exemplary embodiment of this application. Please refer to... Figure 7 The client 701 receives dialogue data input by the user and sends the dialogue data to the server 702. Based on the dialogue data, the server 702 uses the target retrieval decision model trained by the above model training method to recall at least one reference knowledge from the retrieval knowledge base, and generates a dialogue response based on the reference knowledge and the dialogue data.

[0188] In this embodiment of the dialogue system, the client receives dialogue data input by the user; the server, based on the dialogue data, uses a target retrieval decision model trained using a model training method to retrieve at least one reference knowledge from the retrieval knowledge base, and generates a dialogue response based on the reference knowledge and the dialogue data. Since the target retrieval decision model is trained using a model training method, compared to target retrieval decision models trained on standard answers in related technologies, it has a poor ability to handle open-ended questions. This embodiment, without relying on standard answers, allows the trained model to provide better reference knowledge in open-ended question-and-answer scenarios, thereby improving the quality and credibility of the dialogue response output by the dialogue system.

[0189] Figure 8 A schematic diagram of the structure of a model training apparatus provided for an exemplary embodiment of this application. Please refer to... Figure 8 The model training device 800 may include: a retrieval module 801, a generation module 802, and an optimization module 803.

[0190] The retrieval module 801 is used to retrieve at least one reference knowledge from the retrieval knowledge base based on dialogue data samples and using a target retrieval decision model.

[0191] The generation module 802 is used to generate dialogue responses based on reference knowledge and dialogue data samples.

[0192] The optimization module 803 is used to train the target retrieval decision model based on the referencing behavior of reference knowledge when generating dialogue responses.

[0193] In one possible implementation, the referencing behavior is marked by a reference marker contained in the dialogue response; the reference marker is used to indicate the target knowledge referenced when generating the dialogue response; the target knowledge comes from at least one reference knowledge.

[0194] Accordingly, the optimization module 803 is specifically used for:

[0195] Parse the reference markers in the dialogue response and extract the target knowledge referenced by the reference markers;

[0196] The target retrieval decision model is trained based on target knowledge and reference knowledge.

[0197] In one possible implementation, the optimization module 803 is specifically used for:

[0198] Based on the target knowledge and the reference knowledge, a first reward value is obtained; the first reward value is used to characterize the reference signal-to-noise ratio of the reference knowledge; the reference signal-to-noise ratio is the proportion of reference knowledge referenced by the dialogue response.

[0199] The target retrieval decision model is trained based on the first reward value.

[0200] In one possible implementation, the optimization module 803 is specifically used for:

[0201] Based on the target knowledge, determine the number of reference knowledge references that are cited in the dialogue response;

[0202] The first reward value is obtained based on the number of cited reference knowledge and the number of reference knowledge.

[0203] In one possible implementation, the optimization module 803 is specifically used for:

[0204] Based on the target knowledge, identify the reference knowledge texts that are referenced in the dialogue response;

[0205] Based on the cited reference knowledge text, and the reference knowledge, obtain the first reward value.

[0206] In one possible implementation, the optimization module 803 is specifically used for:

[0207] Based on the dialogue data samples and dialogue responses, a second reward value is obtained; the second reward value is used to characterize the generation quality of the dialogue response.

[0208] Based on the first reward value, the target retrieval decision model is trained, including:

[0209] The total reward value is obtained based on the first reward value and the second reward value.

[0210] The target retrieval decision model is trained based on the total reward value.

[0211] In one possible implementation, the optimization module 803 is specifically used for:

[0212] Based on the training objective, or the training progress of the objective retrieval decision model, determine the first weight corresponding to the first reward value, and the second weight corresponding to the second reward value;

[0213] The total reward value is determined based on the product of the first reward value and the first weight, and the product of the second reward value and the second weight.

[0214] In one possible implementation, the generation module 802 is specifically used for:

[0215] Based on reference knowledge and dialogue data samples, prompt words are generated; the prompt words are used to instruct the generative model to mark the referenced target knowledge in the dialogue response by reference tags when generating dialogue responses based on reference knowledge and dialogue data samples.

[0216] Based on prompt words, a generative model is used to generate dialogue responses from dialogue data samples.

[0217] The model training device provided in this application embodiment can execute the technical solution shown in the above method embodiment. Its implementation principle and beneficial effects are similar, and will not be described again here.

[0218] Figure 9 A schematic diagram of the structure of a first dialogue processing apparatus provided for an exemplary embodiment of this application. Please refer to... Figure 9 The dialogue processing device 900 may include: an acquisition module 901, a retrieval module 902, and a generation module 903.

[0219] Module 901 is used to acquire user dialogue data;

[0220] The retrieval module 902 is used to retrieve at least one reference knowledge from the retrieval knowledge base based on the dialogue data and the target retrieval decision model trained by any of the model training methods in the above embodiments.

[0221] The generation module 903 is used to generate dialogue responses based on reference knowledge and dialogue data.

[0222] The first dialogue processing device provided in this application embodiment can execute the technical solution shown in the above method embodiment. Its implementation principle and beneficial effects are similar, and will not be repeated here.

[0223] Figure 10 A schematic diagram of the structure of a second dialogue processing apparatus provided for an exemplary embodiment of this application. Please refer to... Figure 10 The dialogue processing device 1000 may include: an acquisition module 1001 and a dialogue module 1002.

[0224] The acquisition module 1001 is used to respond to input operations on the operation interface and display the dialogue data entered by the user on the operation interface.

[0225] The dialogue module 1002 is used to respond to dialogue operations on the operation interface, and uses a target retrieval decision model trained by any of the model training methods in the above embodiments to recall at least one reference knowledge from the retrieval knowledge base, and generates a dialogue response based on the reference knowledge and dialogue data, and displays the dialogue response on the operation interface.

[0226] The second dialogue processing device provided in this application embodiment can execute the technical solution shown in the above method embodiment. Its implementation principle and beneficial effects are similar, and will not be described again here.

[0227] Figure 11 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Please refer to... Figure 11 The electronic device 1100 may include a processor 1101 and a memory 1102. Exemplarily, the processor 1101 and the memory 1102 are interconnected via a bus 1103.

[0228] Memory 1102 stores computer-executed instructions;

[0229] The processor 1101 executes computer execution instructions stored in the memory 1102, causing the processor 1101 to perform the method as shown in the above method embodiment.

[0230] Accordingly, embodiments of this application provide a computer-readable storage medium storing computer-executable instructions, which, when executed by a processor, are used to implement the methods described in the above-described method embodiments.

[0231] Accordingly, embodiments of this application may also provide a computer program product, including a computer program, which, when executed by a processor, can implement the methods shown in the above-described method embodiments.

[0232] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, systems, or computer program products. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0233] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0234] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0235] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0236] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0237] Memory may include non-persistent storage in computer-readable media, such as random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash RAM. Memory is an example of computer-readable media.

[0238] Computer-readable media includes both permanent and non-permanent, removable and non-removable media that can store information by any method or technology. Information can be computer-readable instructions, data structures, modules of programs, or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, CD-ROM, digital versatile optical disc (DVD) or other optical storage, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other non-transferable medium that can be used to store information accessible by a computing device. As defined herein, computer-readable media does not include transient computer-readable media, such as modulated data signals and carrier waves.

[0239] It should also be noted that the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such process, method, article, or apparatus. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element.

[0240] The above are merely embodiments of this application and are not intended to limit the scope of this application. Various modifications and variations can be made to this application by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the scope of the claims of this application.

Claims

1. A model training method, characterized in that, The method includes: Based on dialogue data samples, a target retrieval decision model is used to recall at least one reference knowledge from the retrieval knowledge base; Based on the reference knowledge and the dialogue data sample, a dialogue response to the dialogue data sample is generated; the dialogue response contains a reference marker; the reference marker is used to indicate the target knowledge referenced when generating the dialogue response; the target knowledge comes from the at least one reference knowledge. The target retrieval decision model is trained based on the reference markers; The step of generating a dialogue response for the dialogue data sample based on the reference knowledge and the dialogue data sample includes: generating prompt words based on the reference knowledge and the dialogue data sample; the prompt words instructing the generation model to mark the referenced target knowledge in the dialogue response by means of the reference tag when generating the dialogue response based on the reference knowledge and the dialogue data sample; generating a dialogue response for the dialogue data sample using the generation model based on the prompt words; and marking the referenced target knowledge in the dialogue response.

2. The method according to claim 1, characterized in that, The step of training the target retrieval decision model based on the reference tag includes: Parse the reference markers in the dialogue response and extract the target knowledge referenced by the reference markers; The target retrieval decision model is trained based on the target knowledge and the reference knowledge.

3. The method according to claim 2, characterized in that, The step of training the target retrieval decision model based on the target knowledge and the reference knowledge includes: Based on the target knowledge and the reference knowledge, a first reward value is obtained; the first reward value is used to characterize the reference signal-to-noise ratio of the reference knowledge; the reference signal-to-noise ratio is the proportion of reference knowledge referenced by the dialogue response in the reference knowledge. The target retrieval decision model is trained based on the first reward value.

4. The method according to claim 3, characterized in that, The step of obtaining the first reward value based on the target knowledge and the reference knowledge includes: Based on the target knowledge, determine the number of reference knowledge referenced by the dialogue response in the reference knowledge; The first reward value is obtained based on the number of reference knowledge cited and the number of reference knowledge.

5. The method according to claim 3, characterized in that, The step of obtaining the first reward value based on the target knowledge and the reference knowledge includes: Based on the target knowledge, determine the referenced knowledge texts that are referenced by the dialogue response in the reference knowledge; Based on the cited reference knowledge text, and the reference knowledge, the first reward value is obtained.

6. The method according to claim 3, characterized in that, The method further includes: Based on the dialogue data sample and the dialogue response, a second reward value is obtained; the second reward value is used to characterize the generation quality of the dialogue response. The step of training the target retrieval decision model based on the first reward value includes: Based on the first reward value and the second reward value, obtain the total reward value; The target retrieval decision model is trained based on the total reward value.

7. The method according to claim 6, characterized in that, The step of obtaining the total reward value based on the first reward value and the second reward value includes: Based on the training objective, or the training progress of the objective retrieval decision model, determine the first weight corresponding to the first reward value, and the second weight corresponding to the second reward value; The total reward value is determined based on the product of the first reward value and the first weight, and the product of the second reward value and the second weight.

8. A dialogue processing method, characterized in that, The method includes: Obtain user conversation data; Based on the dialogue data, at least one reference knowledge is retrieved from the retrieval knowledge base using the target retrieval decision model trained by the method described in any one of claims 1-7. Based on the reference knowledge and the dialogue data, a dialogue response to the dialogue data is generated.

9. A dialogue processing method, characterized in that, include: In response to input operations applied to the user interface, the user-input dialogue data is displayed on the user interface. In response to a dialogue operation on the operation interface, the system uses a target retrieval decision model trained by the method described in any one of claims 1-8 to recall at least one reference knowledge from the retrieval knowledge base, and generates a dialogue response based on the reference knowledge and the dialogue data, and displays the dialogue response on the operation interface.

10. A dialogue system, characterized in that, The dialogue system includes: a client and a server; The client is used to receive dialogue data input by the user; The server is configured to, based on the dialogue data, use a target retrieval decision model trained by the method described in any one of claims 1-8 to recall at least one reference knowledge from the retrieval knowledge base, and generate a dialogue response based on the reference knowledge and the dialogue data. The step of generating a dialogue response for the dialogue data sample based on the reference knowledge and the dialogue data sample includes: generating prompt words based on the reference knowledge and the dialogue data sample; the prompt words instructing the generation model to mark the referenced target knowledge in the dialogue response by means of the reference tag when generating the dialogue response based on the reference knowledge and the dialogue data sample; generating a dialogue response for the dialogue data sample using the generation model based on the prompt words; and marking the referenced target knowledge in the dialogue response.

11. The dialogue system according to claim 10, characterized in that, The server-side includes: a search agent, a search engine, and a generative model; The search agent is used to retrieve a list of candidate knowledge from the retrieval knowledge base based on dialogue data through the search engine, and to obtain at least one reference knowledge from the candidate knowledge list using the target retrieval decision model; the dialogue response contains a reference marker; the reference marker is used to indicate the target knowledge referenced when generating the dialogue response; the target knowledge comes from the at least one reference knowledge; The generative model is used to generate a dialogue response based on the reference markers.

12. An electronic device, characterized in that, include: At least one processor; and a memory communicatively connected to the at least one processor; The memory stores instructions executable by the at least one processor, which, when executed by the at least one processor, cause the electronic device to perform the method according to any one of claims 1-9.

13. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer-executable instructions, which, when executed by a processor, implement the method as described in any one of claims 1-9.

14. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the method as described in any one of claims 1-9.

Images