Dialogue generation method and device for improving content correctness
By constructing knowledge representation and counterfactual scenario contexts, and using a word-by-word subtraction decoding model to improve dialogue generation, the problem of generating erroneous information in knowledge representation dialogue systems is solved, and high-quality dialogue generation is achieved.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- BEIJING KNOWLEDGE ATLAS TECHNOLOGY CO LTD
- Filing Date
- 2022-12-14
- Publication Date
- 2026-07-10
Smart Images
Figure CN116127030B_ABST
Abstract
Description
Technical Field
[0001] This application relates to the field of natural language processing technology, and in particular to a dialogue generation method and apparatus for improving the accuracy of content. Background Technology
[0002] Knowledge-based dialogue systems aim to increase the accuracy and information richness of generated dialogues by appropriately incorporating knowledge-based content. However, even with state-of-the-art large-scale language models (including general language models and pre-trained dialogue models), dialogue systems still occasionally generate erroneous information that contradicts the facts. These unpredictable errors severely damage the credibility and sincerity of the dialogue system, leading to a significant decline in dialogue quality. This phenomenon of generating dialogue content that contradicts the facts is called the "illusion" of the dialogue system, also known as knowledge bias or fact bias.
[0003] Existing Knowledge Visualization (KGD) dialogue systems mainly consist of two parts: a retrieval system that searches for external knowledge; and a generator based on large models or other generative models. To increase the credibility and accuracy of the content generated by the dialogue system, existing methods primarily focus on two aspects:
[0004] One approach is to improve the retrieval engine. This approach mainly involves training a stronger network retrieval engine using a dialogue dataset with external knowledge enhancement, thereby controlling the quality of the retrieved knowledge and improving the factual accuracy of the generated results.
[0005] Secondly, an additional fact verification module is added to retrieve the knowledge again after generation and guide the generation module to regenerate the dialogue.
[0006] Existing methods all assume that external knowledge is perfect and strive to optimize the generation process by retrieving and using high-quality external knowledge resources. However, such methods encounter many problems in practical deployment. Existing external knowledge bases are still very incomplete, so even with the most powerful search engines, there is a possibility that high-quality external resources cannot be retrieved. Secondly, even in correctly matched knowledge resources, there may be extra redundant content or even noise, which can still lead to errors in the generated content. Summary of the Invention
[0007] This application aims to at least partially address one of the technical problems in the related art.
[0008] Therefore, the first objective of this application is to propose a dialogue generation method that improves the accuracy of content, thereby solving the technical problem that existing dialogue systems will generate erroneous information that conflicts with the facts from time to time, which greatly undermines the credibility and sincerity of the dialogue system, and effectively improves the accuracy of dialogue generation content.
[0009] The second objective of this application is to propose a dialogue generation device that improves the accuracy of content.
[0010] To achieve the above objectives, the first aspect of this application proposes a dialogue generation method to improve the accuracy of content, comprising: acquiring dialogue history and retrieving relevant knowledge content based on the dialogue history; constructing a knowledge visualization context and a counterfactual scenario context based on the dialogue history and relevant knowledge content; inputting the knowledge visualization context and the counterfactual scenario context into a word-by-word subtraction decoding model, and outputting the dialogue result.
[0011] The dialogue generation method for improving content accuracy in this application improves the decoding process of the generation model, thereby maintaining a high tolerance for noisy external knowledge. Even with incomplete knowledge, it can still achieve the function of generating high-quality dialogue. At the same time, this application does not require additional training steps, so it can be more widely deployed in language models with different structures.
[0012] Optionally, in one embodiment of this application, a knowledge visualization context and a counterfactual scenario context are constructed based on the dialogue history and related knowledge content, including:
[0013] By combining historical dialogues with relevant knowledge content, a concrete context for knowledge is obtained;
[0014] Design counterfactual scenarios by retaining dialogue content related to the retrieved knowledge content, deleting or replacing the remaining dialogue content, and combining them into a counterfactual scenario context.
[0015] Optionally, in one embodiment of this application, a decay function is used to prevent the decoding model from collapsing due to excessive decay during dialogue generation, wherein the decay function is expressed as:
[0016] λ(i)=α i-1
[0017] Where α represents the attenuation coefficient.
[0018] Optionally, in one embodiment of this application, the knowledge representation context and the counterfactual scenario context are input into a word-by-word subtraction decoding model, represented as:
[0019] g i =C d,k ||w[1:i]
[0020]
[0021]
[0022] Among them, C d,k , These represent the context of knowledge visualization and the context of counterfactual scenarios, respectively. w represents each word in the sentence and the entire searchable vocabulary.
[0023] To achieve the above objectives, a second aspect of the present invention provides a method comprising an acquisition module, a construction module, and a generation module, wherein,
[0024] The acquisition module is used to acquire the dialogue history and retrieve relevant knowledge content based on the dialogue history;
[0025] The building module is used to construct knowledge visualization contexts and counterfactual scenario contexts based on dialogue history and related knowledge content;
[0026] The generation module is used to input the knowledge visualization context and the counterfactual scenario context into the decoding model, which subtracts word by word, and outputs the dialogue result.
[0027] Optionally, in one embodiment of this application, the construction module is specifically used for:
[0028] By combining historical dialogues with relevant knowledge content, a concrete context for knowledge is obtained;
[0029] Design counterfactual scenarios by retaining dialogue content related to the retrieved knowledge content, deleting or replacing the remaining dialogue content, and combining them into a counterfactual scenario context.
[0030] Optionally, in one embodiment of this application, a decay function is used to prevent the decoding model from collapsing due to excessive decay during dialogue generation, wherein the decay function is expressed as:
[0031] λ(i)=α i-1
[0032] Where α represents, and i represents.
[0033] Optionally, in one embodiment of this application, the knowledge representation context and the counterfactual scenario context are input into a word-by-word subtraction decoding model, represented as:
[0034] g i =C d,k ||w[1:i]
[0035]
[0036]
[0037] Among them, C d,k , These represent the context of knowledge visualization and the context of counterfactual scenarios, respectively. w represents each word in the sentence and the entire searchable vocabulary.
[0038] Additional aspects and advantages of this application will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of this application. Attached Figure Description
[0039] The above and / or additional aspects and advantages of this application will become apparent and readily understood from the following description of the embodiments taken in conjunction with the accompanying drawings, wherein:
[0040] Figure 1 A flowchart illustrating a dialogue generation method for improving content accuracy provided in Embodiment 1 of this application;
[0041] Figure 2 This is a flowchart of the dialogue system according to an embodiment of this application;
[0042] Figure 3 This is a schematic diagram of the structure of a dialogue generation device for improving content accuracy, provided in an embodiment of this application. Detailed Implementation
[0043] The embodiments of this application are described in detail below. Examples of these embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary and intended to explain this application, and should not be construed as limiting this application.
[0044] The following description, with reference to the accompanying drawings, describes a dialog generation method and apparatus for improving content correctness according to embodiments of this application.
[0045] Figure 1 This is a flowchart illustrating a dialogue generation method for improving content accuracy, as provided in Embodiment 1 of this application.
[0046] like Figure 1 As shown, the dialogue generation method for improving content accuracy includes the following steps:
[0047] Step 101: Obtain the dialogue history and retrieve relevant knowledge content based on the dialogue history;
[0048] Step 102: Construct knowledge visualization context and counterfactual scenario context based on dialogue history and relevant knowledge content;
[0049] Step 103: Input the knowledge visualization context and the counterfactual scenario context into the word-by-word subtraction decoding model, and output the dialogue result.
[0050] The dialogue generation method for improving content accuracy in this application improves the decoding process of the generation model, thereby maintaining a high tolerance for noisy external knowledge. Even with incomplete knowledge, it can still achieve the function of generating high-quality dialogue. At the same time, this application does not require additional training steps, so it can be more widely deployed in language models with different structures.
[0051] Optionally, in one embodiment of this application, a knowledge visualization context and a counterfactual scenario context are constructed based on the dialogue history and related knowledge content, including:
[0052] By combining historical dialogues with relevant knowledge content, a concrete context for knowledge is obtained;
[0053] Design counterfactual scenarios by retaining dialogue content related to the retrieved knowledge content, deleting or replacing the remaining dialogue content, and combining them into a counterfactual scenario context.
[0054] Optionally, in one embodiment of this application, a decay function is used to prevent the decoding model from collapsing due to excessive decay during dialogue generation, wherein the decay function is expressed as:
[0055] λ(i)=α i-1
[0056] Where α represents the attenuation coefficient.
[0057] Optionally, in one embodiment of this application, the knowledge representation context and the counterfactual scenario context are input into a word-by-word subtraction decoding model, represented as:
[0058] g i =C d,k ||w[1:i]
[0059]
[0060]
[0061] Among them, C d,k , These represent the context of knowledge visualization and the context of counterfactual scenarios, respectively. w represents each word in the sentence and the entire searchable vocabulary.
[0062] Figure 2 This is a flowchart of the dialogue system according to an embodiment of this application. The following is in conjunction with... Figure 2 This application introduces the dialogue system.
[0063] To provide an accurate overview of the entire dialogue system, we will first perform a basic mathematical formalization:
[0064] Dialogue history D = {u1,s1,…,u t-1 ,s t-1 ,u t}, where u i ,s i These represent the dialogue content between the user and the system in each round, while the external knowledge K consists of multiple knowledge fragments. The ultimate goal of the entire dialogue system is to generate a reasonable dialogue result R.
[0065] Next, we will use an example to describe the process of its technical execution.
[0066] During the input phase, assume the historical dialogue is as follows:
[0067] A: I really like Renaissance artists, like Leonardo da Vinci and Raphael. B: Me too! Renaissance artists were all so incredibly talented, and they created countless works that have survived to this day. A: By the way, whose work is that famous Sistine Madonna?
[0068] Retrieve relevant knowledge content and obtain the most relevant knowledge content K.
[0069] "Raphael's masterpieces include 'Saint George and the Dragon' and 'The Sistine Madonna.' He studied Michelangelo's paintings in the Sistine Chapel and eventually developed his own style."
[0070] By combining historical dialogues with relevant knowledge content, we obtain context C. A typical approach is to transform the knowledge into a question-and-answer format.
[0071] Design a counterfactual scenario where only dialogue content related to the retrieved knowledge K is retained, while most of the dialogue content is deleted or replaced, and combined into another set of context C′.
[0072] Together, they complete the decoding and generation process, and subtract the logit from each token. The main formula is as follows:
[0073] g i =C d,k ||w[1:i]
[0074]
[0075]
[0076] Among them, C d,k , These represent the context of knowledge visualization and the context of counterfactual scenarios, respectively. w represents each word in the sentence and the entire searchable vocabulary.
[0077] It is worth noting that the above model includes a decay function to prevent model collapse caused by excessive decay during generation. (A reference value is provided here: α = 0.3 can be set as an initial trial value.)
[0078] Ultimately, the reply was, "This is Raphael's masterpiece! His style was influenced by Leonardo da Vinci, Michelangelo, and others, and eventually formed his own unique artistic characteristics."
[0079] To achieve the above embodiments, this application also proposes a dialogue generation apparatus to improve the accuracy of content.
[0080] Figure 3 This is a schematic diagram of the structure of a dialogue generation device for improving content accuracy, provided in an embodiment of this application.
[0081] like Figure 3 As shown, the dialogue generation device for improving content accuracy includes an acquisition module, a construction module, and a generation module, wherein...
[0082] The acquisition module is used to acquire the dialogue history and retrieve relevant knowledge content based on the dialogue history;
[0083] The building module is used to construct knowledge visualization contexts and counterfactual scenario contexts based on dialogue history and related knowledge content;
[0084] The generation module is used to input the knowledge visualization context and the counterfactual scenario context into the decoding model, which subtracts word by word, and outputs the dialogue result.
[0085] Optionally, in one embodiment of this application, the construction module is specifically used for:
[0086] By combining historical dialogues with relevant knowledge content, a concrete context for knowledge is obtained;
[0087] Design counterfactual scenarios by retaining dialogue content related to the retrieved knowledge content, deleting or replacing the remaining dialogue content, and combining them into a counterfactual scenario context.
[0088] Optionally, in one embodiment of this application, a decay function is used to prevent the decoding model from collapsing due to excessive decay during dialogue generation, wherein the decay function is expressed as:
[0089] λ(i)=α i-1
[0090] Where α represents the attenuation coefficient.
[0091] Optionally, in one embodiment of this application, the knowledge representation context and the counterfactual scenario context are input into a word-by-word subtraction decoding model, represented as:
[0092] g i =C d,k ||w[1:i]
[0093]
[0094]
[0095] Among them, C d,k , These represent the context of knowledge visualization and the context of counterfactual scenarios, respectively. w represents each word in the sentence and the entire searchable vocabulary.
[0096] It should be noted that the foregoing explanation of the dialogue generation method embodiment for improving content accuracy also applies to the dialogue generation device for improving content accuracy in this embodiment, and will not be repeated here.
[0097] In the description of this specification, the references to terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of this application. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples. Moreover, without contradiction, those skilled in the art can combine and integrate the different embodiments or examples described in this specification, as well as the features of different embodiments or examples.
[0098] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one of that feature. In the description of this application, "multiple" means at least two, such as two, three, etc., unless otherwise explicitly specified.
[0099] Any process or method description in the flowchart or otherwise herein can be understood as representing a module, segment, or portion of code comprising one or more executable instructions for implementing custom logic functions or processes, and the scope of the preferred embodiments of this application includes additional implementations in which functions may be performed not in the order shown or discussed, including substantially simultaneously or in reverse order depending on the functions involved, as should be understood by those skilled in the art to which embodiments of this application pertain.
[0100] The logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered as a sequenced list of executable instructions for implementing logical functions, and can be embodied in any computer-readable medium for use by, or in conjunction with, an instruction execution system, apparatus, or device (such as a computer-based system, a processor-included system, or other system that can fetch and execute instructions from, an instruction execution system, apparatus, or device). For the purposes of this specification, "computer-readable medium" can be any means that can contain, store, communicate, propagate, or transmit programs for use by, or in conjunction with, an instruction execution system, apparatus, or device. More specific examples (a non-exhaustive list) of computer-readable media include: an electrical connection having one or more wires (electronic device), a portable computer disk drive (magnetic device), random access memory (RAM), read-only memory (ROM), erasable and editable read-only memory (EPROM or flash memory), fiber optic devices, and portable optical disc read-only memory (CDROM). Alternatively, the computer-readable medium may be paper or other suitable media on which the program can be printed, since the program can be obtained electronically, for example, by optically scanning the paper or other medium, followed by editing, interpreting, or otherwise processing as necessary, and then stored in a computer memory.
[0101] It should be understood that various parts of this application can be implemented using hardware, software, firmware, or a combination thereof. In the above embodiments, multiple steps or methods can be implemented using software or firmware stored in memory and executed by a suitable instruction execution system. For example, if implemented in hardware as in another embodiment, it can be implemented using any one or a combination of the following techniques known in the art: discrete logic circuits having logic gates for implementing logical functions on data signals, application-specific integrated circuits (ASICs) having suitable combinational logic gates, programmable gate arrays (PGAs), field-programmable gate arrays (FPGAs), etc.
[0102] Those skilled in the art will understand that all or part of the steps of the methods in the above embodiments can be implemented by a program instructing related hardware. The program can be stored in a computer-readable storage medium, and when executed, the program includes one or a combination of the steps of the method embodiments.
[0103] Furthermore, the functional units in the various embodiments of this application can be integrated into a processing module, or each unit can exist physically separately, or two or more units can be integrated into a module. The integrated module can be implemented in hardware or as a software functional module. If the integrated module is implemented as a software functional module and sold or used as an independent product, it can also be stored in a computer-readable storage medium.
[0104] The storage medium mentioned above can be a read-only memory, a disk, or an optical disk, etc. Although embodiments of this application have been shown and described above, it is understood that the above embodiments are exemplary and should not be construed as limiting this application. Those skilled in the art can make changes, modifications, substitutions, and variations to the above embodiments within the scope of this application.
Claims
1. A dialogue generation method to improve content accuracy, characterized in that, Includes the following steps: Obtain the dialogue history and retrieve relevant knowledge content based on the dialogue history; Construct a knowledge visualization context and a counterfactual scenario context based on the dialogue history and the relevant knowledge content; The knowledge visualization context and counterfactual scenario context are input into a word-by-word subtraction decoding model to output the dialogue result; Specifically, a decay function is used to prevent the decoding model from collapsing due to excessive decay during dialogue generation. The decay function is expressed as follows: in, Indicates the attenuation coefficient; In the decoding model that inputs the knowledge visualization context and the counterfactual scenario context into a word-by-word subtraction, it is represented as: in, , These respectively represent the knowledge visualization context and the counterfactual scenario context. V represents each word in the sentence, and V represents the entire searchable word list.
2. The method as described in claim 1, characterized in that, The construction of knowledge visualization context and counterfactual scenario context based on the dialogue history and related knowledge content includes: By combining the historical dialogues with the relevant knowledge content, a knowledge-concretizing context is obtained; Design counterfactual scenarios by retaining dialogue content associated with the retrieved knowledge content, deleting or replacing the remaining dialogue content, and combining them into the counterfactual scenario context.
3. A dialogue generation device for improving content accuracy, characterized in that, This includes modules for obtaining information, building components, and generating data. The acquisition module is used to acquire the dialogue history and retrieve relevant knowledge content based on the dialogue history; The construction module is used to construct a knowledge visualization context and a counterfactual scenario context based on the dialogue history and the relevant knowledge content. The generation module is used to input the knowledge visualization context and the counterfactual scenario context into a word-by-word subtraction decoding model and output the dialogue result. Specifically, a decay function is used to prevent the decoding model from collapsing due to excessive decay during dialogue generation. The decay function is expressed as follows: in, Indicates the attenuation coefficient; In the decoding model that inputs the knowledge visualization context and the counterfactual scenario context into a word-by-word subtraction, it is represented as: in, , These respectively represent the knowledge visualization context and the counterfactual scenario context. V represents each word in the sentence, and V represents the entire searchable word list.
4. The apparatus as described in claim 3, characterized in that, The building module is specifically used for: By combining the historical dialogues with the relevant knowledge content, a knowledge-concretizing context is obtained; Design counterfactual scenarios by retaining dialogue content associated with the retrieved knowledge content, deleting or replacing the remaining dialogue content, and combining them into the counterfactual scenario context.