Text dialogue generation model training method, text dialogue generation method and device

Abstract

The present invention discloses a text dialogue generation model training method, a text dialogue generation method and a device, comprising: obtaining a preset amount of basic knowledge by searching a knowledge base of question-answer pairs and document types; constructing a first prompt word and a second prompt word based on the basic knowledge, current context and current question; constructing a main model and an enhanced model, respectively using the first prompt word to perform task training on the main model, and using the second prompt word to perform task training on the enhanced model, and fusing the obtained models to obtain an initial fusion model; and training the initial fusion model to obtain a target fusion model. The present invention improves the accuracy of text dialogue generation.

Description

Technical Field

[0001] The present invention relates to the technical field of natural language processing, and in particular to a text dialogue generation model training method, a text dialogue generation method and a device. Background Art

[0002] With the development of artificial intelligence technology, intelligent answering has been widely used. Intelligent answering needs to search based on the question and return the appropriate answer. In order to improve the efficiency and accuracy of the answer, it is often necessary to retrieve relevant information from a large document collection, and then use the retrieved information to guide the generation of text, thereby improving the quality and accuracy of the prediction.

[0003] In some existing implementations, the top N (topN) text knowledge is recalled from the knowledge base through the Embedding model, and a prompt word is built based on the topN knowledge and the current query queue to input into a large language model for retrieval-based generative question answering.

[0004] In the process of implementing the present application, the inventors found that the existing methods have at least the following problems: First, the coverage and accuracy of the knowledge recalled by the vector model, that is, if more knowledge is recalled, it may contain noise, and if less knowledge is recalled, the accuracy may be insufficient; second, the model capabilities of different large models may be different, especially the capabilities of large models in different fields / different tasks are different. These problems lead to the problem of low accuracy and efficiency of existing text dialogue generation. Summary of the invention

[0005] The embodiments of the present invention provide a text dialogue generation model training method, a text dialogue generation method, an apparatus, a computer device and a storage medium to improve the accuracy and efficiency of text dialogue generation.

[0006] In order to solve the above technical problems, the present application embodiment provides a text dialogue generation model training method, including:

[0007] In order to solve the above technical problems, the present application embodiment provides a text dialogue generation method, including:

[0008] Obtain the target corpus, which is a question-answer pair type or a document type;

[0009] The target corpus is input into the target fusion model for task training to obtain the target text dialogue.

[0010] In order to solve the above technical problems, the embodiment of the present application also provides a text dialogue generation model training device, including:

[0011] The knowledge retrieval module is used to search the knowledge base of question-answer pairs and document types to obtain a preset amount of basic knowledge;

[0012] A prompt word construction module, used to construct a first prompt word and a second prompt word based on the basic knowledge, the current context and the current question, wherein the first prompt word is selected based on the basic knowledge and is used to obtain corresponding knowledge from the basic knowledge, and the second prompt word is used to generate a reply keyword, and the reply keyword is used to generate a response answer according to the current context and the current question;

[0013] A model fusion module, used to construct a main model and an enhanced model, respectively use the first prompt word to perform task training on the main model, use the second prompt word to perform task training on the enhanced model, and fuse the obtained models to obtain an initial fusion model;

[0014] The model training module is used to train the initial fusion model to obtain a target fusion model.

[0015] Optionally, the knowledge retrieval module includes:

[0016] A retrieval unit is used to search the knowledge base of question-answer pairs and document types to obtain M pieces of knowledge, where M is a positive integer;

[0017] The first encoding unit is used to encode each knowledge in the knowledge base into a vector to obtain a knowledge base vector , , where the knowledge base is , For the first to knowledge, is the encoding vector corresponding to the i-th knowledge, is the vector dimension;

[0018] The second encoding unit is used to encode the current question to obtain a question vector ;

[0019] A similarity calculation unit, used to calculate the cosine similarity between the knowledge base vector and the question vector ;

[0020] A screening unit, used to select the knowledge in the knowledge base vector whose cosine similarity value with the question vector is greater than a preset threshold as knowledge to be screened;

[0021] The selection unit is used to sort the knowledge to be screened from large to small based on the cosine similarity value to obtain a knowledge sorting sequence, and select a preset number of knowledge from the front to the back as the basic knowledge.

[0022] Optionally, the first encoding unit includes:

[0023] A first encoding subunit, configured to encode the question in the question-answer pair if the knowledge is in the form of a question-answer pair, to obtain an encoding vector corresponding to the knowledge;

[0024] The second encoding subunit is used to encode the document to obtain an encoding vector if the knowledge is in the form of a document.

[0025] Optionally, the model fusion module includes:

[0026] A vectorization unit, used to input the first prompt word into the main model to obtain a first encoding vector , and input the second prompt word into the enhanced model to obtain a second encoding vector ,in, The first prompt word passes through the main model Tokens vector after Token segmentation and word vectorization, Indicates that the second prompt word is enhanced by the model Tokens vector after Token segmentation and word vectorization, The number of tokens representing the first prompt word, Represents the number of tokens of the second prompt word, represents the vector dimension;

[0027] A two-dimensional vector generating unit is used to feed-forward the first encoding vector through the j-layer of the main model to obtain a first two-dimensional vector , and the second encoding vector is fed forward through the i-th layer of the enhancement model to obtain a second two-dimensional vector , where the total number of layers of the main model and the enhanced model are , , the number of fusion layers set is , j = - ,i= - ;

[0028] Fusion unit for Starting from the layer, the main model and the enhanced model are fused based on the first two-dimensional vector and the second two-dimensional vector to obtain an initial fused model.

[0029] In order to solve the above technical problems, the embodiment of the present application further provides a text dialogue generating device, including:

[0030] A corpus acquisition module is used to acquire target corpus, where the target corpus is a question-answer pair type or a document type;

[0031] The dialogue generation module is used to input the target corpus into the target fusion model for task training to obtain the target text dialogue.

[0032] In order to solve the above technical problems, an embodiment of the present application also provides a computer device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of the above text dialogue generation model training method when executing the computer program.

[0033] In order to solve the above technical problems, an embodiment of the present application also provides a computer-readable storage medium, which stores a computer program. When the computer program is executed by a processor, the steps of the above-mentioned text dialogue generation model training method are implemented.

[0034] The text dialogue generation model training method, text dialogue generation method, device, computer equipment and storage medium provided in the embodiment of the present invention retrieve the knowledge base of question-answer pairs and document types to obtain a preset amount of basic knowledge; based on the basic knowledge, current context and current question, construct a first prompt word and a second prompt word, the first prompt word is selected based on the basic knowledge and is used to obtain corresponding knowledge from the basic knowledge, the second prompt word is used to generate a reply keyword, and the reply keyword is used to generate a response answer according to the current context and the current question; construct a main model and an enhanced model, respectively use the first prompt word to perform task training on the main model, use the second prompt word to perform task training on the enhanced model, and fuse the obtained models to obtain an initial fusion model; train the initial fusion model to obtain a target fusion model. An end-to-end knowledge selection and response generation scheme is implemented, the two model tasks are combined for end-to-end training, and the ability fusion of different models is considered at the same time, a set of text dialogue generation strategies based on model fusion is constructed, and the accuracy of text dialogue generation is improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0035] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings required for use in the description of the embodiments of the present invention will be briefly introduced below. Obviously, the accompanying drawings in the following description are only some embodiments of the present invention. For ordinary technicians in this field, other accompanying drawings can be obtained based on these accompanying drawings without paying creative labor.

[0036] Figure 1 is an exemplary system architecture diagram to which the present application may be applied;

[0037] Figure 2 It is a flowchart of an embodiment of the text dialogue generation model training method of the present application;

[0038] Figure 3is a flowchart of an embodiment of a text dialogue generation method of the present application;

[0039] Figure 4 It is a structural schematic diagram of an embodiment of a text dialogue generation model training device according to the present application;

[0040] Figure 5 is a structural schematic diagram of an embodiment of a text dialogue generating device according to the present application;

[0041] Figure 6 It is a structural diagram of an embodiment of a computer device according to the present application. DETAILED DESCRIPTION

[0042] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those commonly understood by technicians in the technical field of the present application; the terms used in the specification of the application herein are only for the purpose of describing specific embodiments and are not intended to limit the present application; the terms "including" and "having" and any variations thereof in the specification and claims of the present application and the above-mentioned drawings are intended to cover non-exclusive inclusions. The terms "first", "second", etc. in the specification and claims of the present application or the above-mentioned drawings are used to distinguish different objects, not to describe a specific order.

[0043] Reference to "embodiments" herein means that a particular feature, structure, or characteristic described in conjunction with the embodiments may be included in at least one embodiment of the present application. The appearance of the phrase in various locations in the specification does not necessarily refer to the same embodiment, nor is it an independent or alternative embodiment that is mutually exclusive with other embodiments. It is explicitly and implicitly understood by those skilled in the art that the embodiments described herein may be combined with other embodiments.

[0044] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0045] See also Figure 1 ,like Figure 1 As shown, the system architecture 100 may include terminal devices 101, 102, 103, a network 104 and a server 105. The network 104 is used to provide a medium for communication links between the terminal devices 101, 102, 103 and the server 105. The network 104 may include various connection types, such as wired, wireless communication links or optical fiber cables, etc.

[0046] Users can use terminal devices 101 , 102 , 103 to interact with server 105 via network 104 to receive or send messages, etc.

[0047] Terminal devices 101, 102, 103 can be various electronic devices with display screens and supporting web browsing, including but not limited to smart phones, tablet computers, e-book readers, MP3 players (Moving Picture Experts Group Audio Layer III), MP4 (Moving Picture Experts Group Audio Layer IV), laptop computers, desktop computers, etc.

[0048] The server 105 may be a server that provides various services, such as a background server that provides support for web pages displayed on the terminal devices 101 , 102 , and 103 .

[0049] It should be noted that the text dialogue generation model training method and the text dialogue generation method provided in the embodiments of the present application are executed by a server, and accordingly, the text dialogue generation model training device and the text dialogue generation device are arranged in the server.

[0050] It should be understood that Figure 1 The number of terminal devices, networks and servers in the embodiment is only for illustration. According to the implementation requirements, there may be any number of terminal devices, networks and servers. The terminal devices 101, 102, 103 in the embodiment of the present application may specifically correspond to application systems in actual production.

[0051] See also Figure 2 , Figure 2 A text dialogue generation model training method provided by an embodiment of the present invention is shown, and the method is applied in Figure 1 The server in is used as an example to illustrate the details as follows:

[0052] S201: Search the knowledge base of question-answer pairs and document types to obtain a preset amount of basic knowledge.

[0053] In a specific optional implementation, searching the knowledge base of question-answer pairs and document types to obtain a preset amount of basic knowledge includes:

[0054] Search the knowledge base of question-answer pairs and document types to obtain M pieces of knowledge, where M is a positive integer;

[0055] Encode each knowledge in the knowledge base into a vector and get the knowledge base vector , , where the knowledge base is , For the first to knowledge, is the encoding vector corresponding to the i-th knowledge, is the vector dimension;

[0056] Encode the current question to get the question vector ;

[0057] Calculate the cosine similarity between the knowledge base vector and the question vector ;

[0058] The knowledge in the knowledge base vector whose cosine similarity value with the question vector is greater than a preset threshold is used as the knowledge to be screened;

[0059] The knowledge to be screened is sorted from large to small based on the cosine similarity value to obtain a knowledge sorting sequence, and a preset number of knowledge is selected from the front to the back as the basic knowledge.

[0060] In a specific optional implementation, each piece of knowledge in the knowledge base is encoded into a vector to obtain a knowledge base vector include:

[0061] If the knowledge is in the form of a question-answer pair, the question in the question-answer pair is encoded to obtain the encoding vector corresponding to the knowledge;

[0062] If the knowledge is in the form of a document, the document is encoded to obtain an encoding vector.

[0063] S202: Based on the basic knowledge, the current context and the current question, a first prompt word and a second prompt word are constructed. The first prompt word is selected based on the basic knowledge and is used to obtain corresponding knowledge from the basic knowledge. The second prompt word is used to generate a reply keyword. The reply keyword is used to generate a response answer according to the current context and the current question.

[0064] S203: constructing a main model and an enhanced model, respectively using the first prompt word to perform task training on the main model, and using the second prompt word to perform task training on the enhanced model, and fusing the obtained models to obtain an initial fusion model.

[0065] In a specific optional implementation, a main model and an enhanced model are constructed, the first prompt word is used to perform task training on the main model, and the second prompt word is used to perform task training on the enhanced model, and the obtained models are fused to obtain an initial fusion model including:

[0066] Input the first prompt word into the main model to obtain the first encoding vector , and input the second prompt word into the enhanced model to obtain the second encoding vector ,in, The first prompt word passes through the main model Tokens vector after Token segmentation and word vectorization, Indicates that the second prompt word has been enhanced by the model Tokens vector after Token segmentation and word vectorization, Represents the number of Tokens of the first prompt word, Represents the number of tokens of the second prompt word, represents the vector dimension;

[0067] The first encoded vector is fed forward through the jth layer of the main model to obtain the first two-dimensional vector The second encoding vector is fed forward through the enhanced model i layer to obtain the second two-dimensional vector , where the total number of layers of the main model and the enhanced model are , , the number of fusion layers set is , j = - ,i= - ;

[0068] from Starting from the first two-dimensional vector and the second two-dimensional vector, the main model and the enhanced model are fused to obtain an initial fused model.

[0069] In a specific optional implementation, from Starting from the first layer, the main model and the enhanced model are fused based on the first two-dimensional vector and the second two-dimensional vector to obtain the initial fusion model including:

[0070] for Each layer in the layer , construct the transformation matrix , transform the second two-dimensional vector into the same latitude as the first two-dimensional vector, and obtain the second transformed vector ;

[0071] The second transformed vector and the first two-dimensional vector are fused using a multi-head attention mechanism to obtain a fused vector , the fusion vector It is superimposed with the first two-dimensional vector as the input vector of the next layer, and the next layer is fused until the last layer is fused to obtain the initial fusion model.

[0072] Specifically, in the model fusion stage, two large models need to be built, and both models need to be chat models that have undergone SFT. Unlike the traditional RAG, the RAG solution for model fusion needs to be trained to combine the two models. The specific steps are as follows:

[0073] First, the two models are divided into main models and enhanced models , that is, using the enhanced model To improve the main model The effect, enter To do the reply generation task, enter To do knowledge selection tasks, such as Figure 2 As shown:

[0074] First, the second prompt word and the first cue word Enter your respective model , Get their respective token encoding vectors and ,in represent go through Tokens vector after model Token segmentation and word vectorization, express go through Tokens vector after model Token segmentation and word vectorization, represent The number of tokens, represent The number of tokens, Represents the vector dimension, where the vector dimension Not necessarily equal;

[0075] , Each passed , The model performs feed-forward calculations, where After j layers of calculation, we get a two-dimensional vector , After the i-th layer calculation, we get ; As for the calculation method of i, j, it is assumed that , The total number of layers is , , the number of fusion layers set is , then j= - ,i= - ; That is, the two models are at the end To merge.

[0076] at the end Layer by layer , , and the specific method is as follows: Each layer , construct a transformation matrix ,Will Convert to The same dimensions, ; then , Proceed as Figure 2 The attention calculation shown is obtained

[0077] ;

[0078] in, , (the above operation is a multi-head attention operation, and multi-head representation is not used here for simplicity), where are all trainable mapping matrices, Already included in The semantic information in Since it is a large model of SFT and its knowledge selection is related to prompt, More attention scores will be assigned to knowledge tokens related to the current query, thus The acquired semantic information contains the information for knowledge screening, which achieves end-to-end knowledge screening and model fusion. Through The vectors are added to get the input vector of the next layer, that is ; Then the model is fused layer by layer until the last layer.

[0079] S204: Train the initial fusion model to obtain a target fusion model.

[0080] Specifically, since there are new trainable parameters in the model fusion solution, it is necessary to train according to the training data. is frozen, and Freeze or partially train according to the specific situation, and add a new matrix These are all parameters that need to be trained.

[0081] At the same time, the two models in this scheme can be trained and optimized separately, that is, data training suitable for knowledge selection , and then use the model to reply to class data training , and then combine the two types of data to train scheme; through the optimization of each model, the effect of the RAG dialogue scheme based on model fusion is comprehensively improved.

[0082] In this embodiment, a preset amount of basic knowledge is obtained by searching the knowledge base of question-answer pairs and document types; based on the basic knowledge, current context and current question, a first prompt word and a second prompt word are constructed, the first prompt word is selected based on the basic knowledge and is used to obtain corresponding knowledge from the basic knowledge, and the second prompt word is used to generate reply keywords, and the reply keywords are used to generate answer answers according to the current context and current question; a main model and an enhanced model are constructed, and the first prompt word is used to perform task training on the main model and the second prompt word is used to perform task training on the enhanced model, and the obtained models are fused to obtain an initial fusion model; the initial fusion model is trained to obtain a target fusion model. An end-to-end knowledge selection and reply generation scheme is implemented, and the two model tasks are combined for end-to-end training. At the same time, the capability fusion of different models is considered, and a set of text dialogue generation strategies based on model fusion is constructed to improve the accuracy of text dialogue generation.

[0083] See also Figure 3 , Figure 3 A text dialogue generation method provided by an embodiment of the present invention is shown, and the method is applied in Figure 1 The server in is used as an example to illustrate the details as follows:

[0084] S205: Obtain target corpus, which is a question-answer pair type or a document type;

[0085] S206: Input the target corpus into the target fusion model for task training to obtain the target text dialogue.

[0086] In this embodiment, a target corpus is obtained, which is a question-answer pair type or a document type; the target corpus is input into a target fusion model for task training to obtain a target text dialogue. The target fusion model trained in advance is used to quickly generate the target text dialogue, thereby improving the efficiency of generating the target text dialogue.

[0087] It should be understood that the order of execution of the steps in the above embodiment does not necessarily mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiment of the present invention.

[0088] Figure 4 The principle block diagram of the text dialogue generation model training device corresponding to the text dialogue generation model training method in the above embodiment is shown. Figure 4 As shown, the text dialogue generation model training device includes a knowledge retrieval module 31, a prompt word construction module 32, a model fusion module 33 and a model training module 34. The detailed description of each functional module is as follows:

[0089] The knowledge retrieval module 31 is used to search the knowledge base of question-answer pairs and document types to obtain a preset amount of basic knowledge;

[0090] A prompt word construction module 32 is used to construct a first prompt word and a second prompt word based on the basic knowledge, the current context and the current question, wherein the first prompt word is selected based on the basic knowledge and is used to obtain corresponding knowledge from the basic knowledge, and the second prompt word is used to generate a reply keyword, and the reply keyword is used to generate a response answer according to the current context and the current question;

[0091] A model fusion module 33 is used to construct a main model and an enhanced model, respectively use the first prompt word to perform task training on the main model, use the second prompt word to perform task training on the enhanced model, and fuse the obtained models to obtain an initial fusion model;

[0092] The model training module 34 is used to train the initial fusion model to obtain a target fusion model.

[0093] Optionally, the knowledge retrieval module 31 includes:

[0094] A retrieval unit is used to search the knowledge base of question-answer pairs and document types to obtain M pieces of knowledge, where M is a positive integer;

[0095] The first encoding unit is used to encode each knowledge in the knowledge base into a vector to obtain a knowledge base vector , , where the knowledge base is , For the first to knowledge, is the encoding vector corresponding to the i-th knowledge, is the vector dimension;

[0096] The second encoding unit is used to encode the current question to obtain a question vector ;

[0097] Similarity calculation unit, used to calculate the cosine similarity between the knowledge base vector and the question vector ;

[0098] A screening unit, used to select the knowledge in the knowledge base vector whose cosine similarity value with the question vector is greater than a preset threshold as the knowledge to be screened;

[0099] The selection unit is used to sort the knowledge to be screened from large to small based on the cosine similarity value to obtain a knowledge sorting sequence, and select a preset number of knowledge from the front to the back as basic knowledge.

[0100] Optionally, the first encoding unit includes:

[0101] A first encoding subunit is used for encoding the question in the question-answer pair to obtain an encoding vector corresponding to the knowledge if the knowledge is in the form of a question-answer pair;

[0102] The second encoding subunit is used to encode the document if the knowledge is in the form of a document to obtain an encoding vector.

[0103] Optionally, the model fusion module 33 includes:

[0104] A vectorization unit is used to input the first prompt word into the main model to obtain a first encoding vector , and input the second prompt word into the enhanced model to obtain the second encoding vector ,in, The first prompt word passes through the main model Tokens vector after Token segmentation and word vectorization, Indicates that the second prompt word has been enhanced by the model Tokens vector after Token segmentation and word vectorization, Represents the number of Tokens of the first prompt word, Represents the number of tokens of the second prompt word, represents the vector dimension;

[0105] A two-dimensional vector generating unit is used to feed-forward the first encoding vector through the j-layer of the main model to obtain a first two-dimensional vector , and the second encoding vector is fed forward through the enhanced model i layer to obtain the second two-dimensional vector , where the total number of layers of the main model and the enhanced model are , , the number of fusion layers set is , j = - ,i= - ;

[0106] Fusion unit for Starting from the first two-dimensional vector and the second two-dimensional vector, the main model and the enhanced model are fused to obtain an initial fused model.

[0107] Figure 5 The principle block diagram of the text dialogue generation device corresponding to the text dialogue generation method in the above embodiment is shown. Figure 5 As shown, the text dialogue generation device includes a corpus acquisition module 35 and a dialogue generation module 36. The detailed description of each functional module is as follows:

[0108] A corpus acquisition module 35 is used to acquire target corpus, where the target corpus is a question-answer pair type or a document type;

[0109] The dialogue generation module 36 is used to input the target corpus into the target fusion model for task training to obtain the target text dialogue.

[0110] The specific definition of the text dialogue generation model training device / text dialogue generation device can be found in the definition of the text dialogue generation model training method / text dialogue generation method above, which will not be repeated here. Each module in the above-mentioned text dialogue generation model training device / text dialogue generation device can be implemented in whole or in part by software, hardware and a combination thereof. The above-mentioned modules can be embedded in or independent of the processor in the computer device in the form of hardware, or can be stored in the memory in the computer device in the form of software, so that the processor can call and execute the operations corresponding to the above modules.

[0111] To solve the above technical problems, the present application also provides a computer device. Figure 6 , Figure 6 This is a basic structural block diagram of the computer device in this embodiment.

[0112] The computer device 4 includes a memory 41, a processor 42, and a network interface 43 that are interconnected through a system bus. It should be noted that the figure only shows a computer device 4 with components connected to the memory 41, the processor 42, and the network interface 43, but it should be understood that it is not required to implement all the components shown, and more or fewer components can be implemented instead. Among them, those skilled in the art can understand that the computer device here is a device that can automatically perform numerical calculations and / or information processing according to pre-set or stored instructions, and its hardware includes but is not limited to microprocessors, application specific integrated circuits (Application Specific Integrated Circuit, ASIC), programmable gate arrays (Field-Programmable Gate Array, FPGA), digital processors (Digital Signal Processor, DSP), embedded devices, etc.

[0113] The computer device may be a computing device such as a desktop computer, a notebook, a PDA, a cloud server, etc. The computer device may interact with a user through a keyboard, a mouse, a remote controller, a touch pad, or a voice control device.

[0114] The memory 41 includes at least one type of readable storage medium, and the readable storage medium includes flash memory, hard disk, multimedia card, card-type memory (for example, SD or D interface display memory, etc.), random access memory (RAM), static random access memory (SRAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), programmable read-only memory (PROM), magnetic memory, disk, optical disk, etc. In some embodiments, the memory 41 can be an internal storage unit of the computer device 4, such as a hard disk or memory of the computer device 4. In other embodiments, the memory 41 can also be an external storage device of the computer device 4, such as a plug-in hard disk, a smart memory card (Smart Media Card, SMC), a secure digital (Secure Digital, SD) card, a flash card (Flash Card), etc. equipped on the computer device 4. Of course, the memory 41 can also include both the internal storage unit of the computer device 4 and its external storage device. In this embodiment, the memory 41 is generally used to store the operating system and various application software installed on the computer device 4, such as the program code of the text dialogue generation model training method, etc. In addition, the memory 41 can also be used to temporarily store various types of data that have been output or are to be output.

[0115] The processor 42 may be a central processing unit (CPU), a controller, a microcontroller, a microprocessor, or other data processing chips in some embodiments. The processor 42 is generally used to control the overall operation of the computer device 4. In this embodiment, the processor 42 is used to run the program code stored in the memory 41 or process data, such as running the program code of the text dialogue generation model training method.

[0116] The network interface 43 may include a wireless network interface or a wired network interface. The network interface 43 is generally used to establish a communication connection between the computer device 4 and other electronic devices.

[0117] The present application also provides another implementation, namely, providing a computer-readable storage medium, wherein the computer-readable storage medium stores an interface display program, and the interface display program can be executed by at least one processor so that the at least one processor performs the steps of the text dialogue generation model training method as described above.

[0118] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus a necessary general hardware platform, and of course by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present application, or the part that contributes to the prior art, can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, a magnetic disk, or an optical disk), and includes a number of instructions for enabling a terminal device (which can be a mobile phone, a computer, a server, an air conditioner, or a network device, etc.) to execute the methods described in each embodiment of the present application.

[0119] Obviously, the embodiments described above are only some embodiments of the present application, rather than all embodiments. The preferred embodiments of the present application are given in the accompanying drawings, but they do not limit the patent scope of the present application. The present application can be implemented in many different forms. On the contrary, the purpose of providing these embodiments is to make the understanding of the disclosure of the present application more thorough and comprehensive. Although the present application is described in detail with reference to the aforementioned embodiments, for those skilled in the art, it is still possible to modify the technical solutions recorded in the aforementioned specific implementation methods, or to perform equivalent replacement of some of the technical features therein. Any equivalent structure made using the contents of the specification and drawings of this application, directly or indirectly used in other related technical fields, is similarly within the scope of patent protection of this application.

Claims

1. A text dialogue generation model training method, characterized in that: include: Search the knowledge base of question-answer pairs and document types to obtain a preset amount of basic knowledge; Based on the basic knowledge, the current context and the current question, construct a first prompt word and a second prompt word, wherein the first prompt word is selected from the basic knowledge and is used to obtain corresponding knowledge from the basic knowledge, and the second prompt word is used to generate a reply keyword, and the reply keyword is used to generate a response answer according to the current context and the current question; Constructing a main model and an enhanced model, respectively using the first prompt word to perform task training on the main model, using the second prompt word to perform task training on the enhanced model, and fusing the obtained models to obtain an initial fused model; Training the initial fusion model to obtain a target fusion model; The step of constructing the main model and the enhanced model, respectively using the first prompt word to perform task training on the main model, and using the second prompt word to perform task training on the enhanced model, and fusing the obtained models to obtain an initial fusion model includes: The first prompt word is input into the main model to obtain the first encoding vector TB = {tb1, ..., tb F }, The second prompt word is input into the enhanced model to obtain a second encoding vector TA = {ta1, ..., ta L }, Among them, TB is the first prompt word after the main model M B Tokens vector after token segmentation and word vectorization. TA indicates that the second prompt word is enhanced by the model M A Tokens vector after token segmentation and word vectorization, F represents the number of tokens of the first prompt word, L represents the number of tokens of the second prompt word, d1 and d2 represent the vector dimensions; The first encoding vector is fed forward through the j-th layer of the main model to obtain a first two-dimensional vector The second encoding vector is fed forward through the i-th layer of the enhancement model to obtain a second two-dimensional vector The total number of layers of the main model and the enhanced model are L B , L A , the number of fusion layers is set to L S , j = L B -L S , i=L A -L S ; From L S Starting from the layer, the main model and the enhanced model are fused based on the first two-dimensional vector and the second two-dimensional vector to obtain an initial fused model; Among them, the S Starting from the layer, the main model and the enhanced model are fused based on the first two-dimensional vector and the second two-dimensional vector to obtain an initial fusion model including: For L s Each layer in the layer g = 1, ..., L s , construct the transformation matrix The second two-dimensional vector is transformed into the same latitude as the first two-dimensional vector to obtain a second transformed vector The second transformed vector and the first two-dimensional vector are fused using a multi-head attention mechanism to obtain a fused vector TBW j , the fusion vector TBW j It is superimposed with the first two-dimensional vector as the input vector of the next layer, and the next layer is fused until the last layer is fused to obtain the initial fusion model.

2. The text dialogue generation model training method according to claim 1, characterized in that: The search of the knowledge base of question-answer pairs and document types to obtain a preset amount of basic knowledge includes: Search the knowledge base of question-answer pairs and document types to obtain M pieces of knowledge, where M is a positive integer; Encode each knowledge in the knowledge base into a vector and obtain the knowledge base vector KE = {ke1,...,ke M },ke i ∈R d ,i=1,...,M, where the knowledge base is K={k1,...,k M },K1,...,K M For the first to Mth knowledge ke i is the encoding vector corresponding to the i-th knowledge, and d is the vector dimension; Encode the current question to get the question vector queryE∈R d ; Calculate the cosine similarity cosine(queryE,KE) between the knowledge base vector and the question vector; The knowledge in the knowledge base vector whose cosine similarity value with the question vector is greater than a preset threshold is used as the knowledge to be screened; The knowledge to be screened is sorted from large to small based on the cosine similarity value to obtain a knowledge sorting sequence, and a preset number of knowledge is selected from the front to the back as the basic knowledge.

3. The text dialogue generation model training method according to claim 2, characterized in that: The knowledge base vector KE={ke1,...,ke M }include: If the knowledge is in the form of a question-answer pair, the question in the question-answer pair is encoded to obtain an encoding vector corresponding to the knowledge; If the knowledge is in the form of a document, the document is encoded to obtain an encoding vector.

4. A text dialogue generation method, characterized in that: include: Obtaining a target corpus, wherein the target corpus is a question-answer pair type or a document type; The target corpus is input into a target fusion model for task training to obtain a target text dialogue, wherein the target fusion model is obtained according to the method according to any one of claims 1 to 3.

5. A text dialogue generation model training device, characterized in that: The text dialogue generation model training method according to claims 1 to 3 is adapted, wherein the device comprises: The knowledge retrieval module is used to search the knowledge base of question-answer pairs and document types to obtain a preset amount of basic knowledge; A prompt word construction module, used to construct a first prompt word and a second prompt word based on the basic knowledge, the current context and the current question, wherein the first prompt word is selected based on the basic knowledge and is used to obtain corresponding knowledge from the basic knowledge, and the second prompt word is used to generate a reply keyword, and the reply keyword is used to generate a response answer according to the current context and the current question; A model fusion module, used to construct a main model and an enhanced model, respectively use the first prompt word to perform task training on the main model, use the second prompt word to perform task training on the enhanced model, and fuse the obtained models to obtain an initial fusion model; The model training module is used to train the initial fusion model to obtain a target fusion model.

6. A text dialogue generating device, characterized in that: include: An acquisition module is used to acquire a target corpus, wherein the target corpus is a question-answer pair type or a document type; A generation module is used to input the target corpus into a target fusion model for task training to obtain a target text dialogue, wherein the target fusion model is obtained according to the method according to any one of claims 1 to 3.

7. A computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, it implements the text dialogue generation model training method as described in any one of claims 1 to 3, or, when the processor executes the computer program, it implements the text dialogue generation method as described in claim 4.

8. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by the processor, it implements the text dialogue generation model training method as described in any one of claims 1 to 3, or, when the computer program is executed by the processor, it implements the text dialogue generation method as described in claim 4.

Images