Training method and device of dialogue model

Abstract

The application discloses a training method and device of a dialogue model, wherein the method comprises: obtaining dialogue information, obtaining keywords in the dialogue information through a keyword extraction tool, and inputting the keywords into a search engine in sequence to obtain search knowledge documents corresponding to each keyword; for each candidate query and the searched knowledge documents thereof, performing similarity matching between the knowledge documents and dialogue replies in dialogue corpus, and recording matching scores corresponding to each query; using reinforcement learning to train a constructed query generation model, wherein the query generation model inputs dialogue context to predict a query; during the training, the recorded matching scores are used as reward functions of the corresponding queries to guide the model to predict queries with high matching scores; through the obtained query generation model, corresponding queries are predicted for the dialogue information, and knowledge documents are searched from the search engine to train a dialogue reply generation model; and thus, advantages such as mass and instant knowledge are obtained to generate dialogue replies with higher quality.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of human-computer dialogue, and particularly relates to a dialogue model training method, a computer readable storage medium, a computer device and a dialogue model training device. BACKGROUND

[0002] In the related art, human-computer dialogue as a basic application of natural language processing has always been valued by the academic and industrial circles; knowledge-assisted dialogue reply generation aims to add relevant external knowledge to the chat robot in order to generate dialogue replies containing rich common sense knowledge; most of the existing works assume that the relevant knowledge is given as input or is retrieved from a static knowledge pool; but this assumption is contrary to the real world situation, in which the knowledge is constantly updated, and the chat robot must dynamically retrieve useful knowledge, thereby resulting in poor dialogue reply quality and poor user experience. SUMMARY

[0003] The present application aims to at least solve one of the above technical problems to some extent. To this end, one object of the present application is to provide a dialogue model training method, which can retrieve knowledge from a search engine and uses a weakly supervised training method without manual annotation, evaluates the quality of candidate queries through dialogue corpus and retrieved knowledge, and then designs a reward function to train a query generation model using reinforcement learning; the query generation model obtained by training can better fit the target field scenario and obtain a higher target knowledge recall rate than the traditional keyword generation model.

[0004] A second object of the present application is to provide a computer readable storage medium.

[0005] A third object of the present application is to provide a computer device.

[0006] A fourth object of the present application is to provide a dialogue model training device.

[0007] To achieve the above object, the first aspect of the present application proposes a dialogue model training method, comprising the following steps: obtaining dialogue information, and extracting a plurality of keywords from the dialogue information through a keyword extraction tool to obtain a candidate query set; inputting each keyword in the candidate query set into a search engine in turn to obtain a search knowledge document corresponding to each keyword; performing similarity matching between the search knowledge document corresponding to each keyword and a dialogue reply in a dialogue corpus to obtain a matching score corresponding to each keyword; constructing a first network model, training the first network model according to the dialogue information and the matching score corresponding to each keyword to obtain a trained query generation model, wherein the training process includes a pre-training phase of directly fitting the keyword with the highest matching score for training and a fine-tuning phase of further training the pre-trained query generation model by taking the standardized matching score corresponding to each keyword as a reinforcement learning reward function; constructing a second network model, training the second network model according to the dialogue information, the output result of the trained query generation model and the dialogue reply to obtain a trained dialogue reply generation model, so as to obtain a dialogue reply matched with the dialogue information to be predicted according to the trained query generation model and the trained dialogue reply generation model.

[0008] According to the dialogue model training method of the present application, first, the dialogue information is obtained, and a plurality of keywords are extracted from the dialogue information through a keyword extraction tool to obtain a candidate query set; then, each keyword in the candidate query set is input into a search engine in turn to obtain a search knowledge document corresponding to each keyword; then, similarity matching is performed between the search knowledge document corresponding to each keyword and a dialogue reply in a dialogue corpus to obtain a matching score corresponding to each keyword; then, a first network model is constructed, and the first network model is trained according to the dialogue information and the matching score corresponding to each keyword to obtain a trained query generation model, wherein the training process includes a pre-training phase of directly fitting the keyword with the highest matching score for training and a fine-tuning phase of further training the pre-trained query generation model by taking the standardized matching score corresponding to each keyword as a reinforcement learning reward function; finally, a second network model is constructed, and the second network model is trained according to the dialogue information, the output result of the trained query generation model and the dialogue reply to obtain a trained dialogue reply generation model, so as to obtain a dialogue reply matched with the dialogue information to be predicted according to the trained query generation model and the trained dialogue reply generation model; thus, without any manual annotation, the knowledge obtained from the search engine can be used, which is different from the traditional static knowledge source, and the model has the advantages of obtaining massive and instant knowledge, etc., and further generates a dialogue reply with higher quality.

[0009] In addition, the method for training a dialogue model according to the above embodiments of the present application can further have the following additional technical features.

[0010] Optionally, the matching score corresponding to each keyword is obtained according to the following formula:

[0011]

[0012] for the i-th keyword q i and the corresponding search knowledge document K i ={k1,…,k5}, where u t represents a dialogue reply, the Max function is used to obtain the maximum value in the set, and k i u t is subjected to coreference resolution and stop word removal operations.

[0013] Optionally, the first network model is any one of an extraction-based query generation model or a sequence generation-based query generation model.

[0014] Optionally, the second network model is any one of a ranking-based dialogue reply generation model and a merging-based dialogue generation model.

[0015] To achieve the above object, the second aspect of the present application proposes a computer readable storage medium having a dialogue model training program stored thereon, which, when executed by a processor, implements the dialogue model training method as described above.

[0016] The computer readable storage medium according to the embodiments of the present application stores a dialogue model training program, so that when the processor executes the dialogue model training program, the dialogue model training method as described above is implemented. Thus, without any manual annotation, the knowledge obtained from search engine retrieval can be used. Unlike traditional static knowledge sources, this model can obtain massive and instant knowledge and other advantages, and thus generate higher quality dialogue replies.

[0017] To achieve the above object, the third aspect of the present application proposes a computer device, which includes a memory, a processor, and a computer program stored on the memory and executable on the processor. When the processor executes the program, the dialogue model training method as described above is implemented.

[0018] According to the computer device provided in the embodiments of the present application, the training program of the dialogue model is stored in the memory, so that the processor can implement the dialogue model training method as described above when executing the training program of the dialogue model. Therefore, without any manual annotation, the knowledge obtained from the search engine can be used, and the model can obtain mass and instant knowledge, which is different from the traditional static knowledge source, and thus a dialogue reply with higher quality can be generated.

[0019] To achieve the above object, the fourth aspect of the present application provides a dialogue model training device, comprising: an acquisition module configured to acquire dialogue information and extract a plurality of keywords from the dialogue information by using a keyword extraction tool to obtain a candidate query set; a search module configured to input each keyword in the candidate query set into a search engine in sequence to obtain a search knowledge document corresponding to each keyword; a matching module configured to perform similarity matching between the search knowledge document corresponding to each keyword and a dialogue reply in dialogue corpus to obtain a matching score corresponding to each keyword; a first training module configured to construct a first network model, train the first network model according to the dialogue information and the matching score corresponding to each keyword to obtain a trained query generation model, wherein the training process includes a pre-training stage of directly fitting the keyword with the highest matching score for training and a fine-tuning stage of further training the pre-trained query generation model by taking the standardized matching score corresponding to each keyword as a reinforcement learning reward function; and a second training module configured to construct a second network model, train the second network model according to the dialogue information, the output result of the trained query generation model and the dialogue reply to obtain a trained dialogue reply generation model, so as to obtain a dialogue reply matched with the dialogue information to be predicted according to the trained query generation model and the trained dialogue reply generation model.

[0020] According to the dialogue model training device provided in the embodiments of the present application, without any manual annotation, the knowledge obtained from the search engine can be used, and the model can obtain mass and instant knowledge, which is different from the traditional static knowledge source, and thus a dialogue reply with higher quality can be generated.

[0021] In addition, the dialogue model training device provided in the above embodiments of the present application can have the following additional technical features:

[0022] Optionally, the matching score corresponding to each keyword is obtained according to the following formula:

[0023]

[0024] For the i-th keyword q i and the corresponding search knowledge document K i= {k1,…,k5}, where u t This represents a dialogue response. The Max function is used to obtain the maximum value in the set, and in the BM25 algorithm, it calculates the maximum value for k. i ,u t The operations of dereference resolution and stop word removal were performed.

[0025] Optionally, the first network model is either an extraction-based query generation model or a sequence-based query generation model.

[0026] Optionally, the second network model is either a ranking-based dialogue response generation model or a merging-based dialogue generation model. Attached Figure Description

[0027] Figure 1 This is a flowchart illustrating a training method for a dialogue model according to an embodiment of this application;

[0028] Figure 2 This is a schematic diagram of the overall network structure according to an embodiment of this application;

[0029] Figure 3 This is a block diagram of a training apparatus for a dialogue model according to an embodiment of this application. Detailed Implementation

[0030] The embodiments of this application are described in detail below. Examples of these embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary and intended to explain this application, and should not be construed as limiting this application.

[0031] To better understand the above technical solutions, exemplary embodiments of this application will be described in more detail below with reference to the accompanying drawings. Although exemplary embodiments of this application are shown in the drawings, it should be understood that this application can be implemented in various forms and should not be limited to the embodiments set forth herein. Rather, these embodiments are provided to enable a more thorough understanding of this application and to fully convey the scope of this application to those skilled in the art.

[0032] To better understand the above technical solutions, the following will provide a detailed explanation of the technical solutions in conjunction with the accompanying drawings and specific implementation methods.

[0033] Figure 1 This is a flowchart illustrating the training method of a dialogue model according to an embodiment of this application, as shown below. Figure 1 As shown, the training method for this dialogue model includes the following steps:

[0034] S101, obtain dialogue information, and extract a plurality of keywords in the dialogue information through a keyword extraction tool, so as to obtain a candidate query set.

[0035] That is, the current dialogue context information is obtained, and a plurality of keywords are extracted in the dialogue context information through a keyword extraction tool, so as to form a candidate query set through the plurality of keywords.

[0036] It should be noted that the keyword extraction tool can adopt two representative keyword extraction tools of TagMe or Yake, the former is based on a keyword dictionary and has higher precision; the latter is an advanced model based on unsupervised extraction and has better recall rate, but introduces additional noise.

[0037] S102, input each keyword in the candidate query set to a search engine in turn, so as to obtain a retrieval knowledge document corresponding to each keyword.

[0038] It should be noted that the retrieval knowledge document can be web content.

[0039] As an embodiment, for the current dialogue context information D ＜t , n keywords are obtained as a candidate query set Q = q1, q2, …, qn by using a keyword extraction tool KeywordExtractor, t is the number of dialogue rounds. n Each keyword is input to a search engine in turn to obtain retrieval knowledge documents K1, K2, …, K m It should be noted that each query often retrieves a plurality of retrieval knowledge documents, and the number of retrieval knowledge documents corresponding to a single keyword can be limited to no more than 5, so as to be processed subsequently, that is, K i = {k1, k2, …, k5}.

[0040] S103, similarity matching is performed between the retrieval knowledge document corresponding to each keyword and the dialogue reply in the dialogue corpus, so as to obtain a matching score corresponding to each keyword.

[0041] As an embodiment, the matching score corresponding to each keyword is obtained according to the following formula:

[0042]

[0043] For the i-th keyword q i and the corresponding retrieval knowledge document K i = {k1, …, k5}, wherein u t represents the dialogue reply, the Max function is used to obtain the maximum value in the set, and k i , u t are subjected to coreference resolution and stop word removal operations.

[0044] It should be noted that the i-th keyword q i is obtained i The quality evaluation s i of the original BM25 algorithm is additionally performed on the text content (i.e. k t , u ＜t ) to eliminate ambiguity and text noise.

[0045] S104, a first network model is constructed, and the first network model is trained according to the dialogue information and the matching score corresponding to each keyword to obtain a trained query generation model, wherein the training process includes a pre-training stage of directly fitting the keyword with the highest matching score for training and a fine-tuning stage of further training the pre-trained query generation model by taking the matching score corresponding to each standardized keyword as a reinforcement learning reward function.

[0046] As an embodiment, the first network model is any one of an extraction-based query generation model or a sequence generation-based query generation model.

[0047] It should be noted that the extraction-based model can only take the string appearing in the dialogue context as the query, the search space is small, and the accuracy is higher; while the sequence generation-based model is not limited by the dialogue context, and can more freely generate queries that do not appear in the context, but the search space is large, and the accuracy is slightly lower.

[0048] In addition, as shown in Figure 2 , the query generation model (Query Producer) inputs the dialogue context D ＜t and the candidate query set Q, and outputs the predicted keyword q i .

[0049] It should be noted that the extraction-based query generation model (QP-ext) can only predict keywords that exist in the dialogue context. This model is based on a pre-trained language model (PLM), obtains candidate query representation through mean pooling (Mean-Pooling) means, and predicts candidate query probability based on a multi-layer perception (MLP) and Softmax standardization:

[0050]

[0051]

[0052] H=PLM(D ＜t )

[0053] Wherein, H represents the vector representation obtained by PLM encoding dialogue context, h i is a single character representation in the vector representation, bedi end i Representing candidate query q respectively i In context D ＜t The first and last position indices of P are given, where P is the set of predicted probabilities of QP-ext for candidate query Q.

[0054] The query generation model based on sequence generation (QP-gen) can make predictions without being limited to the dialogue context. This model is based on a pre-trained generative language model (PGM) and directly obtains the predicted probability of each candidate query.

[0055]

[0056] H i =PGM(D ＜t ,q i )

[0057] Similarly, P is the set of predicted probabilities of QP-gen for candidate query Q.

[0058] As an example, all of the above query generation models employ two-stage training.

[0059] During the pre-training phase, obtain the query q with the highest matching score. i As a pseudo-training objective, the model is trained using the maximum likelihood loss function to enable the query generation model to converge quickly, where i = argmax({s i} i≤i≤N Training was performed using standard cross-entropy loss (CE).

[0060] loss = -logp i

[0061] Where p i To query the generative model pair q in the predicted probability set P i The predicted probability.

[0062] During the fine-tuning phase, this method uses the standardized similarity matching score as the reinforcement learning reward function to further train the pre-trained query generation model, thereby improving model performance.

[0063] loss = -δ(r) a ,r b logp a

[0064]

[0065] Where r iis the normalized reward function (also called cheap noisy supervision), max and min are the maximum and minimum of the similarity matching scores in the candidate queries; a represents the sequence number of the candidate query obtained by sampling the model prediction distribution P, and b represents the sequence number of the candidate query with the maximum model prediction probability, used to stabilize the variance value of the reward function.

[0066] S105, constructing a second network model, training the second network model according to the dialogue information, the output result of the trained query generation model and the dialogue reply, to obtain a trained dialogue reply generation model, so as to obtain a dialogue reply matched with the dialogue information to be predicted according to the trained query generation model and the trained dialogue reply generation model.

[0067] As an embodiment, the second network model is any one of a ranking-based dialogue reply generation model and a merging-based dialogue generation model.

[0068] It should be noted that the ranking-based dialogue reply generation model focuses on the selection of candidate knowledge documents, and only selects a single document from the ranked candidate knowledge documents to generate a dialogue reply; while the merging-based dialogue generation model focuses on the simultaneous use of multiple knowledge documents, and inputs all candidate documents to the model to generate a dialogue reply.

[0069] In addition, as shown in Figure 2 , the dialogue reply generation model (Response Generator) inputs the dialogue context D ＜t and the query q i predicted by the trained query generation model i , and outputs the dialogue reply u t .

[0070] It should be noted that the ranking-based dialogue reply generation model (Rank-Gen) includes a ranking module (Ranker) and a generation module (Generator), wherein the ranking module is initialized by PLM, and the generation module is initialized by PGM; Rank-Gen first ranks the candidate knowledge documents by Ranker, selects the document with the highest score to input Generator to generate the final dialogue reply:

[0071] u t =Generator(D ＜t ,k j )

[0072] k j =Ranker(D ＜t ,K i )

[0073] During training, the Generator is trained by standard cross-entropy loss, while the Ranker is trained using reinforcement learning with the performance of the Generator as the reward function:

[0074] Loss = Loss rank + Loss gen

[0075]

[0076] Loss gen = -logP(u t |D ＜t ,k i )

[0077] where a, b are the document sequence numbers corresponding to the random sampling and the maximum probability in the probability distribution of the Generator inputting the knowledge documents to predict the dialogue reply, respectively.

[0078] The Merge-Gen dialogue reply generation model is a reproduction of the previous work (FiD). The model is initialized by PGM and includes an encoder (Encoder) and a decoder (Decoder). Unlike the Rank-Gen, the Merge-Gen uses a large number of knowledge documents to generate dialogue replies:

[0079] u t = Decoder([H1, H2, …, H5])

[0080] H i = Encoder(D ＜t ,k i )

[0081] The model is trained by the standard maximum likelihood function:

[0082] Loss = -logP(u t |D ＜t ,K i )

[0083] Thus, based on the trained query generation model and dialogue reply model, a complete dialogue system is formed for model testing and actual use. During operation, any query generation model and any dialogue reply generation model can be selected. The query generation model is used to obtain external knowledge documents required for dialogue from a search engine to assist the dialogue reply generation model in generating dialogue replies.

[0084] In summary, according to the training method of the dialogue model, first, dialogue information is obtained, and a plurality of keywords are extracted from the dialogue information by a keyword extraction tool to obtain a candidate query set; then, each keyword in the candidate query set is input into a search engine in turn to obtain a search knowledge document corresponding to each keyword; then, the search knowledge document corresponding to each keyword is matched with a dialogue reply in dialogue corpus in terms of similarity to obtain a matching score corresponding to each keyword; then, a first network model is constructed, and the first network model is trained according to the dialogue information and the matching score corresponding to each keyword to obtain a trained query generation model, wherein the training process includes a pre-training stage of directly fitting the keyword with the highest matching score for training and a fine-tuning stage of further training the pre-trained query generation model by taking the matching score corresponding to each keyword after standardization as a reinforcement learning reward function; finally, a second network model is constructed, and the second network model is trained according to the dialogue information, the output result of the trained query generation model and the dialogue reply to obtain a trained dialogue reply generation model, so as to obtain a dialogue reply matched with the dialogue information to be predicted according to the trained query generation model and the trained dialogue reply generation model; thus, without any manual annotation, the knowledge obtained from the search engine can be used, which is different from the traditional static knowledge source, and the model has advantages such as obtaining massive and instant knowledge, and thus generates a dialogue reply with higher quality.

[0085] To implement the above-mentioned embodiments, the embodiments of the present application provide a computer readable storage medium having a training program of a dialogue model stored thereon, which, when executed by a processor, implements the training method of the dialogue model as described above.

[0086] According to the computer readable storage medium of the embodiments of the present application, by storing the training program of the dialogue model, when the processor executes the training program of the dialogue model, the training method of the dialogue model as described above is implemented, so that when the dialogue system program is executed by the processor, relevant knowledge documents can be automatically obtained from the search engine according to the current dialogue content to generate dialogue replies containing rich common sense knowledge.

[0087] To implement the above-mentioned embodiments, the embodiments of the present application provide a computer device, which includes a memory, a processor and a computer program stored on the memory and executable on the processor, and when the processor executes the program, the training method of the dialogue model as described above is implemented.

[0088] According to the computer device provided in the embodiment of the present application, the training program of the dialogue model is stored in the memory, so that the processor can implement the dialogue model training method as described above when executing the training program of the dialogue model. Therefore, the model can refer to the common sense knowledge obtained from the search engine when generating a dialogue reply, thereby improving the dialogue reply generation quality.

[0089] To implement the above-mentioned embodiments, the embodiment of the present application further provides a dialogue model training device, as shown in the figure. Figure 3 The dialogue model training device comprises an acquisition module 10, a search module 20, a matching module 30, a first training module 40 and a second training module 50.

[0090] The acquisition module 10 is configured to acquire dialogue information and extract a plurality of keywords from the dialogue information by using a keyword extraction tool, so as to obtain a candidate query set. The search module 20 is configured to input each keyword in the candidate query set into a search engine in sequence, so as to obtain a retrieval knowledge document corresponding to each keyword. The matching module 30 is configured to perform similarity matching between the retrieval knowledge document corresponding to each keyword and a dialogue reply in dialogue corpus, so as to obtain a matching score corresponding to each keyword. The first training module 40 is configured to construct a first network model, train the first network model according to the dialogue information and the matching score corresponding to each keyword, so as to obtain a trained query generation model. The training process comprises a pre-training stage of directly fitting the keyword with the highest matching score for training and a fine-tuning stage of further training the pre-trained query generation model by taking the standardized matching score corresponding to each keyword as a reinforcement learning reward function. The second training module 50 is configured to construct a second network model, train the second network model according to the dialogue information, the output result of the trained query generation model and the dialogue reply, so as to obtain a trained dialogue reply generation model, so as to obtain a dialogue reply matched with the dialogue information to be predicted according to the trained query generation model and the trained dialogue reply generation model.

[0091] As an embodiment, the matching score corresponding to each keyword is obtained according to the following formula:

[0092]

[0093] For the i-th keyword q i and the corresponding retrieval knowledge document K i ={k1,…,k5}, wherein u t represents a dialogue reply, the Max function is used to obtain the maximum value in the set, and k i ,u t has been subjected to coreference resolution and stop word removal operation.

[0094] As an embodiment, the first network model is any one of an extract-based query generation model or a sequence-based query generation model.

[0095] As an embodiment, the second network model is any one of a ranking-based dialogue reply generation model and a merging-based dialogue generation model.

[0096] It should be noted that the above description of the training method of the dialogue model is also applicable to the training apparatus of the dialogue model, and thus is not repeated here. Figure 1 The training method of the dialogue model described above is also applicable to the training apparatus of the dialogue model, and thus is not repeated here.

[0097] In summary, according to the training apparatus of the dialogue model of the embodiments of the present application, without any manual annotation, the knowledge obtained from the search engine retrieval can be used, and the model can obtain the advantages of massive and instant knowledge, etc. different from the traditional static knowledge source, and thus a dialogue reply with higher quality can be generated.

[0098] Those skilled in the art should understand that the embodiments of the present application can be provided as a method, a system, or a computer program product. Therefore, the present application can take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment combining software and hardware aspects. Moreover, the present application can take the form of a computer program product implemented on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROMs, optical storage, etc.) containing computer-usable program code.

[0099] The present application is described with reference to flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each flow and / or block in the flowcharts and / or block diagrams, and the combination of the flows and / or blocks in the flowcharts and / or block diagrams can be implemented by computer program instructions. These computer program instructions can be provided to a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing apparatus to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing apparatus produce a device that implements the functions specified in the flowcharts and / or block diagrams. Figure 1 The functions specified in one or more flows and / or blocks Figure 1 The apparatus that implements the functions specified in one or more flows and / or blocks.

[0100] These computer program instructions can also be stored in a computer-readable memory that can cause the computer or other programmable data processing apparatus to work in a specific manner, so that the instructions stored in the computer-readable memory produce a manufactured product including instruction apparatus that implements the functions specified in the flowcharts and / or block diagrams. Figure 1 The functions specified in one or more flows and / or blocks Figure 1 The apparatus that implements the functions specified in one or more flows and / or blocks.

[0101] These computer program instructions can also be loaded into a computer or other programmable data processing devices, so that a series of operation steps are performed on the computer or other programmable devices to generate computer-implemented processes, thus the instructions executed on the computer or other programmable devices provide a process for implementing the functions specified in the flowcharts Figure 1 one or more flowcharts and / or blocks Figure 1 one or more flowcharts and / or blocks

[0102] It should be noted that in the claims, any reference signs placed between parentheses shall not be construed as limiting the claim. The word "comprising" does not exclude the presence of elements or steps not listed in a claim. The word "a" or "an" preceding an element does not exclude the presence of a plurality of such elements. The application can be implemented by means of both hardware and software, and any combination thereof. In a unit claim, several devices can be listed with a comma. The use of the term "first", "second", etc. does not imply any order. These terms can be understood as names.

[0103] Although the preferred embodiments of the application have been described, those skilled in the art will be able to make additional modifications and variations without departing from the scope of the application. Accordingly, the appended claims are intended to encompass all such modifications and variations as falling within the scope of the application.

[0104] Obviously, many modifications and variations of the present application are possible in light of the above teachings. It is, therefore, to be understood that within the scope of the appended claims and their equivalents, the application can be practiced otherwise than as specifically described.

[0105] In the description of the application, it is to be understood that the terms "first", "second", etc. are used only for descriptive purposes and not to connote or imply relative importance. Thus, a feature described as "first" can implicitly or explicitly include one or more of the same feature. In the description of the application, the meaning of "a", "an" and "the" includes plural references unless the context clearly dictates otherwise.

[0106] In the present application, unless specifically defined otherwise, the terms "mounting", "connected", "connection", "fixed", and the like should be construed broadly and do not necessarily mean fixedly connected, but can mean removably connected, or integral; can mean mechanical connection, or electrical connection; can mean direct connection, or indirect connection via an intermediate medium; can mean internal communication between two elements, or interaction between two elements. The specific meaning of the above terms in the present application can be understood according to the specific circumstances by those of ordinary skill in the art.

[0107] In the present application, unless specifically defined otherwise, the first feature is "on" or "under" the second feature can be that the first and second features are in direct contact, or the first and second features are in indirect contact through an intermediate medium. Moreover, the first feature "above", "over" and "on" the second feature can be that the first feature is directly above or obliquely above the second feature, or only means that the horizontal height of the first feature is higher than that of the second feature. The first feature "below", "under" and "under" the second feature can be that the first feature is directly below or obliquely below the second feature, or only means that the horizontal height of the first feature is less than that of the second feature.

[0108] In the description of the present application, the description of the terms "one embodiment", "some embodiments", "example", "specific example", or "some examples" means that the specific features, structures, materials or characteristics described in connection with the embodiment or example are included in at least one embodiment or example of the present application. In the present application, the illustrative description of the above terms should not be understood as necessarily referring to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described can be combined in any appropriate manner in any one or more embodiments or examples. In addition, different embodiments or examples described in the present application and the features of different embodiments or examples can be combined and combined by those skilled in the art without contradiction.

[0109] Although the embodiments of the present application have been shown and described above, it should be understood that the above embodiments are exemplary and should not be construed as limiting the present application, and those of ordinary skill in the art can make changes, modifications, replacements and variations to the above embodiments within the scope of the present application.

Claims

1. A method for training a dialogue model, characterized in that, Includes the following steps: Obtain dialogue information and extract multiple keywords from the dialogue information using a keyword extraction tool to obtain a candidate query set; Each keyword in the candidate query set is sequentially input into the search engine to obtain the corresponding retrieval knowledge document for each keyword; The similarity of the retrieval knowledge document corresponding to each keyword with the dialogue responses in the dialogue corpus is performed to obtain the matching score for each keyword; A first network model is constructed, and the first network model is trained based on the dialogue information and the matching score corresponding to each keyword to obtain a trained query generation model. The training process includes a pre-training stage in which the keyword with the highest matching score is directly fitted for training, and a fine-tuning stage in which the matching score corresponding to each standardized keyword is used as a reinforcement learning reward function to further train the pre-trained query generation model. The first network model is either an extraction-based query generation model or a sequence-based query generation model. A second network model is constructed. The second network model is trained based on the dialogue information, the output of the trained query generation model, and the dialogue response to obtain a trained dialogue response generation model. This model is used to obtain a dialogue response that matches the dialogue information to be predicted based on the trained query generation model and the trained dialogue response generation model. The second network model is either a ranking-based dialogue response generation model or a merging-based dialogue generation model.

2. The training method for the dialogue model as described in claim 1, characterized in that, The matching score for each keyword is obtained using the following formula: For the Keyword and its corresponding retrieval knowledge documents ,in, Indicates a reply to the conversation. The function is used to find the maximum value in the set, and in the BM25 algorithm, it is applied to... The operations of dereference resolution and stop word removal were performed.

3. A computer-readable storage medium, characterized in that, It stores a training program for a dialogue model, which, when executed by a processor, implements the training method for the dialogue model as described in any one of claims 1-2.

4. A computer device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the training method for the dialogue model as described in any one of claims 1-2.

5. A training device for a dialogue model, characterized in that, include: The acquisition module is used to acquire dialogue information and extract multiple keywords from the dialogue information using a keyword extraction tool in order to obtain a candidate query set. The search module is used to input each keyword in the candidate query set into the search engine in sequence so as to obtain the search knowledge document corresponding to each keyword; The matching module is used to perform similarity matching between the retrieval knowledge document corresponding to each keyword and the dialogue response in the dialogue corpus to obtain the matching score for each keyword. The first training module is used to construct a first network model and train the first network model according to the dialogue information and the matching score corresponding to each keyword to obtain a trained query generation model. The training process includes a pre-training stage in which the keyword with the highest matching score is directly fitted for training, and a fine-tuning stage in which the matching score corresponding to each standardized keyword is used as a reinforcement learning reward function to further train the pre-trained query generation model. The first network model is either an extraction-based query generation model or a sequence-based query generation model. The second training module is used to construct a second network model. The second network model is trained based on the dialogue information, the output of the trained query generation model, and the dialogue response to obtain a trained dialogue response generation model. This model is used to obtain a dialogue response that matches the dialogue information to be predicted based on the trained query generation model and the trained dialogue response generation model. The second network model is either a ranking-based dialogue response generation model or a merging-based dialogue response generation model.

6. The training apparatus for the dialogue model as described in claim 5, characterized in that, The matching score for each keyword is obtained using the following formula: For the Keyword and its corresponding retrieval knowledge documents ,in, Indicates a reply to the conversation. The function is used to find the maximum value in the set, and in the BM25 algorithm, it is applied to... The operations of dereference resolution and stop word removal were performed.

Images