Man-machine conversation method, device, storage medium and electronic equipment
By randomly selecting the latent vectors of the text model to generate the target response text, the problem of monotonous chatbot responses is solved, and the diversity and fun of the dialogue are enhanced.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- BEIJING XIAOMI MOBILE SOFTWARE CO LTD
- Filing Date
- 2021-10-08
- Publication Date
- 2026-07-21
AI Technical Summary
Existing chatbots provide monotonous and uninteresting answers to the same question, failing to simulate the diversity of many-to-one conversations in the real world.
By randomly selecting latent vectors from a pre-trained text model, target response text is generated through an embedding layer and a decoder. The model is trained using a quantized vector-variable autoencoder framework to increase the diversity of dialogues.
By randomly selecting latent vectors to generate target response texts, the diversity of the text is improved, making the dialogue more natural and interesting.
Smart Images
Figure CN113901832B_ABST
Abstract
Description
Technical Field
[0001] This disclosure relates to the field of computer technology, and in particular to a human-computer interaction method, apparatus, storage medium, and electronic device. Background Technology
[0002] Chatbots are currently widely used on instant messaging platforms, generally for entertainment, retail marketing, and customer service purposes. A chatbot is a computer program that converses via dialogue or text, simulating human conversation. The program responds to pre-set answers using algorithms. Developers store natural human conversation content in a database for the program to learn from. When a chatbot receives a conversation, it identifies phrases and sentences, and through dialogue prediction, finds the most appropriate answer from the database to reply. Summary of the Invention
[0003] To overcome the problems existing in related technologies, this disclosure provides a human-computer dialogue method, apparatus, storage medium, and electronic device.
[0004] According to a first aspect of the present disclosure, a human-computer dialogue method is provided, comprising:
[0005] Get the original text of the input;
[0006] A first target hidden vector is randomly selected from multiple candidate hidden vectors, wherein the multiple candidate hidden vectors are multiple preset parameter vectors of a pre-trained text model, and different preset parameter vectors are used to represent the semantic information of different responses;
[0007] Based on the original text and the first target latent vector, the target response text corresponding to the original text is generated through the text model.
[0008] In some embodiments, the text model includes an embedding layer and a decoder;
[0009] The step of generating the target response text corresponding to the original text using the text model based on the original text and the first target latent vector includes:
[0010] Add a preset start character and a preset separator to the beginning and end of the original text respectively to obtain the target concatenated text;
[0011] The target concatenated text is input into the embedding layer to obtain the first feature vector corresponding to each character in the target concatenated text;
[0012] Each of the first feature vectors is added to the first target latent vector to obtain multiple second feature vectors;
[0013] Based on the plurality of second feature vectors, the target response text corresponding to the original text is generated through the embedding layer and the decoder.
[0014] In some embodiments, generating the target response text corresponding to the original text based on the plurality of second feature vectors through the embedding layer and the decoder includes:
[0015] The plurality of second feature vectors are input into the decoder to obtain a first predicted character, and the first predicted character is used as the current character;
[0016] The current character is input into the embedding layer to obtain the third feature vector corresponding to the current character, and the third feature vector is added to the first target latent vector to obtain the fourth feature vector;
[0017] The plurality of second feature vectors and the fourth feature vector are input into the decoder to obtain a second predicted character, and the second predicted character is used as the new current character, and the plurality of second feature vectors and the fourth feature vector are used as a new plurality of second feature vectors;
[0018] The process of inputting the current character into the embedding layer to obtain the third feature vector corresponding to the current character, and adding the third feature vector to the first target latent vector to obtain the fourth feature vector, continues until the new current character is a preset end character.
[0019] Based on the first predicted character and the second predicted character generated each time, the target response text corresponding to the original text is generated.
[0020] In some embodiments, generating the target response text corresponding to the original text based on the first predicted character and the second predicted character generated each time includes:
[0021] The first predicted character and each of the second predicted characters obtained each time, excluding the preset end character, are concatenated in order of generation time from earliest to latest to obtain the target response text corresponding to the original text.
[0022] In some embodiments, the text model is trained using a quantized vector-variational autoencoder framework.
[0023] In some embodiments, the text model is trained in the following manner:
[0024] Obtain the sample text and the corresponding sample response text;
[0025] The preset start character, the sample text, the preset separator, the sample response text, and the preset end character are concatenated sequentially to obtain the sample concatenated text;
[0026] The text model is trained by using the sample concatenated text as input to the embedding layer, adding the fifth feature vector corresponding to each character in the sample concatenated text output by the embedding layer to the second target latent vector to obtain multiple sixth feature vectors as input to the decoder, and using the sample response text as the target output of the decoder. The second target latent vector is the preset parameter vector with the highest similarity to the fifth feature vector corresponding to the preset start character among the multiple preset parameter vectors of the current embedding layer.
[0027] In some embodiments, the text model is trained in the following manner:
[0028] Obtain unsupervised corpus;
[0029] The text model is pre-trained in unsupervised mode using the unsupervised corpus to obtain an initial model;
[0030] Obtain the sample text and the corresponding sample response text;
[0031] The preset start character, the sample text, the preset separator, the sample response text, and the preset end character are concatenated sequentially to obtain the sample concatenated text;
[0032] The text model is trained by using the sample concatenated text as input to the embedding layer of the initial model, adding the fifth feature vector corresponding to each character in the sample concatenated text output by the embedding layer of the initial model to the second target latent vector to obtain multiple sixth feature vectors, and using the sample response text as the target output of the decoder of the initial model. The second target latent vector is the preset parameter vector with the highest similarity to the fifth feature vector corresponding to the preset start character among the multiple preset parameter vectors of the current embedding layer.
[0033] According to a second aspect of the present disclosure, a human-computer interaction device is provided, comprising:
[0034] The acquisition module is configured to acquire the raw text of the input.
[0035] The selection module is configured to randomly select a first target latent vector from multiple candidate latent vectors, wherein the multiple candidate latent vectors are multiple preset parameter vectors of a pre-trained text model, and different preset parameter vectors are used to represent the semantic information of different responses.
[0036] The generation module is configured to generate the target response text corresponding to the original text based on the original text obtained by the acquisition module and the first target latent vector selected by the selection module, through the text model.
[0037] According to a third aspect of the present disclosure, a computer-readable storage medium is provided, having stored thereon computer program instructions that, when executed by a processor, implement the steps of the human-computer dialogue method provided in the first aspect of the present disclosure.
[0038] According to a fourth aspect of the present disclosure, an electronic device is provided, comprising:
[0039] processor;
[0040] Memory used to store processor-executable instructions;
[0041] The processor is configured to execute the human-computer dialogue method provided in the first aspect of this disclosure.
[0042] The technical solutions provided by the embodiments of this disclosure can include the following beneficial effects: First, the input original text is obtained; simultaneously, a first target latent vector is randomly selected from multiple candidate latent vectors, where the multiple candidate latent vectors are multiple preset parameter vectors of a pre-trained text model, and different preset parameter vectors are used to represent the semantic information of different responses; based on the original text and the first target latent vector, the target response text corresponding to the original text is generated through the text model. Since each dialogue randomly selects a latent vector as the first target latent vector from multiple candidate latent vectors representing the semantic information of different responses, and then generates the target response text corresponding to the original text based on the randomly selected first target latent vector and the original text, the semantic information represented by the randomly selected first target latent variable in each generated target response text is different for the same original text. Therefore, the diversity of dialogue generation can be improved, making the dialogue more natural and interesting.
[0043] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and are not intended to limit this disclosure. Attached Figure Description
[0044] The accompanying drawings, which are incorporated in and form a part of this specification, illustrate embodiments consistent with this disclosure and, together with the description, serve to explain the principles of this disclosure.
[0045] Figure 1 This is a flowchart illustrating a human-computer dialogue method according to an exemplary embodiment of the present disclosure.
[0046] Figure 2This is a flowchart illustrating another human-computer dialogue method according to an exemplary embodiment of the present disclosure.
[0047] Figure 3 This is a flowchart illustrating a human-computer dialogue method according to an exemplary embodiment of the present disclosure.
[0048] Figure 4 This is a flowchart illustrating a text model training method according to an exemplary embodiment of the present disclosure.
[0049] Figure 5 This is a flowchart illustrating another text model training method according to an exemplary embodiment of the present disclosure.
[0050] Figure 6 This is a block diagram illustrating a human-computer interaction device according to an exemplary embodiment of the present disclosure.
[0051] Figure 7 This is a block diagram illustrating a human-computer interaction device according to an exemplary embodiment of the present disclosure.
[0052] Figure 8 This is a block diagram illustrating a human-computer interaction device according to an exemplary embodiment of the present disclosure. Detailed Implementation
[0053] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numerals in different drawings denote the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this disclosure. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this disclosure as detailed in the appended claims.
[0054] Chatbots are currently widely used on instant messaging platforms, generally for entertainment, retail marketing, and customer service purposes. A chatbot is a computer program that converses via dialogue or text, simulating human conversation. The program responds to pre-set answers using algorithms. Developers store natural human conversation content in a database for the program to learn from. When the chatbot receives a dialogue, it identifies phrases and sentences, and through dialogue prediction, finds the most appropriate answer from the database. Because the most appropriate answer is always found from the database, the response to the same question is always the same, resulting in a lack of engagement. However, in the real world, dialogues can be many-to-one; for example, a question like "Have you eaten?" can elicit a variety of responses. Therefore, this disclosure provides a human-computer dialogue method, apparatus, storage medium, and electronic device.
[0055] Figure 1This is a flowchart illustrating a human-computer dialogue method according to an exemplary embodiment, wherein the method is used in terminal devices such as smartphones, tablets, and personal computers, and can also be used in servers. Figure 1 As shown, the above human-computer dialogue method may include the following S101 to S103.
[0056] In S101, the original input text is obtained.
[0057] In this disclosure, the original text is typically a question in a dialogue. When the human-computer dialogue method is applied to a terminal device, the original text can be text obtained by the terminal device through its own input interface, or text sent to the terminal device by other devices via network communication. When the human-computer dialogue method is applied to a server, the original text can be text input by the user and sent by other devices to the server. These other devices can be, for example, smartphones, tablets, or other terminal devices.
[0058] In S102, the first target hidden vector is randomly selected from multiple candidate hidden vectors.
[0059] In this disclosure, multiple (e.g., 20) candidate latent vectors are multiple preset parameter vectors of a pre-trained text model, with different preset parameter vectors used to represent the semantic information of different responses. These preset parameter vectors can be one of the unused parameter vectors of the text model.
[0060] For example, the text model mentioned above can be a GPT2 (Generative Pre-Training) model, a Bidirectional Encoder Representation from Transformers (BERT) model, or an ELMo (Embeddings from Language Models) word vector model.
[0061] In S103, the target response text corresponding to the original text is generated through the text model based on the original text and the first target latent vector.
[0062] For example, the above-mentioned candidate latent vectors include candidate latent vector 1, candidate latent vector 2, and candidate latent vector 3; the original text is "Have you eaten today?", the first target latent vector is candidate latent vector 1, and the target response text generated by the text model is "I have eaten".
[0063] For example, the above-mentioned candidate latent vectors include candidate latent vector 1, candidate latent vector 2 and candidate latent vector 3; the original text is "Have you eaten today?", the first target latent vector is candidate latent vector 2, and the target response text generated by the text model is "I'm very full".
[0064] For example, the above-mentioned candidate latent vectors include candidate latent vector 1, candidate latent vector 2, and candidate latent vector 3; the original text is "Have you eaten today?", the first target latent vector is candidate latent vector 3, and the target response text generated by the text model is "Haven't eaten yet".
[0065] The technical solutions provided by the embodiments of this disclosure can include the following beneficial effects: First, the input original text is obtained; simultaneously, a first target latent vector is randomly selected from multiple candidate latent vectors, where the multiple candidate latent vectors are multiple preset parameter vectors of a pre-trained text model, and different preset parameter vectors are used to represent the semantic information of different responses; based on the original text and the first target latent vector, the target response text corresponding to the original text is generated through the text model. Since each dialogue randomly selects a latent vector as the first target latent vector from multiple candidate latent vectors representing the semantic information of different responses, and then generates the target response text corresponding to the original text based on the randomly selected first target latent vector and the original text, for the same original text, the target response text generated each time is different depending on the semantic information represented by the selected first target latent variable. Therefore, the diversity of dialogue generation can be improved, making the dialogue more natural and interesting.
[0066] The following provides a detailed explanation of the specific implementation method for generating the target response text corresponding to the original text based on the original text and the first target latent vector in step S103 above, using a text model. Specifically, the text model includes an embedding layer and a decoder. At this point, it can be achieved through... Figure 2 This is achieved through S1031 to S1034 shown in the figure.
[0067] In S1031, preset start characters and preset separators are added to the beginning and end of the original text respectively to obtain the target concatenated text.
[0068] In S1032, the target concatenated text is input into the embedding layer to obtain the first feature vector corresponding to each character in the target concatenated text.
[0069] In this disclosure, the embedding layer of the text model is used to encode the target concatenated text to obtain a first feature vector corresponding to each character in the target concatenated text.
[0070] In S1033, each first feature vector is added to the first target latent vector to obtain multiple second feature vectors.
[0071] In S1034, the target response text corresponding to the original text is generated based on multiple second feature vectors through an embedding layer and a decoder.
[0072] The following provides a detailed explanation of the specific implementation method in S1034 above, which involves generating the target response text corresponding to the original text based on multiple second feature vectors through an embedding layer and a decoder. Specifically, it can be achieved through... Figure 3 The following steps are implemented using S10341 to S10345 as shown:
[0073] In S10341, multiple second feature vectors are input into the decoder to obtain the first predicted character, and the first predicted character is used as the current character.
[0074] In this disclosure, multiple second feature vectors are input into the decoder to obtain the first character in the target response text corresponding to the original text, i.e., the first predicted character.
[0075] In S10342, the current character is input into the embedding layer to obtain the third feature vector corresponding to the current character, and the third feature vector is added to the first target latent vector to obtain the fourth feature vector.
[0076] In S10343, multiple second feature vectors and fourth feature vectors are input into the decoder to obtain the second predicted character, and the second predicted character is used as the new current character, and multiple second feature vectors and fourth feature vectors are used as new multiple second feature vectors.
[0077] In this disclosure, multiple second feature vectors and fourth feature vectors are input into the decoder of the text model to obtain the next character of the current character in the target response text corresponding to the original text, i.e., the second predicted character.
[0078] In S10344, it is determined whether the new current character is the preset end character.
[0079] If the new current character is not the preset end character, it means that the complete target response text has not been obtained. In this case, it is necessary to continue predicting the next character of the new current character, that is, return to S10342 above; if the new current character is the preset end character, it means that the complete target response text has been obtained. In this case, stop character prediction, and then execute S10345 below.
[0080] In S10345, the target response text corresponding to the original text is generated based on the first predicted character and the second predicted character generated each time.
[0081] Specifically, the first predicted character and each second preset character among the second predicted characters obtained each time, excluding the preset end character, can be concatenated in order of generation time from earliest to latest to obtain the target response text corresponding to the original text.
[0082] Exemplarily, the original text is "Have you eaten today", the first target hidden vector is candidate hidden vector 1, the preset start character is "CLS", the preset separator is "SEP", and the preset end character is "SEP". At this time, the corresponding target response text can be generated through the following steps (1) to step (11):
[0083] (1) Add the preset start character "CLS" and the preset separator "SEP" to the beginning and end of the original text "Have you eaten today" respectively to obtain the target concatenated text "CLS Have you eaten today SEP";
[0084] (2) Input the target concatenated text "CLS Have you eaten today SEP" into the embedding layer of the text model to obtain the first feature vector corresponding to "CLS", the first feature vector corresponding to "Have", the first feature vector corresponding to "you", the first feature vector corresponding to "eaten", the first feature vector corresponding to "today", the first feature vector corresponding to "SEP";
[0085] (3) Add each of the first feature vectors obtained in the above step (2) to the first target hidden vector (i.e., candidate hidden vector 1) respectively to obtain multiple second feature vectors;
[0086] (4) Input the multiple second feature vectors obtained in the above step (3) into the decoder of the text model to predict the first character in the target response text corresponding to the original text "Have you eaten today", and obtain the first predicted character "eaten". At this time, the current character is "eaten";
[0087] (5) Input the current character "eaten" into the embedding layer of the text model to obtain the third feature vector corresponding to the current character "eaten", and add the third feature vector corresponding to the current character "eaten" to the first target hidden vector to obtain the fourth feature vector;
[0088] (6) Input the multiple second feature vectors obtained in the above step (3) and the fourth feature vector obtained in the above step (5) into the decoder of the text model to predict the second character in the target response text corresponding to the original text "Have you eaten today", and obtain the second predicted character "before". At this time, the current character is "before", and the new multiple second feature vectors include the multiple second feature vectors obtained in the above step (3) and the fourth feature vector obtained in the above step (5);
[0089] (7) The current character "过" is not the preset end character "SEP". At this time, the current character "过" is input into the embedding layer of the text model to obtain the third feature vector corresponding to the current character "过", and the third feature vector corresponding to the current character "过" is added to the first target hidden vector to obtain the fourth feature vector;
[0090] (8) The new multiple second feature vectors (including the multiple second feature vectors obtained in step (3) above and the fourth feature vector obtained in step (5) above) and the fourth feature vector obtained in step (7) above are input into the decoder of the text model to predict the third character in the target reply text corresponding to the original text "你今天吃了吗", and the second predicted character "了" is obtained. At this time, the current character is "了", and the new multiple second feature vectors include the multiple second feature vectors obtained in step (3) above, the fourth feature vector obtained in step (5) above, and the fourth feature vector obtained in step (7) above;
[0091] (9) The current character "了" is not the preset end character "SEP". At this time, the current character "了" is input into the embedding layer of the text model to obtain the third feature vector corresponding to the current character "了", and the third feature vector corresponding to the current character "了" is added to the first target hidden vector to obtain the fourth feature vector;
[0092] (10) The new multiple second feature vectors (including the multiple second feature vectors obtained in step (3) above, the fourth feature vector obtained in step (5) above, and the fourth feature vector obtained in step (7) above) and the fourth feature vector obtained in step (9) above are input into the decoder of the text model to predict the fourth character in the target reply text corresponding to the original text, and the second predicted character "SEP" is obtained. At this time, the current character is "SEP", and the new multiple second feature vectors include the multiple second feature vectors obtained in step (3) above, the fourth feature vector obtained in step (5) above, the fourth feature vector obtained in step (7) above, and the fourth feature vector obtained in step (9) above;
[0093] (11) The current character "SEP" is the preset end character "SEP". At this time, the first predicted character "吃", and each second preset character (including "过", "了", "SEP") obtained each time, except for the preset end character "SEP", are concatenated in the order from earliest generation time to latest generation time to obtain the target reply text "吃过了" corresponding to the original text.
[0094] The following details the specific training methods for the aforementioned text model. In this disclosure, a Vector Quantized-Variational Auto Encoder (VQ-VAE) framework can be used to train the text model. Specifically, the text model can be trained in various ways. In one implementation, it can be trained using… Figure 4 The text model shown is trained using S401 to S403.
[0095] In S401, obtain the sample text and the corresponding sample response text.
[0096] In S402, the preset start character, sample text, preset separator, sample reply text, and preset end character are concatenated in sequence to obtain the sample concatenated text.
[0097] In S403, the model is trained by using the sample concatenated text as the input to the embedding layer, adding the fifth feature vector corresponding to each character in the sample concatenated text output by the embedding layer to the second target latent vector to obtain multiple sixth feature vectors as the input to the decoder, and using the sample response text as the target output of the decoder.
[0098] In this disclosure, the second target latent vector is the preset parameter vector among the multiple preset parameter vectors of the current embedding layer that has the highest similarity to the fifth feature vector corresponding to the preset start symbol. For example, the similarity between each preset parameter vector and the fifth feature vector corresponding to the preset start symbol can be measured using methods such as Euclidean distance or cosine distance. During the model training phase, the multiple preset parameter vectors of the embedding layer can learn and capture different semantic information.
[0099] In another implementation, it can be achieved by... Figure 5 The text model shown is trained using S501 to S505.
[0100] In S501, unsupervised corpus is obtained.
[0101] In this disclosure, unsupervised corpora can be obtained from corpora or from various types of books, such as suspense, horror, and romance novels. Furthermore, the type of unsupervised corpora can be classical Chinese, ancient poetry, vernacular Chinese, etc. This disclosure does not impose any specific limitations on these types of corpora.
[0102] In S502, the text model is pre-trained in unsupervised corpus to obtain the initial model.
[0103] In S503, obtain the sample text and the corresponding sample response text.
[0104] In S504, the preset start character, sample text, preset separator, sample reply text, and preset end character are concatenated in sequence to obtain the sample concatenated text.
[0105] In S505, the model is trained by using the sample concatenated text as the input to the embedding layer of the initial model, adding the fifth feature vector corresponding to each character in the sample concatenated text output by the embedding layer of the initial model to the second target latent vector to obtain multiple sixth feature vectors, and using the sample response text as the target output of the decoder of the initial model, so as to obtain the text model.
[0106] After unsupervised pre-training of the text model using unsupervised corpus, an initial model is obtained. Then, the pre-trained initial model is fine-tuned using sample texts and their corresponding sample responses (i.e., dialogue corpus), resulting in the aforementioned text model. Since the pre-trained initial model is scalable, training (fine-tuning) it with a small amount of dialogue corpus allows the trained text model to output response text to the input text. Furthermore, the acquisition of unsupervised corpus is simple, thus reducing costs.
[0107] In addition, the following loss function can be used for model training during the above text model training process:
[0108]
[0109] Where, loss rec is the loss function; x is the sample text; y is the sample response text corresponding to the sample text; sg is the stop gradient calculation; h (x,y) The fifth feature vector corresponding to the preset start symbol; z is the second target latent vector; -log p(y|x, h) (x,y) +sg[zh (x,y) ]) is the maximum likelihood estimate, where p is the probability and β is a constant, for example, β = 0.25.
[0110] It should be noted that when the text model described above is a GPT2 model, the left-to-right restriction in the decoder part of the GPT2 model can be removed (i.e., the decoder can predict characters using context). In this way, during the self-attention mechanism operation, every pair of characters has already made contact with each other, and the GPT2 model is then converted into a BERT model. Therefore, the feature vector corresponding to the preset start character simultaneously covers both the sample text and the corresponding sample response text, i.e., it covers both the question and the answer. Thus, the fifth feature vector corresponding to the preset start character can be used to perform similarity calculations with the current preset parameter vectors of the embedding layer to select the latent variable most relevant to the original text.
[0111] Based on the same inventive concept, this disclosure also provides a human-computer dialogue device. For example... Figure 6 As shown, the dialogue device 600 includes:
[0112] The acquisition module 601 is configured to acquire the raw text of the input.
[0113] Selection module 602 is configured to randomly select a first target latent vector from multiple candidate latent vectors, wherein the multiple candidate latent vectors are multiple preset parameter vectors of a pre-trained text model, and different preset parameter vectors are used to represent the semantic information of different responses.
[0114] The generation module 603 is configured to generate the target response text corresponding to the original text based on the original text obtained by the acquisition module 601 and the first target latent vector selected by the selection module 602, through the text model.
[0115] The technical solutions provided by the embodiments of this disclosure can include the following beneficial effects: First, the input original text is obtained; simultaneously, a first target latent vector is randomly selected from multiple candidate latent vectors, where the multiple candidate latent vectors are multiple preset parameter vectors of a pre-trained text model, and different preset parameter vectors are used to represent the semantic information of different responses; based on the original text and the first target latent vector, the target response text corresponding to the original text is generated through the text model. Since each dialogue randomly selects a latent vector as the first target latent vector from multiple candidate latent vectors representing the semantic information of different responses, and then generates the target response text corresponding to the original text based on the randomly selected first target latent vector and the original text, for the same original text, the target response text generated each time is different depending on the semantic information represented by the selected first target latent variable. Therefore, the diversity of dialogue generation can be improved, making the dialogue more natural and interesting.
[0116] In some embodiments, the text model includes an embedding layer and a decoder;
[0117] The generation module 603 includes:
[0118] Add a submodule, configured to add a preset start character and a preset separator to the beginning and end of the original text respectively, to obtain the target concatenated text;
[0119] The first input submodule is configured to input the target concatenated text into the embedding layer to obtain a first feature vector corresponding to each character in the target concatenated text;
[0120] The processing submodule is configured to add each of the first feature vectors to the first target latent vector to obtain multiple second feature vectors;
[0121] The first generation submodule is configured to generate the target response text corresponding to the original text based on the plurality of second feature vectors, through the embedding layer and the decoder.
[0122] In some embodiments, the first generation submodule includes:
[0123] The second input submodule is configured to input the plurality of second feature vectors into the decoder to obtain a first predicted character, and to use the first predicted character as the current character;
[0124] The third input submodule is configured to input the current character into the embedding layer to obtain the third feature vector corresponding to the current character, and add the third feature vector to the first target latent vector to obtain the fourth feature vector;
[0125] The fourth input submodule is configured to input the plurality of second feature vectors and the fourth feature vector into the decoder to obtain a second predicted character, and to use the second predicted character as the new current character, and the plurality of second feature vectors and the fourth feature vector as new plurality of second feature vectors;
[0126] The trigger submodule is configured to trigger the third input submodule to input the current character into the embedding layer, obtain the third feature vector corresponding to the current character, and add the third feature vector to the first target latent vector to obtain the fourth feature vector, until the new current character is a preset end character;
[0127] The second generation submodule is configured to generate the target response text corresponding to the original text based on the first predicted character and the second predicted character generated each time.
[0128] In some embodiments, generating the target response text corresponding to the original text based on the first predicted character and the second predicted character generated each time includes:
[0129] The first predicted character and each of the second predicted characters obtained each time, excluding the preset end character, are concatenated in order of generation time from earliest to latest to obtain the target response text corresponding to the original text.
[0130] In some embodiments, the text model is trained using a quantized vector-variational autoencoder framework.
[0131] In some embodiments, the text model is trained using a first model training device. The first model training device includes:
[0132] The first sample acquisition module is configured to acquire sample text and the sample response text corresponding to the sample text.
[0133] The first splicing module is configured to sequentially splice the preset start character, the sample text, the preset separator, the sample response text, and the preset end character to obtain the sample spliced text.
[0134] The first training module is configured to train the model by using the sample concatenated text as input to the embedding layer, adding the fifth feature vector corresponding to each character in the sample concatenated text output by the embedding layer to the second target latent vector to obtain multiple sixth feature vectors as input to the decoder, and using the sample response text as the target output of the decoder to obtain the text model. The second target latent vector is the preset parameter vector with the highest similarity to the fifth feature vector corresponding to the preset start character among the multiple preset parameter vectors of the current embedding layer.
[0135] In some embodiments, the text model is trained using a second model training device. The second model training device includes:
[0136] The second sample acquisition module is configured to acquire unsupervised corpora.
[0137] The second training module is configured to perform unsupervised pre-training on the text model using the unsupervised corpus to obtain an initial model.
[0138] The third sample acquisition module is configured to acquire sample text and the corresponding sample response text.
[0139] The second splicing module is configured to sequentially splice the preset start character, the sample text, the preset separator, the sample response text, and the preset end character to obtain the sample spliced text.
[0140] The third training module is configured to train the model by using the sample concatenated text as input to the embedding layer of the initial model, adding the fifth feature vector corresponding to each character in the sample concatenated text output by the embedding layer of the initial model to the second target latent vector to obtain multiple sixth feature vectors as input to the decoder of the initial model, and using the sample response text as the target output of the decoder of the initial model to obtain the text model. The second target latent vector is the preset parameter vector with the highest similarity to the fifth feature vector corresponding to the preset start character among the multiple preset parameter vectors of the current embedding layer.
[0141] Regarding the apparatus in the above embodiments, the specific manner in which each module performs its operation has been described in detail in the embodiments related to the method, and will not be elaborated upon here.
[0142] This disclosure also provides a computer-readable storage medium having stored thereon computer program instructions that, when executed by a processor, implement the steps of the human-computer dialogue method described above.
[0143] This disclosure also provides an electronic device, including:
[0144] processor;
[0145] Memory used to store processor-executable instructions;
[0146] The processor is configured to execute the human-computer dialogue method provided in this disclosure.
[0147] Figure 7 This is a block diagram illustrating a human-computer interaction device 800 according to an exemplary embodiment. For example, device 800 may be a mobile phone, computer, digital broadcasting terminal, messaging device, game console, tablet device, medical device, fitness equipment, personal digital assistant, etc.
[0148] Reference Figure 7 The device 800 may include one or more of the following components: a processing component 802, a memory 804, a power component 806, a multimedia component 808, an audio component 810, an input / output (I / O) interface 812, a sensor component 814, and a communication component 816.
[0149] Processing component 802 typically controls the overall operation of device 800, such as operations associated with display, telephone calls, data communication, camera operation, and recording. Processing component 802 may include one or more processors 820 to execute instructions to complete all or part of the steps of the aforementioned human-computer interaction method. Furthermore, processing component 802 may include one or more modules to facilitate interaction between processing component 802 and other components. For example, processing component 802 may include a multimedia module to facilitate interaction between multimedia component 808 and processing component 802.
[0150] Memory 804 is configured to store various types of data to support the operation of device 800. Examples of such data include instructions for any application or method operating on device 800, contact data, phonebook data, messages, pictures, videos, etc. Memory 804 can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic storage, flash memory, magnetic disk, or optical disk.
[0151] The power supply component 806 provides power to the various components of the device 800. The power supply component 806 may include a power management system, one or more power sources, and other components associated with generating, managing, and distributing power to the device 800.
[0152] Multimedia component 808 includes a screen that provides an output interface between the device 800 and the user. In some embodiments, the screen may include a liquid crystal display (LCD) and a touch panel (TP). If the screen includes a touch panel, the screen may be implemented as a touchscreen to receive input signals from the user. The touch panel includes one or more touch sensors to sense touches, swipes, and gestures on the touch panel. The touch sensors may sense not only the boundaries of the touch or swipe action but also the duration and pressure associated with the touch or swipe operation. In some embodiments, multimedia component 808 includes a front-facing camera and / or a rear-facing camera. When the device 800 is in an operating mode, such as a shooting mode or a video mode, the front-facing camera and / or the rear-facing camera may receive external multimedia data. Each front-facing camera and rear-facing camera may be a fixed optical lens system or have focal length and optical zoom capabilities.
[0153] Audio component 810 is configured to output and / or input audio signals. For example, audio component 810 includes a microphone (MIC) configured to receive external audio signals when device 800 is in an operating mode, such as call mode, recording mode, and voice recognition mode. The received audio signals may be further stored in memory 804 or transmitted via communication component 816. In some embodiments, audio component 810 also includes a speaker for outputting audio signals.
[0154] I / O interface 812 provides an interface between processing component 802 and peripheral interface modules, such as keyboards, click wheels, buttons, etc. These buttons may include, but are not limited to, home buttons, volume buttons, power buttons, and lock buttons.
[0155] Sensor assembly 814 includes one or more sensors for providing status assessments of various aspects of device 800. For example, sensor assembly 814 may detect the on / off state of device 800, the relative positioning of components such as the display and keypad of device 800, changes in the position of device 800 or a component of device 800, the presence or absence of user contact with device 800, the orientation or acceleration / deceleration of device 800, and temperature changes of device 800. Sensor assembly 814 may include a proximity sensor configured to detect the presence of nearby objects without any physical contact. Sensor assembly 814 may also include a light sensor, such as a CMOS or CCD image sensor, for use in imaging applications. In some embodiments, sensor assembly 814 may also include an accelerometer, a gyroscope, a magnetometer, a pressure sensor, or a temperature sensor.
[0156] Communication component 816 is configured to facilitate wired or wireless communication between device 800 and other devices. Device 800 can access wireless networks based on communication standards, such as WiFi, 2G, or 3G, or combinations thereof. In one exemplary embodiment, communication component 816 receives broadcast signals or broadcast-related information from an external broadcast management system via a broadcast channel. In one exemplary embodiment, communication component 816 also includes a near-field communication (NFC) module to facilitate short-range communication. For example, the NFC module may be implemented based on radio frequency identification (RFID) technology, Infrared Data Association (IrDA) technology, ultra-wideband (UWB) technology, Bluetooth (BT) technology, and other technologies.
[0157] In an exemplary embodiment, the device 800 may be implemented by one or more application-specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field-programmable gate arrays (FPGAs), controllers, microcontrollers, microprocessors, or other electronic components to perform the human-machine dialogue method described above.
[0158] In an exemplary embodiment, a non-transitory computer-readable storage medium including instructions is also provided, such as a memory 804 including instructions, which can be executed by a processor 820 of the device 800 to complete the aforementioned human-computer interaction method. For example, the non-transitory computer-readable storage medium may be a ROM, random access memory (RAM), CD-ROM, magnetic tape, floppy disk, and optical data storage device, etc.
[0159] In another exemplary embodiment, a computer program product is also provided, the computer program product comprising a computer program executable by a programmable device, the computer program having a code portion for performing the above-described human-computer dialogue method when executed by the programmable device.
[0160] Figure 8 This is a block diagram illustrating a human-computer interaction device 1900 according to an exemplary embodiment. For example, device 1900 may be provided as a server. (Refer to...) Figure 8 The device 1900 includes a processing component 1922, which further includes one or more processors, and memory resources represented by memory 1932 for storing instructions, such as application programs, that can be executed by the processing component 1922. The application programs stored in memory 1932 may include one or more modules, each corresponding to a set of instructions. Furthermore, the processing component 1922 is configured to execute instructions to perform the aforementioned human-computer interaction method.
[0161] Device 1900 may also include a power supply component 1926 configured to perform power management of device 1900, a wired or wireless network interface 1950 configured to connect device 1900 to a network, and an input / output (I / O) interface 1958. Device 1900 can operate on an operating system, such as Windows Server, stored in memory 1932. TM macOS X TM Unix TM Linux TM FreeBSD TM Or similar.
[0162] Other embodiments of this disclosure will readily occur to those skilled in the art upon consideration of the specification and practice of this disclosure. This application is intended to cover any variations, uses, or adaptations of this disclosure that follow the general principles of this disclosure and include common knowledge or customary techniques in the art not disclosed herein. The specification and examples are to be considered exemplary only, and the true scope and spirit of this disclosure are indicated by the following claims.
[0163] It should be understood that this disclosure is not limited to the precise structures described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from its scope. The scope of this disclosure is limited only by the appended claims.
Claims
1. A human-computer dialogue method, characterized in that, include: Get the original text of the input; A first target hidden vector is randomly selected from multiple candidate hidden vectors, wherein the multiple candidate hidden vectors are multiple preset parameter vectors of a pre-trained text model, and different preset parameter vectors are used to represent the semantic information of different responses; Based on the original text and the first target latent vector, the text model is used to superimpose the first target latent vector onto the first feature vector obtained based on the original text to generate the target response text corresponding to the original text. The text model includes an embedding layer and a decoder; the step of generating the target response text corresponding to the original text by superimposing the first target latent vector onto a first feature vector obtained based on the original text through the text model, based on the original text and the first target latent vector, includes: Add a preset start character and a preset separator to the beginning and end of the original text respectively to obtain the target concatenated text; The target concatenated text is input into the embedding layer to obtain the first feature vector corresponding to each character in the target concatenated text; Each of the first feature vectors is added to the first target latent vector to obtain multiple second feature vectors; Based on the plurality of second feature vectors, the target response text corresponding to the original text is generated through the embedding layer and the decoder.
2. The method according to claim 1, characterized in that, The step of generating the target response text corresponding to the original text based on the plurality of second feature vectors, through the embedding layer and the decoder, includes: The plurality of second feature vectors are input into the decoder to obtain a first predicted character, and the first predicted character is used as the current character; The current character is input into the embedding layer to obtain the third feature vector corresponding to the current character, and the third feature vector is added to the first target latent vector to obtain the fourth feature vector; The plurality of second feature vectors and the fourth feature vector are input into the decoder to obtain a second predicted character, and the second predicted character is used as the new current character, and the plurality of second feature vectors and the fourth feature vector are used as a new plurality of second feature vectors; The process of inputting the current character into the embedding layer to obtain the third feature vector corresponding to the current character, and adding the third feature vector to the first target latent vector to obtain the fourth feature vector, continues until the new current character is a preset end character. Based on the first predicted character and the second predicted character generated each time, the target response text corresponding to the original text is generated.
3. The method according to claim 2, characterized in that, The step of generating the target response text corresponding to the original text based on the first predicted character and the second predicted character generated each time includes: The first predicted character and each of the second predicted characters obtained each time, excluding the preset end character, are concatenated in order of generation time from earliest to latest to obtain the target response text corresponding to the original text.
4. The method according to claim 1, characterized in that, The text model is trained using a quantized vector-variational autoencoder framework.
5. The method according to claim 4, characterized in that, The text model was trained in the following way: Obtain the sample text and the corresponding sample response text; The preset start character, the sample text, the preset separator, the sample response text, and the preset end character are concatenated sequentially to obtain the sample concatenated text; The text model is trained by using the sample concatenated text as input to the embedding layer, adding the fifth feature vector corresponding to each character in the sample concatenated text output by the embedding layer to the second target latent vector to obtain multiple sixth feature vectors as input to the decoder, and using the sample response text as the target output of the decoder. The second target latent vector is the preset parameter vector with the highest similarity to the fifth feature vector corresponding to the preset start character among the multiple preset parameter vectors of the current embedding layer.
6. The method according to claim 4, characterized in that, The text model was trained in the following way: Obtain unsupervised corpus; The text model is pre-trained in unsupervised mode using the unsupervised corpus to obtain an initial model; Obtain the sample text and the corresponding sample response text; The preset start character, the sample text, the preset separator, the sample response text, and the preset end character are concatenated sequentially to obtain the sample concatenated text; The text model is trained by using the sample concatenated text as input to the embedding layer of the initial model, adding the fifth feature vector corresponding to each character in the sample concatenated text output by the embedding layer of the initial model to the second target latent vector to obtain multiple sixth feature vectors, and using the sample response text as the target output of the decoder of the initial model. The second target latent vector is the preset parameter vector with the highest similarity to the fifth feature vector corresponding to the preset start character among the multiple preset parameter vectors of the current embedding layer.
7. A human-computer dialogue device, characterized in that, include: The acquisition module is configured to acquire the raw text of the input. The selection module is configured to randomly select a first target latent vector from multiple candidate latent vectors, wherein the multiple candidate latent vectors are multiple preset parameter vectors of a pre-trained text model, and different preset parameter vectors are used to represent the semantic information of different responses. The generation module is configured to, based on the original text obtained by the acquisition module and the first target latent vector selected by the selection module, superimpose the first target latent vector onto the first feature vector obtained based on the original text through the text model to generate the target response text corresponding to the original text; The text model includes an embedding layer and a decoder; the generation module includes: Add a submodule, configured to add a preset start character and a preset separator to the beginning and end of the original text respectively, to obtain the target concatenated text; The first input submodule is configured to input the target concatenated text into the embedding layer to obtain a first feature vector corresponding to each character in the target concatenated text; The processing submodule is configured to add each of the first feature vectors to the first target latent vector to obtain multiple second feature vectors; The first generation submodule is configured to generate the target response text corresponding to the original text based on the plurality of second feature vectors, through the embedding layer and the decoder.
8. A computer-readable storage medium having computer program instructions stored thereon, characterized in that, When executed by a processor, the program instructions implement the steps of the method described in any one of claims 1 to 6.
9. An electronic device, characterized in that, include: processor; Memory used to store processor-executable instructions; The processor is configured to execute the human-computer dialogue method according to any one of claims 1 to 6.