Sentence generation method and device, and computer readable storage medium

Abstract

Embodiments of the present application disclose a kind of sentence generation method, device and computer readable storage medium, wherein the method comprises: when obtaining target question and answer sentence, the history sentence of the target question and answer sentence is obtained, here, history sentence is the multiple question and answer sentences that are acquired and / or generated before obtaining target question and answer sentence.Target question and answer sentence and history sentence are input into dialogue model, and the multiple candidate response sentences of target question and answer sentence output by dialogue model are acquired.Multiple candidate response sentences are input into sentence coherence classification model, and the coherence score of each candidate response sentence and history sentence is acquired based on sentence coherence classification model.Target response sentence of target question and answer sentence is determined from multiple candidate response sentences based on the coherence score of each candidate response sentence.By using the present application, the generation accuracy of target response sentence of target question and answer sentence can be improved, the coherence of target response sentence is improved, sentence generation efficiency is high, and applicability is strong.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of artificial intelligence, and particularly relates to a sentence generation method and device and a computer readable storage medium. BACKGROUND

[0002] With the development of artificial intelligence (AI) technology and the emergence of various human-computer dialogue products (such as dialogue robots), people's daily life has been greatly enriched. The present application found in the research and practice process that in the prior art, in the use process of various human-computer dialogue products, the context content correlation of human-computer dialogue is weak, the context content logic of human-computer dialogue is poor, and there are usually repeated or contradictory contents in the reply of the context content.

[0003] For example, in the prior art, in the use process of various human-computer dialogue products, there is often a reply of "the weather is very sunny today" in the previous human-computer dialogue content, while the reply to a similar question in the current human-computer dialogue content is "it will rain today, not suitable for going out", the previous and subsequent contents of human-computer dialogue contradict each other, the user experience effect of human-computer dialogue is poor, and the applicability is low. SUMMARY

[0004] The embodiments of the present application provide a sentence generation method, device and computer readable storage medium, which can improve the generation accuracy of the target response sentence of the target question and answer sentence, improve the coherence of the target response sentence, have high sentence generation efficiency and strong applicability.

[0005] In a first aspect, the embodiments of the present application provide a sentence generation method, which comprises:

[0006] When the target question and answer sentence is obtained, a history sentence of the target question and answer sentence is obtained, wherein the history sentence is a plurality of question and answer sentences obtained and / or generated before the target question and answer sentence is obtained;

[0007] The target question and answer sentence and the history sentence are input into a dialogue model, and a plurality of candidate response sentences of the target question and answer sentence output by the dialogue model are obtained;

[0008] The plurality of candidate response sentences are input into a sentence coherence classification model, and a coherence score of each candidate response sentence and the history sentence is obtained based on the sentence coherence classification model;

[0009] A target response sentence of the target question and answer sentence is determined from the plurality of candidate response sentences based on the coherence score of each candidate response sentence.

[0010] In a possible implementation, before the target question-answer sentence and the historical sentence are input into the dialogue model, the method further includes:

[0011] obtaining a plurality of sample question-answer sentences and historical sentences of each sample question-answer sentence;

[0012] determining a baseline dialogue model, the baseline dialogue model having the capability of outputting a response sentence of any input sentence based on the input sentence and the historical sentence of the input sentence;

[0013] determining a search space size of a target beam search algorithm, and training the dialogue model by using the target beam search algorithm with the search space size based on the plurality of sample question-answer sentences, the historical sentences of each sample question-answer sentence, and the baseline dialogue model.

[0014] In a possible implementation, the training of the dialogue model by using the target beam search algorithm with the search space size based on the plurality of sample question-answer sentences, the historical sentences of each sample question-answer sentence, and the baseline dialogue model includes:

[0015] determining an initial dialogue model based on the baseline dialogue model, the initial dialogue model having the same network structure and network parameters as the baseline dialogue model;

[0016] inputting each sample question-answer sentence and the historical sentence thereof into the initial dialogue model, and performing beam search on the initial dialogue model by using the target beam search algorithm to obtain a response sentence of each sample question-answer sentence, wherein the number of response sentences of each sample question-answer sentence is the same as the search space size;

[0017] inputting the response sentences of each sample question-answer sentence into the sentence coherence classification model, and obtaining a coherence score of the response sentence of each sample question-answer sentence based on the sentence coherence classification model;

[0018] performing reinforcement learning training on the initial dialogue model based on the response sentences of each sample question-answer sentence and the coherence scores thereof, and / or the baseline dialogue model to obtain a reinforced dialogue model.

[0019] In a possible implementation, the reinforcement learning training on the initial dialogue model based on the response sentences of each sample question-answer sentence and the coherence scores thereof to obtain the reinforced dialogue model includes:

[0020] The coherence score of the response sentence of each sample question and answer sentence is taken as a reward parameter of reinforcement learning, and reinforcement learning is performed on the initial dialogue model based on the reward parameter and the response sentence of each sample question and answer sentence to adjust the network parameter of the initial dialogue model, so as to obtain the dialogue model after reinforcement.

[0021] In a possible implementation, the initial dialogue model is trained by reinforcement learning based on the response sentence of each sample question and answer sentence and the baseline response sentence of each sample question and answer sentence to obtain the dialogue model after reinforcement, including:

[0022] The baseline response sentence of each sample question and answer sentence output by the baseline dialogue model is obtained by inputting each sample question and answer sentence and the historical sentence of each sample question and answer sentence into the baseline dialogue model.

[0023] The coherence correction value of the response sentence of each sample question and answer sentence is obtained based on the response sentence of each sample question and answer sentence and the baseline response sentence of each sample question and answer sentence.

[0024] The reward parameter of reinforcement learning is obtained based on the coherence correction value of the response sentence of each sample question and answer sentence and the coherence score, and reinforcement learning is performed on the initial dialogue model based on the reward parameter and the response sentence of each sample question and answer sentence to adjust the network parameter of the initial dialogue model, so as to obtain the dialogue model after reinforcement.

[0025] In a possible implementation, the coherence correction value of the response sentence of each sample question and answer sentence is obtained based on the response sentence of each sample question and answer sentence and the baseline response sentence of each sample question and answer sentence, including:

[0026] The first generation probability of any response sentence of each sample question and answer sentence generated by the initial dialogue model is obtained, and the second generation probability of the baseline response sentence of each sample question and answer sentence generated by the baseline dialogue model and the same as the any response sentence is obtained.

[0027] The relative entropy of the initial dialogue model and the baseline dialogue model is obtained based on the first generation probability and the second generation probability, and the relative entropy is determined as the coherence correction value.

[0028] In a second aspect, an embodiment of the present application provides a sentence generation device, which comprises:

[0029] The obtaining module is configured to, when a target question and answer sentence is obtained, obtain a historical sentence of the target question and answer sentence, wherein the historical sentence is a plurality of question and answer sentences obtained and / or generated before the target question and answer sentence is obtained.

[0030] The response sentence generation module is configured to input the target question and answer sentence and the historical sentence obtained by the obtaining module into a dialogue model, and output a plurality of candidate response sentences of the target question and answer sentence based on the dialogue model.

[0031] The coherence scoring module is configured to input the plurality of candidate response sentences output by the response sentence generation module based on the dialogue model into a sentence coherence classification model, and output a coherence score of each candidate response sentence and the historical sentence based on the sentence coherence classification model.

[0032] The response sentence determination module is configured to determine a target response sentence of the target question and answer sentence from the plurality of candidate response sentences based on the coherence score of each candidate response sentence.

[0033] In a third aspect, an embodiment of the present application provides a computer device, which comprises a processor, a memory and a network interface.

[0034] The processor is connected with the memory and the network interface, wherein the network interface is configured to provide a data communication function, the memory is configured to store program code, and the processor is configured to invoke the program code to execute the method in the first aspect of the embodiments of the present application.

[0035] In a fourth aspect, an embodiment of the present application provides a computer readable storage medium, which stores a computer program, and the computer program comprises program instructions, and the program instructions are used to execute the method in the first aspect of the embodiments of the present application when the processor executes the program instructions. BRIEF DESCRIPTION OF DRAWINGS

[0036] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings needed in the embodiment description will be briefly introduced. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can be obtained by those skilled in the art without creative labor.

[0037] Figure 1 FIG. 1 is a system architecture schematic diagram provided by an embodiment of the present application;

[0038] Figure 2 FIG. 2 is a flowchart of a sentence generation method provided by an embodiment of the present application;

[0039] Figure 3 FIG. 3 is a scene schematic diagram of a sentence generation method provided by an embodiment of the present application;

[0040] Figure 4 FIG. 4 is a coherence score generation schematic diagram provided by an embodiment of the present application;

[0041] Figure 5 is a reward parameter generation schematic diagram provided by an embodiment of the present application;

[0042] Figure 6 is a structural schematic diagram of a sentence generation device provided by an embodiment of the present application;

[0043] Figure 7 is a structural schematic diagram of a computer device provided by an embodiment of the present application. DETAILED DESCRIPTION

[0044] The technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all the embodiments of the present application. Based on the embodiments in the present application, all other embodiments obtained by a person of ordinary skill in the art without creative labor fall within the protection scope of the present application.

[0045] Artificial intelligence is to use a digital computer or a machine controlled by a digital computer to simulate, extend and expand human intelligence, perceive the environment, acquire knowledge and use the knowledge to obtain the best results. In other words, artificial intelligence is a comprehensive technology of computer science, which attempts to understand the essence of intelligence and produce a new intelligent machine that can react in a similar way to human intelligence. Artificial intelligence is to study the design principles and implementation methods of various intelligent machines, so that the machines have the functions of perception, reasoning and decision-making.

[0046] Artificial intelligence technology is a comprehensive discipline, involving a wide range of fields, both hardware and software technologies. Artificial intelligence basic technologies generally include technologies such as sensors, special artificial intelligence chips, cloud computing, distributed storage, big data processing technology, operation / interaction system, mechatronics, etc. Artificial intelligence software technology mainly includes computer vision technology, speech processing technology, natural language processing technology, and machine learning / deep learning, etc.

[0047] Natural language processing (Nature Language processing, NLP) is an important direction in the field of computer science and artificial intelligence. It studies various theories and methods that can realize effective communication between people and computers using natural language. Natural language processing is a science that integrates linguistics, computer science and mathematics. Therefore, the research in this field will involve natural language, i.e. the language used in daily life, so it is closely related to the study of linguistics. Natural language processing technology usually includes text processing, semantic understanding, machine translation, robot question and answer, knowledge graph, etc.

[0048] Machine Learning (ML) is a multi-disciplinary subject involving probability theory, statistics, approximation theory, convex analysis, algorithmic complexity theory, etc. It is a specialized study of how computers simulate or implement human learning behavior to acquire new knowledge or skills, reorganize existing knowledge structure, and continuously improve their own performance. Machine learning is the core of artificial intelligence and the fundamental approach to making computers intelligent. Its applications are widespread in various fields of artificial intelligence. Machine learning and deep learning usually include artificial neural networks, belief networks, reinforcement learning, transfer learning, inductive learning, and rule-based learning.

[0049] The scheme provided by the embodiments of the present application relates to natural language processing, machine learning and other technologies in the field of artificial intelligence. The embodiments are specifically described as follows:

[0050] The sentence generation method (or simply the method provided by the embodiments of the present application) provided by the embodiments of the present application is applicable to the development process of various human-computer dialogue technologies or products (for convenience of description, human-computer dialogue products will be taken as examples for illustration, such as dialogue robots). In order to enable the dialogue model deployed in the human-computer dialogue product to achieve better human-computer interaction experience, the coherence between context contents in the human-computer dialogue process can be improved to meet the user's interaction needs. For example, if the human-computer dialogue product continuously produces semantic incoherent replies or the generated reply contents are harsh and difficult to meet the user's interaction needs in the human-computer dialogue process, the interaction round of human-computer interaction will be reduced, and the user's human-computer interaction experience will be affected. The method provided by the embodiments of the present application can enable the dialogue model deployed in the human-computer dialogue product to output response sentences with better coherence and closer real dialogue effect in the human-computer dialogue process. In other words, based on the method provided by the present application, the dialogue model can automatically generate response sentences (such as replies based on user questions, for convenience of description, target response sentences will be taken as examples for illustration) with better coherence and closer real dialogue effect based on the received user sentences (such as user questions, for convenience of description, target question and answer sentences will be taken as examples for illustration), so as to make the human-computer dialogue product obtain better human-computer interaction effect.

[0051] In the method provided in the embodiments of the present application, the continuity in the human-computer conversation process mainly refers to the consistency and relevance of the target response sentence generated by the current human-computer conversation product and the previous historical human-computer conversation content in the interaction theme, logic, etc. For example, when receiving the target question and answer sentence "How is the weather today?" (i.e. the current user wants to obtain information about the weather) from the user, the human-computer conversation product (such as a dialogue robot) can generate the target response sentence "The weather is very sunny today" based on the deployed dialogue model, that is, the information about the weather that the user wants is informed through the target response sentence, and the continuity in the dialogue theme is met. If the target response sentence "There are three new movies released today" is generated based on the above target question and answer sentence, the generated target response sentence (movie related) is irrelevant to the theme of the target question and answer sentence from the user (weather related theme), which cannot meet the user's demand and has poor continuity. Or, after generating the target response sentence "The weather is very sunny today" based on the received target question and answer sentence "How is the weather today?", if the target question and answer sentence "I want to go out for exercise today" is received from the user in the subsequent conversation, the target response sentence "It is not suitable to go out today due to heavy rain" is generated based on the target question and answer sentence, which contradicts the "The weather is very sunny today" in the historical conversation sentence, and will cause confusion to the user's interaction process, cannot meet the user's demand, and has poor continuity. Therefore, training the dialogue model deployed for the human-computer conversation product to generate a target response sentence with better continuity and closer to the real conversation effect based on the received target question and answer sentence in the human-computer conversation process can further improve the human-computer interaction experience, which is simple to operate and has high applicability. Here, the target question and answer sentence can be obtained from the user's question sentence in various forms including text, picture or voice, etc. For example, when the user's question sentence is a picture or includes a picture in the above user's question sentence, the target question and answer sentence corresponding to the user's question sentence can be obtained from the picture based on image text recognition. When the user's question sentence is a voice or includes a voice in the above user's question sentence, the target question and answer sentence corresponding to the user's question sentence can be obtained from the voice based on voice text recognition. The form of the above user's question sentence and the acquisition method of the target question and answer sentence can be determined according to the actual application scenario, which is not limited here.

[0052] In the method provided in the embodiments of the present application, in the generation process of the target response sentence, the dialogue model can obtain a plurality of candidate response sentences of the target question and answer sentence based on the obtained target question and answer sentence and its historical sentence, and the coherence scores of the plurality of candidate response sentences are obtained through the sentence coherence classification model, so that the target response sentence of the target question and answer sentence can be obtained from the plurality of candidate response sentences based on the coherence scores of the plurality of candidate response sentences. Here, the coherence scores of the plurality of candidate response sentences are obtained through the sentence coherence classification model, which can ensure that the target response sentence determined from the plurality of candidate response sentences has coherence with its historical sentence, that is, the historical sentence is consistent and has strong association in terms of dialogue theme, logic and the like, the operation is simple, the generated target response sentence has strong coherence, the human-computer interaction experience effect is good, and the applicability is strong.

[0053] Referring to Figure 1 , Figure 1 is a system architecture schematic diagram provided by the embodiments of the present application. As Figure 1As shown, the system architecture may include a business server 100 and a terminal cluster. The terminal cluster may include terminal devices 200a, 200b, 200c, ..., 200n. The business server 100 may be an independent physical server, a server cluster composed of multiple physical servers, or a distributed system. It may also be a cloud server providing basic cloud computing services such as cloud databases, cloud services, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, CDN, and big data and artificial intelligence platforms. The terminal devices (including terminal devices 200a, 200b, 200c, ..., 200n) may be smartphones, tablets, laptops, desktop computers, PDAs, mobile internet devices (MIDs), wearable devices (such as smartwatches and smart bracelets), smart computers, smart in-vehicle devices, and other smart terminals. The business server 100 can establish communication connections with each terminal device in the terminal cluster, and the terminal devices in the terminal cluster can also establish communication connections with each other. In other words, the service server 100 can establish communication connections with each of the terminal devices 200a, 200b, 200c, ..., 200n. For example, a communication connection can be established between terminal device 200a and the service server 100. A communication connection can be established between terminal device 200a and terminal device 200b, and a communication connection can also be established between terminal device 200a and terminal device 200c. The communication connection method is not limited; it can be established directly or indirectly through wired communication or wireless communication, etc. The specific method can be determined according to the actual application scenario, and this application does not impose any restrictions.

[0054] It should be understood that, such as Figure 1 Each terminal device in the terminal cluster shown can have an application client installed. When the application client runs on each terminal device, it can interact with the aforementioned... Figure 1The business servers 100 shown interact with each other, enabling each business server 100 to receive business data from each terminal device. The application client can be a chatbot application client (hereinafter referred to as a chatbot client), meaning that users can send target question-and-answer statements to the business server 100 through this application client. The business server 100, acting as the server for the chatbot client, can be a collection of multiple servers, including the client's corresponding backend server and data processing server. The business server 100 can receive the target question-and-answer statements from the chatbot client, generate corresponding target response statements based on these statements, and return the generated target response statements to the chatbot client for display to the user. The application client can be a standalone client or an embedded sub-client integrated into a client (such as an instant messaging client, social client, etc.), depending on the actual application scenario, and is not limited here. The method provided in this application embodiment can be performed by, for example... Figure 1 The business server 100 shown can execute the service, or it can be executed by any terminal device (such as...). Figure 1 The terminal device 200a, terminal device 200b, ..., terminal device 200n shown can be used to execute the operation. Alternatively, the terminal device and the business server can be used together to execute the operation. The specific execution method can be determined according to the actual application scenario, and there are no restrictions here.

[0055] In some possible implementation manners, the terminal device 200a can be taken as a provider of a target question and answer sentence, and the service server 100 generates a target response sentence based on the target question and answer sentence and historical sentences of the terminal device 200a. The service server 100 can obtain the target question and answer sentence and historical sentences. The target question and answer sentence can be one or more target question and answer sentences input (or composed and input) by a user through a dialog robot client loaded in the terminal device 200a. The service server 100 can be deployed with a dialog model. The target question and answer sentence can be input into the dialog model, and a plurality of candidate response sentences of the target question and answer sentence can be output by the dialog model, that is, the plurality of candidate response sentences obtained can be selected to obtain the target response sentence. Meanwhile, the service server 100 can be deployed with a sentence coherence classification model. The plurality of candidate response sentences and historical sentences can be input into the sentence coherence classification model, and a coherence score of each candidate response sentence can be obtained by using the sentence coherence classification model. The coherence score reflects the coherence between each response sentence and the historical sentence. The higher the coherence score is, the more consistent and strongly associated the historical sentence is in terms of a dialog theme, logic, and the like. After the coherence score of each candidate response sentence is obtained by using the sentence coherence classification model, each candidate response sentence is sorted, and the candidate response sentence with the highest coherence score is selected as the target response sentence. It can be ensured that the target response sentence determined from the plurality of candidate response sentences has coherence with the historical sentence, the operation is simple, the target response sentence generated has strong coherence, the human-computer interaction experience effect is good, and the applicability is strong.

[0056] In some possible implementation manners, the terminal device 200a can obtain the target question and answer sentence of the user and the historical sentence thereof through the dialogue robot client loaded by the terminal device 200a, and generate a target response sentence based on the target question and answer sentence and the historical sentence thereof. The dialogue model can be deployed in the terminal device 200a (for example, the dialogue robot client loaded by the terminal device 200a), the target question and answer sentence can be input into the dialogue model, and a plurality of candidate response sentences of the target question and answer sentence can be output through the dialogue model, that is, the plurality of candidate response sentences obtained above can be selected to obtain the target response sentence. Meanwhile, the sentence coherence classification model can be deployed in the terminal device 200a (for example, the dialogue robot client loaded by the terminal device 200a), the plurality of candidate response sentences and the historical sentence thereof can be input into the sentence coherence classification model, and the coherence scores of the candidate response sentences can be obtained through the sentence coherence classification model. The coherence score reflects the coherence between the response sentence and the historical sentence, and the higher the coherence score is, the more consistent and strongly associated the historical sentence is in the dialogue theme, logic and the like. After the coherence scores of the candidate response sentences are obtained through the sentence coherence classification model, the candidate response sentences are sorted, and the candidate response sentence with the highest coherence score is selected as the target response sentence, so that the target response sentence determined from the plurality of candidate response sentences has coherence with the historical sentence, the operation is simple, the target response sentence generated has strong coherence, the human-computer interaction experience effect is good, and the applicability is strong.

[0057] For convenience of description, in the following, the terminal device is taken as an execution subject of the method provided by the embodiments of the present application, and the implementation manner of question and answer sentence generation through the terminal device is specifically described through an embodiment.

[0058] Referring to Figure 2 , Figure 2 is a flowchart of the sentence generation method provided by the embodiments of the present application. As Figure 2 indicated, the method includes the following steps:

[0059] S101, when the target question and answer sentence is obtained, the historical sentence of the target question and answer sentence is obtained.

[0060] In some possible implementation manners, the terminal device (such as the terminal device 200a) can obtain a target question and answer sentence, which can be one or more target question and answer sentences input (or composed and input) by a user through a dialogue robot client loaded in the terminal device. The terminal device can obtain the target question and answer sentence through the dialogue robot client. The dialogue robot client can be a standalone client, an embedded sub-client integrated in a certain client (for example, an instant messaging client, a social client, and the like), or a web application accessed through a browser, which can be determined according to an actual application scenario, and is not limited herein. The embodiments of the present application will be described by taking the dialogue robot client as a standalone client, which will not be described herein again.

[0061] Please refer to Figure 3 , Figure 3 is a scenario diagram of a sentence generation method provided by the embodiments of the present application. As shown in Figure 3 , a user can use the dialogue robot client as shown in the interface 100a in Figure 3 . Specifically, the target question and answer sentence adding control 10a and / or the voice input control 10b can be displayed in the interface 100a. If the user wants to send a target question and answer sentence (such as asking the dialogue robot), the user can input the target question and answer sentence in the target question and answer sentence adding control 10a, or the user can click the voice input control 10b to input the target question and answer sentence in the form of voice. For example, it is assumed that the user B continues to compose and input the target question and answer sentence "today's weather is how?" in the target question and answer sentence adding control 10a in the process of dialogue with the dialogue robot A, and clicks the sending control 10c after the target question and answer sentence is input. After the sending control 10c is clicked, the terminal device can respond to the triggering operation on the control 10c and obtain the target question and answer sentence "today's weather is how?" from the dialogue robot client. In addition, the terminal device can obtain the historical sentence corresponding to the target question and answer sentence when the target question and answer sentence is obtained. Here, the historical sentence of the target question and answer sentence is one or more question and answer sentences obtained and / or generated before the target question and answer sentence is obtained. For example, the historical sentence of the target question and answer sentence can be all or part of the dialogue sentences before the target question and answer sentence, as shown in the interface 100a in Figure 3 . At this time, the historical sentence of the target question and answer sentence "today's weather is how?" can be "are you there?" sent by the user B and "yes, do you need help?" fed back by the dialogue robot A.

[0062] S102, input the target question and answer sentence and the historical sentence into the dialogue model, and obtain a plurality of candidate response sentences of the target question and answer sentence output by the dialogue model.

[0063] In some feasible embodiments, the dialogue model can be deployed in the terminal device. After the terminal device obtains the target question and answer sentence and the historical sentence thereof, the terminal device can input the target question and answer sentence and the historical sentence thereof into the dialogue model, obtain a plurality of candidate response sentences of the target question and answer sentence output by the dialogue model, and select the plurality of candidate response sentences to obtain a target response sentence. Specifically, the dialogue model can be a dialogue model constructed based on a Long Short-Term Memory (LSTM) model, a gated recurrent unit (GRU) model, a Bidirectional and Auto-Regressive Transformers (BART) model, etc., which is not limited herein. For convenience of description, the embodiments of the present application take the dialogue model constructed based on the BART model as an example to illustrate the generation of a plurality of candidate response sentences of the target question and answer sentence and the historical sentence thereof obtained by the terminal device. The BART model is a pre-training language model using the overall structure of the Transformer model. Since the BART model uses the context information of the text, i.e., the bidirectional semantic information, in the pre-training encoding process, the fluency and coherence of the candidate response sentence output by the dialogue model can be stronger. For example, after the terminal device obtains the target question and answer sentence "What is the weather like today?" and the historical sentences "Are you there?" and "Yes, what do you need help with?" written by the user B through the dialogue robot client, the terminal device can obtain a plurality of candidate response sentences of the target question and answer sentence through the deployed dialogue model.

[0064] In some possible implementation manners, before the terminal device inputs the target question-answer sentence and the historical sentence into the dialogue model, the terminal device can also obtain a plurality of sample sentences and historical sentences of each sample sentence, and determine a baseline dialogue model, and perform reinforcement learning training based on the plurality of sample sentences and the historical sentences of each sample sentence on the basis of the baseline dialogue model to obtain the dialogue model (i.e., the dialogue model subjected to reinforcement learning training, which can have the same network structure and network parameters as the baseline dialogue model before the reinforcement learning training (for convenience of description, an initial dialogue model can be exemplarily described) can have the same network structure and network parameters as the baseline dialogue model). The baseline dialogue model can be a model constructed based on a BART model and having the capability of outputting a response sentence based on any input sentence and the historical sentence of the sentence. Here, the sample sentences can include sample question-answer sentences and sample response sentences, and the baseline dialogue model (including the initial dialogue model) can be trained based on each sample question-answer sentence and the BART model to enable the baseline dialogue model to output a response sentence based on any sample question-answer sentence. In the training process of the model, the network parameters (such as a loss function) of the model can be adjusted based on the comparison between the sample response sentences (i.e., the true values of the sentences) of each sample question-answer sentence and the response sentences (i.e., the model output values of the sentences) of each sample question-answer sentence to enable the output of the model to converge, to obtain the capability of outputting a response sentence based on any input sample question-answer sentence.

[0065] In some possible implementation manners, the dialogue model deployed in the terminal device can be obtained based on reinforcement learning training of the baseline dialogue model, the plurality of sample sentences, and the historical sentences of each sample sentence. In addition, in the training of the dialogue model, a beam search algorithm can also be used for model training, so that the model obtains the ability to obtain a plurality of candidate response sentences based on any input sentence. At this time, the number of response sentences output by the model based on any input sentence is the same as the search space size of the beam search algorithm used for model training. In the training of the dialogue model, the search space size of the beam search algorithm (which can determine the number of candidate response sentences output by the dialogue model based on each target question and answer sentence input) can be determined, and the dialogue model can be trained by using a target beam search algorithm with the search space size based on the plurality of sample sentences (which can be the sample question and answer sentences in each sample sentence), the historical sentences of each sample sentence, and the baseline dialogue model. In other words, the dialogue model herein is a dialogue model obtained based on reinforcement learning training of the initial dialogue model (i.e., the baseline dialogue model), that is, a reinforced model. Specifically, the dialogue model determined based on the baseline dialogue model is subjected to reinforcement learning training based on the plurality of sample sentences and the historical sentences of each sample sentence, so that the dialogue model further strengthens the coherence between the output sentence and the historical sentence based on the baseline dialogue model to obtain a reinforced dialogue model, so that the target response sentence obtained by the dialogue model has coherence with the historical sentence, that is, the historical sentence is consistent with the historical sentence in terms of dialogue theme, logic, and has a strong correlation, thereby enhancing the human-computer interaction experience effect.

[0066] In some possible implementation manners, the plurality of sample sentences and the historical sentences of each sample sentence can be from a sample sentence library, and each sample sentence can include a sample question and answer sentence and a sample response sentence. For example, the sample question and answer sentence "What should I do if I have nothing to watch recently" and the historical sentences of the sample question and answer sentence (i.e., the historical sentences of the sample sentence) "Are you there?", "Yes, do you need any help?", "Can you recommend some activities for the weekend?", and "You can go to the movies on the weekend" can be included in the sample sentence library. Here, the sample sentences included in the sample sentence library can be directly obtained from the relevant field of the Internet or directly obtained from the target question and answer sentence written by a user (such as written through a dialogue robot client), and the like. The specific application scenario can be determined, which is not limited herein. The obtained sample sentences can be placed in a designated storage space of the terminal device to constitute the sample sentence library, and the sample sentences in the sample sentence library can be periodically added, deleted, or the like to update the sample sentences in the sample sentence library. The frequency of the periodic update can be once a week, once a month, or the like, which can be determined according to the actual application scenario requirement, which is not limited herein.

[0067] In some possible implementations, in the process of obtaining a plurality of candidate response sentences through the dialogue model, the dialogue model (such as an output layer) can select a plurality of candidate response sentences with the best coherence by using a beam search algorithm, and the search space (such as k) of the beam search algorithm is the same as the number of candidate response sentences output by the dialogue model. Before reinforcement learning is performed on the dialogue model determined according to the baseline dialogue model to obtain a reinforced dialogue model, the search space of the beam search algorithm can be determined, and the dialogue model is trained by using a target beam search algorithm with a search space size k, so that the trained dialogue model can generate a target response sentence with a higher coherence score based on any target question and answer sentence and historical sentences. Specifically, different search space sizes can be set, the sample sentences and their historical sentences obtained from the sample sentence library are input into the dialogue model with the beam search algorithm with different search space sizes, and the best search space size (such as k) is determined based on the output response sentences. That is, when the target beam search algorithm with the search space size k is used, the response sentence output by the dialogue model has a higher probability of coherence with its historical sentences.

[0068] In some possible implementation manners, after the search space of the bundle search algorithm is determined, each sample sentence and its historical sentence obtained from the sample sentence library are input into a dialogue model (which can be determined based on a baseline dialogue model), and a target bundle search algorithm is used to perform bundle search based on the dialogue model to obtain a response sentence of each sample question and answer sentence in each sample sentence, where the number of response sentences of any sample question and answer sentence is the same as the size of the determined search space. The response sentences of each sample question and answer sentence and their historical sentences (here, the historical sentences of the response sentences include the corresponding sample question and answer sentences and the historical sentences of the sample question and answer sentences) are input into a sentence coherence classification model, and a coherence score of the response sentence of each sample question and answer sentence is obtained based on the sentence coherence classification model. The coherence score reflects the coherence between each response sentence and its historical sentence, and the higher the coherence score is, the more consistent and strongly associated the historical sentence is with the response sentence in terms of a dialogue theme, logic, and the like. Specifically, the sentence coherence classification model can be constructed based on a Bidirectional Encoder Representations from Transformers (BERT) model or a Robustly optimized BERT pretraining approach (RoBERTa) model, and the specific model can be determined according to actual application scenarios, which is not limited herein. For convenience of description, the RoBERTa model will be taken as an example for description. The RoBERTa model is a reinforced version of the BERT model and a more refined version of the BERT model. The RoBERTa model not only improves the optimization function, but also uses a dynamic mask method to train the model, removes the NSP task in the BERT model, and improves the marginal effect. For details, refer to Figure 4 , Figure 4 is a coherence score generation schematic diagram provided by the present application. As Figure 4As shown, the historical sentence (i.e. the historical sentence of the response sentence, including its corresponding sample question and answer sentence and the historical sentence of the sample question and answer sentence) is input into the dialogue model to obtain a response sentence (which can be multiple response sentences of a search space size), and the response sentence and its historical sentence are input into the sentence coherence classification model (which can be a model constructed based on the RoBERTa model) to obtain the coherence score of each response sentence output by the sentence coherence classification model. Specifically, the response sentence (which can be represented by r) and the historical sentence of the response sentence (including its corresponding sample question and answer sentence and the historical sentence of the sample question and answer sentence, which can be represented by U) can be converted into the format of “[CLS]U[SEP]r[SEP]”, and input into the pre-trained RoBERTa model at the same time. The position corresponding to “[CLS]” is input into the last layer of the hidden layer state of the RoBERTa model with a softmax full connection layer to obtain the coherence score of the response sentence.

[0069] In some possible embodiments, in the process of obtaining the coherence score of the dialogue sentence (such as the response sentence) input into the above-mentioned sentence coherence classification model, the cross-entropy (i.e. the loss function) of the above-mentioned sentence coherence classification model can be obtained based on the coherence score output by the sentence coherence classification model and the pre-determined true value, and the sentence coherence classification model can be continuously optimized based on the above-mentioned cross-entropy to improve the accuracy of the coherence score output by the sentence coherence classification model, so that the dialogue model can be trained based on the output coherence score for reinforcement learning, so that the dialogue model after reinforcement can generate a target response sentence with a higher coherence score based on any target question and answer sentence and the historical sentence.

[0070] In some feasible implementations, after obtaining the coherence scores of each response statement based on the aforementioned statement coherence classification model, the initial dialogue model can be trained using reinforcement learning based on the response statements and their coherence scores of the aforementioned sample question-and-answer statements to obtain a reinforced dialogue model. Reinforcement learning (RL), also known as reward learning, evaluation learning, or enhancement learning, is one of the paradigms and methodologies of machine learning. It is used to describe and solve the problem of an agent (which can be the aforementioned dialogue model) learning strategies to maximize rewards or achieve specific goals during its interaction with the environment. Reinforcement learning is a learning method that uses a "trial and error" approach to interact with the environment. By obtaining the expected value of the cumulative reward (which can be the coherence scores of each candidate response statement) after performing an action (which can be the candidate response statement output by the dialogue model) in the current state, it determines which action will achieve the best effect (obtaining the target response statement with better coherence). Specifically, the coherence score of the response statements in each sample question-and-answer statement can be used as the reward parameter for reinforcement learning (this can be achieved by selecting the response statement with the highest coherence score from multiple response statements). The dialogue model (whose network structure and initial network parameters are the same as the baseline dialogue model, and the dialogue model based on the initial network parameters is also called the initial dialogue model, hereinafter referred to as the dialogue model) performs reinforcement learning based on this reward parameter and the response statements of each sample question-and-answer statement to continuously adjust the model's network parameters until the model's output tends to converge, thus obtaining the reinforced dialogue model. For example, for the dialogue model (which can be represented as...) The sample question and answer statement "What should I do if I don't have any movies I want to watch lately?", along with historical statements "Are you there?", "Yes, is there anything I can help you with?", "Can you recommend some activities for the weekend?", and "I can go to the movies this weekend?", were input into the dialogue model. The dialogue model described above can generate multiple response statements. The response statement with the highest coherence score (e.g., "Then we can go for a walk outside") can be selected, and its coherence score can be used as the reward parameter for reinforcement learning. Sample question-and-answer statements and historical statements can be represented as U = {u1, ..., u...} l-1}={x1,…,x T} represents the set of sample question-and-answer statements and historical statements (which can also be collectively referred to as the historical statements of the response statements), u i Let x represent the i-th round of dialogue in U. j Let $\mathbf{j}$ represent the $j$-th word in $U$. For example, the above sample question-and-answer statements and historical statements can be represented as $U = {u1, ..., u5} = {x1, ..., x$. 28 The response statement can be represented by r (or r = u). l ={y1,…,yT′}, where T' represents the number of words in r, y k represents the kth word in the response sentence r), i.e. input U to the dialogue model to obtain the response sentence r. The coherence score of the response sentence calculated using the coherence classifier can be represented as f(U, r), and the coherence score is used as the reward parameter Reward(U, r) = f(U, r) for reinforcement learning of the dialogue model. During the training process, the dialogue model learns a mapping from a series of environment states (State, which can be the sample question and answer sentences received by the current model and the historical sentences) to actions (Action, which can be the response sentence output by the model) based on the reward parameter as feedback, and the principle of outputting the response sentence is to maximize the probability of future cumulative reward parameters. Thus, the dialogue model after reinforcement can generate a target response sentence with a higher coherence score based on any input target question and answer sentence and historical sentences, so that the human-computer interaction product deployed with the dialogue model can obtain better human-computer interaction effect.

[0071] In some possible implementations, the terminal device can also perform reinforcement learning training on the initial dialogue model based on the response sentences and their coherence scores of the above sample question and answer sentences, and in combination with a baseline dialogue model (which can be represented as P θ ) to obtain the dialogue model after reinforcement. Specifically, after obtaining the coherence score of the response sentence, the relative entropy (also known as Kullback-Leibler divergence) of the initial dialogue model and the above baseline dialogue model can be calculated, and the relative entropy reflects the difference between the initial dialogue model and the baseline dialogue model. The relative entropy is used as a coherence correction value of the response sentence to correct the coherence score of each response sentence to obtain the reward parameter. Since the reward parameter contains the constraint of the relative entropy, the initial dialogue model performs reinforcement learning training based on the reward parameter and the response sentences of the sample question and answer sentences, and the dialogue model after reinforcement can generate a target response sentence with a higher coherence score, while not deviating too much from the baseline dialogue model, thereby ensuring the accuracy of the generated target response sentence. See Figure 5 , Figure 5 is a reward parameter generation schematic diagram provided by an embodiment of the present application. As Figure 5As shown, the sample history statement (i.e., the history statement of the sample response statement, including the sample question and answer statement corresponding to the sample response statement and the history statement of the sample question and answer statement) is input into the initial dialogue model and the baseline dialogue model respectively, the first generation probability of the initial dialogue model outputting a certain response statement of the sample question and answer statement in the sample history statement based on the sample history statement is obtained, and the second generation probability of the baseline dialogue model outputting the same response statement (the response statement is the output of the baseline dialogue model, and is also called the baseline response statement) of the sample question and answer statement based on the sample history statement is obtained. The KL divergence is obtained based on the above first generation probability and the second generation probability, that is, the KL divergence is obtained based on the two probabilities of the initial dialogue model and the baseline dialogue model outputting the same response statement for the same input history statement, and then the reward parameter can be obtained through the KL divergence (i.e., the continuity correction value) and the continuity score of the response statement. For example, input U into the initial dialogue model and the baseline dialogue model P θ , the first generation probability of the initial dialogue model outputting the response statement y t is obtained , and the second generation probability P θ of the baseline dialogue model P t outputting the response statement y θ (y t |U,x 1:t-1 ) is obtained, and the KL divergence is calculated through the following formula (1).

[0072]

[0073] In addition, the terminal device can also obtain the continuity score f(U, r) of the response statement output by the initial dialogue model based on the KL divergence and the above statement continuity classification model to obtain the reward parameter Reward(U, r), wherein:

[0074] Reward(U, r) = f(U, r) - KL(U)

[0075] The reward parameter is used for reinforcement learning training (also called Proximal Policy Optimization (PPO)) of the initial dialogue model, so that the dialogue model after reinforcement can generate a target response statement with a higher continuity score based on any input target question and answer statement and history statement, while not deviating too much from the baseline dialogue model.

[0076] Optionally, the terminal device can also obtain the reward parameter Reward(U, r) based on the adaptive KL divergence:

[0077] Reward(U, r) = f(U, r) - βKL(U)

[0078] wherein β is an adjustable constraint parameter. First, an upper limit value and a lower limit value of the received KL divergence are set. If the KL divergence value KL(U) is too large and exceeds the set upper limit value during the reinforcement learning training, it indicates that the constraint of the KL divergence on the reward parameter is too weak at this time, thereby causing the difference between the initial dialogue model and the baseline dialogue model to be too large, and therefore the constraint parameter β can be moderately increased to improve the constraint of the KL divergence. Otherwise, if the KL divergence value KL(U) is too small and lower than the set lower limit value, it indicates that the constraint of the KL divergence on the reward parameter is too strong at this time, thereby causing the difference between the initial dialogue model and the baseline dialogue model to be too small, and therefore the constraint parameter β can be moderately reduced to reduce the constraint of the KL divergence. By adding the constraint parameter β, the KL divergence can better constrain the reward parameter, thereby performing reinforcement learning training on the initial dialogue model based on the reward parameter, so that the dialogue model after reinforcement can generate a target response sentence with a higher coherence score based on any input target question and answer sentence and historical sentence, and at the same time has a moderate difference with the baseline dialogue model, thereby enhancing the human-computer interaction effect of the human-computer dialogue product deploying the dialogue model.

[0079] S103, input each candidate response sentence into the sentence coherence classification model, and obtain the coherence scores of the plurality of candidate response sentences and the historical sentence based on the sentence coherence classification model.

[0080] S104, determine the target response sentence of the target question and answer sentence from the plurality of candidate response sentences based on the coherence scores of the candidate response sentences.

[0081] In some possible implementation manners, after obtaining the plurality of candidate response sentences through the dialogue model, the plurality of candidate response sentences and the historical sentence input sentence are input into a sentence coherence classification model (which can be a sentence coherence classification model constructed based on a RoBERTa model), and a coherence score of each candidate response sentence is obtained through the sentence coherence classification model. The coherence score reflects the coherence between each response sentence and the historical sentence. The higher the coherence score is, the more consistent and strongly associated the historical sentence is in terms of dialogue theme, logic and the like. Based on the coherence scores of the plurality of candidate response sentences, a target response sentence with the strongest coherence with the historical sentence can be obtained (for example, a candidate response sentence with the highest coherence score is selected), thereby enhancing the human-computer interaction effect of a human-computer dialogue product in which the dialogue model is deployed. For example, the dialogue model generates a plurality of candidate response sentences m1, m2,..., mk based on the input target question and answer sentence (which can be “What is the weather like today?”) and the historical sentence (which can be “Are you there?” and “Yes, what do you need help with?”). The number of the plurality of candidate response sentences is the same as the search space size k of the determined bundle search algorithm. The coherence scores of the candidate response sentences are obtained through the sentence coherence classification model and are sorted, and a candidate response sentence with the highest coherence score (for example, m1, “Today the weather is very sunny”) is selected as the target response sentence.

[0082] In some possible implementation manners, the initial dialogue model is trained through reinforcement learning based on the sample sentences in the sample sentence library, so that the dialogue model further enhances the coherence between the model output sentence and the historical sentence on the basis of the baseline dialogue model to obtain a reinforced dialogue model, so that the target response sentence obtained through the dialogue model has coherence with the historical sentence, that is, the historical sentence is consistent and strongly associated with the historical sentence in terms of dialogue theme, logic and the like, thereby enhancing the human-computer interaction experience effect. Through multiple dialogue tests, it can be obtained that the dialogue robot (for example, the dialogue robot A, which is deployed with the dialogue model) has a self-conversation, and whether the generated response sentence is consistent (that is, coherent) with the historical sentence is evaluated by using an independent dialogue coherence classifier. The evaluation result of the dialogue model output is shown in Table 1.

[0083] Table 1

[0084]

[0085] As shown in Table 1, the comparative models include: single-turn dialogue BART, multi-turn dialogue BART, multi-turn dialogue BART with reinforcement learning training, multi-turn dialogue BART with a search space of 5, multi-turn dialogue BART with a search space of 10, and multi-turn dialogue BART with a search space of 20. It can be seen that the average consistency rates of the above-mentioned models for the first five turns are 75.9%, 78.3%, 99.2%, 90.05%, 94.95%, and 98.69%, respectively, and the average consistency rates for the first ten turns are 59.3%, 54.2%, 84.2%, 77.36%, 87.99%, and 95.41%, respectively. That is, for the dialogue model, the target response sentence output by the dialogue model after reinforcement learning training has stronger coherence (reflected in the consistency rate of each turn of dialogue, the stronger the coherence, the higher the consistency rate), and setting different search spaces of the beam search algorithm also affects the coherence of the output target response sentence.

[0086] In some possible implementations, each sample sentence can include a sample question and answer sentence and a sample response sentence. During the process of generating a response sentence for a sample question and answer sentence input by the dialogue model, the cross-entropy (i.e., the loss function) of the dialogue model can be obtained based on the response sentence output by the dialogue model and the sample response sentence in the sample sentence, and the dialogue model can be continuously optimized based on the cross-entropy to improve the accuracy of the response sentence output by the dialogue model, so that the optimized dialogue model can generate a target response sentence with a higher coherence score based on any target question and answer sentence and historical sentences.

[0087] In some possible implementations, during the reinforcement learning training of the initial dialogue model, safety and legality can also be used as reward parameters for reinforcement learning training, that is, the safety score and the legality score of the response sentence can be obtained through a dialogue safety classifier and a dialogue legality classifier, so that the safety score and the legality score are used as reward parameters for reinforcement learning training of the initial dialogue model. The specific training process refers to the reinforcement learning training based on the coherence score, which will not be repeated here.

[0088] In the embodiment of the present application, the terminal device can obtain a target question and answer sentence and its historical sentence. The target question and answer sentence can be one or more target question and answer sentences written by the user through the dialog robot client loaded in the terminal device. After the terminal device obtains the target question and answer sentence and its historical sentence, the terminal device can input the target question and answer sentence and its historical sentence into a dialog model (which can be a dialog model constructed based on a BART model) to output a plurality of candidate response sentences (the number of candidate response sentences is the search space size of the beam search algorithm) of the target question and answer sentence, that is, the plurality of candidate response sentences obtained can be selected to obtain a target response sentence. After obtaining the plurality of candidate response sentences through the dialog model, the plurality of candidate response sentences and their historical sentences are input into a sentence coherence classification model (which can be a sentence coherence classification model constructed based on a RoBERTa model), and the coherence scores of each candidate response sentence are obtained through the sentence coherence classification model. The coherence score reflects the coherence between each response sentence and its historical sentence. The higher the coherence score, the more consistent and strongly associated the historical sentence is in terms of dialogue theme, logic, etc. After obtaining the coherence scores of each candidate response sentence through the sentence coherence classification model, the candidate response sentences are sorted, and the candidate response sentence with the highest coherence score is selected as the target response sentence. This can ensure that the target response sentence determined from the plurality of candidate response sentences has coherence with its historical sentence, the operation is simple, the generated target response sentence has strong coherence, the human-computer interaction experience effect is good, and the applicability is strong.

[0089] Based on the description of the above-mentioned sentence generation method embodiment, the embodiment of the present application also discloses a sentence generation device. The sentence generation device can be applied to Figures 1 to 5 the sentence generation method of the embodiment shown in the figure to execute the steps in the sentence generation method. Here, the sentence generation device can be the service server or the terminal device in the embodiment shown above, that is, the sentence generation device can be the execution subject of the sentence generation method in the embodiment shown above. Please refer to Figures 1 to 5 , Figures 1 to 5 the embodiment shown above. Please refer to Figure 6 , Figure 6 is a structural schematic diagram of the sentence generation device provided by the embodiment of the present application. In the embodiment of the present application, the device can run the following modules:

[0090] The obtaining module 31 is configured to, when the target question and answer sentence is obtained, obtain the historical sentence of the target question and answer sentence, wherein the historical sentence is a plurality of question and answer sentences obtained and / or generated before the target question and answer sentence is obtained.

[0091] The response sentence generation module 32 is configured to input the target question and answer sentence and the historical sentence obtained by the obtaining module 31 into a dialogue model, and output a plurality of candidate response sentences of the target question and answer sentence based on the dialogue model.

[0092] The coherence scoring module 33 is configured to input the plurality of candidate response sentences output by the response sentence generation module 32 based on the dialogue model into a sentence coherence classification model, and output a coherence score of each candidate response sentence and the historical sentence based on the sentence coherence classification model.

[0093] The response sentence determination module 34 is configured to determine a target response sentence of the target question and answer sentence from the plurality of candidate response sentences based on the coherence score of each candidate response sentence.

[0094] In some possible implementation manners, before the target question and answer sentence and the historical sentence are input into the dialogue model, the response sentence generation module 32 is further configured to:

[0095] obtain a plurality of sample question and answer sentences and historical sentences of each sample question and answer sentence;

[0096] determine a baseline dialogue model, the baseline dialogue model having the capability of outputting a response sentence of any input sentence and a historical sentence of the any input sentence;

[0097] determine a search space size of a beam search algorithm, and train the dialogue model by using a target beam search algorithm with the search space size based on the plurality of sample question and answer sentences, the historical sentences of each sample question and answer sentence, and the baseline dialogue model.

[0098] In some possible implementation manners, the response sentence generation module 32 is further configured to:

[0099] determine an initial dialogue model based on the baseline dialogue model, the initial dialogue model having the same network structure and network parameters as the baseline dialogue model;

[0100] input each sample question and answer sentence and the historical sentence thereof into the initial dialogue model, and perform beam search by using the target beam search algorithm based on the initial dialogue model to obtain a response sentence of each sample question and answer sentence, wherein the number of response sentences of any sample question and answer sentence is the same as the search space size;

[0101] input the response sentences of each sample question and answer sentence into the sentence coherence classification model, and obtain a coherence score of the response sentence of each sample question and answer sentence based on the sentence coherence classification model;

[0102] The initial dialogue model is trained by reinforcement learning based on the response sentences of the sample question and answer sentences and the reward variable to adjust network parameters of the initial dialogue model, so as to obtain the dialogue model after reinforcement.

[0103] In some possible implementation manners, the response sentence generation module 32 is further configured to:

[0104] The coherence scores of the response sentences of the sample question and answer sentences are used as reward variables of reinforcement learning, and the initial dialogue model is trained by reinforcement learning based on the reward variables and the response sentences of the sample question and answer sentences to adjust network parameters of the initial dialogue model, so as to obtain the dialogue model after reinforcement.

[0105] In some possible implementation manners, the response sentence generation module 32 is further configured to:

[0106] The baseline dialogue model is input with the sample question and answer sentences and the historical sentences of the sample question and answer sentences, and baseline response sentences of the sample question and answer sentences output by the baseline dialogue model are obtained;

[0107] Based on the response sentences of the sample question and answer sentences and the baseline response sentences of the sample question and answer sentences, coherence correction values of the response sentences of the sample question and answer sentences are obtained;

[0108] The coherence correction values of the response sentences of the sample question and answer sentences and the coherence scores are used to obtain reward variables of reinforcement learning, and the initial dialogue model is trained by reinforcement learning based on the reward variables and the response sentences of the sample question and answer sentences to adjust network parameters of the initial dialogue model, so as to obtain the dialogue model after reinforcement.

[0109] In some possible implementation manners, the response sentence generation module 32 is further configured to:

[0110] The first generation probability of any response sentence of the sample question and answer sentences generated by the initial dialogue model is obtained, and the second generation probability of the baseline response sentence of the sample question and answer sentences generated by the baseline dialogue model and the same as the any response sentence is obtained;

[0111] The relative entropy of the initial dialogue model and the baseline dialogue model is obtained based on the first generation probability and the second generation probability, and the relative entropy is determined as the coherence correction value.

[0112] In some possible implementation manners, according to the Figure 2 corresponding embodiment, Figure 2 The implementation manners described in steps S101 to S104 in the sentence generation method shown in Figure 6The various modules of the illustrated device are executed. For example, the above... Figure 2 The implementation method described in step S101 of the statement generation method shown can be derived from... Figure 6 The acquisition module 31 in the illustrated device performs the execution; the implementation method described in step S102 can be performed by the response statement generation module 32; the implementation method described in step S103 can be performed by the coherence scoring module 33; and the implementation method described in step S104 can be performed by the response statement determination module 34. The implementation methods performed by the acquisition module 31, response statement generation module 32, coherence scoring module 33, and response statement determination module 34 are as described above. Figure 2 The implementation methods provided for each step in the corresponding embodiments will not be repeated here.

[0113] In this embodiment, the statement generation device can acquire a target question-and-answer statement and its historical statements. The target question-and-answer statement can be one or more target question-and-answer statements written by a user through a chatbot client installed on a terminal device. After acquiring the target question-and-answer statement and its historical statements, the statement generation device can input the target question-and-answer statement and its historical statements into a dialogue model (which can be a dialogue model built based on the BART model). The dialogue model outputs multiple candidate response statements for the target question-and-answer statement (the number of candidate response statements is the size of the search space of the beam search algorithm). The device can then select from these multiple candidate response statements to obtain the target response statement. After acquiring multiple candidate response statements through the dialogue model, the multiple candidate response statements and their historical statements are input into a statement coherence classification model (which can be a statement coherence classification model built based on the RoBERTa model). The statement coherence classification model obtains a coherence score for each candidate response statement. The coherence score reflects the coherence between each response statement and its historical statements. A higher coherence score indicates greater consistency and stronger correlation with the historical statements in terms of dialogue topic and logic. After obtaining the coherence scores of each candidate response statement through the statement coherence classification model, the candidate response statements are sorted, and the candidate response statement with the highest coherence score is selected as the target response statement. This ensures that the target response statement determined from the above multiple candidate response statements has coherence with its historical statements. The operation is simple, the generated target response statement has strong coherence, the human-computer interaction experience is good, and the applicability is strong.

[0114] In the embodiments of this application, the above Figure 6Each of the modules in the apparatus shown can be combined into one or several other modules respectively or in total, or some of the modules can be further split into a plurality of modules with smaller functions to constitute, which can achieve the same operation without affecting the implementation of the technical effects of the embodiments of the present application. The above modules are divided based on logical functions, and in actual application, the function of one module can also be implemented by a plurality of modules, or the functions of a plurality of modules can be implemented by one module. In other feasible implementation manners of the present application, the above apparatus can also include other modules, and in actual application, these functions can also be assisted by other modules, and can be implemented by a plurality of modules in cooperation, which is not limited herein.

[0115] Please refer to Figure 7 , Figure 7 is a structural schematic diagram of a computer device provided by the embodiments of the present application. As Figure 7 shown, the computer device 1000 can be the terminal device in the embodiments corresponding to the above Figures 2-5 . The computer device 1000 can include a processor 1001, a network interface 1004 and a memory 1005, in addition, the computer device 1000 can also include a user interface 1003, and at least one communication bus 1002. Wherein, the communication bus 1002 is used to realize the connection communication between the components. Wherein, the user interface 1003 can include a display screen (Display), a keyboard (Keyboard), and the optional user interface 1003 can also include a standard wired interface, a wireless interface. The network interface 1004 can optionally include a standard wired interface, a wireless interface (such as a WI-FI interface). The memory 1004 can be a high-speed RAM memory, or a non-volatile memory, for example, at least one disk memory. The memory 1005 can also be at least one storage device located away from the aforementioned processor 1001. As Figure 7 shown, the memory 1005 as a computer readable storage medium can include an operating system, a network communication module, a user interface module and a device control application program.

[0116] Wherein, optionally, the user interface 1003 can also include a display screen (Display) and a keyboard (Keyboard). In Figure 7 the computer device 1000 shown, the network interface 1004 can provide network communication function; while the user interface 1003 is mainly used to provide an interface for the user (or developer) to input; and the processor 1001 can be used to call the device control application program stored in the memory 1005 to realize the sentence generation method in the embodiments corresponding to the foregoing Figure 2 .

[0117] It should be understood that the computer device 1000 described in the embodiments of the present application can execute the foregoing Figures 2-6 The description of the sentence generation method in the corresponding embodiments will not be repeated here. In addition, the description of the beneficial effects of using the same method will also not be repeated.

[0118] In addition, it should be pointed out here that the embodiments of the present application also provide a computer readable storage medium, and the aforementioned computer readable storage medium stores the computer program executed by the aforementioned sentence generation device, and the aforementioned computer program includes program instructions, and when the aforementioned processor executes the aforementioned program instructions, the aforementioned computer program can execute the foregoing Figures 2-6 The description of the sentence generation method in the corresponding embodiments will not be repeated here. In addition, the description of the beneficial effects of using the same method will also not be repeated. For technical details not disclosed in the computer readable storage medium embodiments involved in the present application, please refer to the description of the method embodiments of the present application.

[0119] Those of ordinary skill in the art can understand that all or part of the processes in the above-mentioned embodiments can be completed by a computer program instructing related hardware, and the aforementioned program can be stored in a computer readable storage medium. When the program is executed, it can include the processes of the above-mentioned embodiments. Among them, the storage medium can be a magnetic disc, an optical disc, a read-only memory (ROM) or a random access memory (RAM) and the like.

[0120] The above only describes the preferred embodiments of the present application, and of course cannot limit the scope of the rights of the present application, so the equivalent changes made according to the claims of the present application still fall within the scope of the present application.

Claims

1. A sentence generation method characterized by, The method comprises: acquiring a plurality of sample question and answer sentences and historical sentences of each sample question and answer sentence, determining a baseline dialogue model, determining an initial dialogue model based on the baseline dialogue model, the initial dialogue model having the same network structure and network parameters as the baseline dialogue model, the baseline dialogue model having the ability to output a response sentence of any input sentence based on the any sentence and the historical sentence of the any sentence; determining a search space size of a beam search algorithm, inputting the sample question and answer sentences and their historical sentences into the initial dialogue model, and performing beam search based on the initial dialogue model using a target beam search algorithm of the search space size to obtain response sentences of the sample question and answer sentences, wherein the number of response sentences of any sample question and answer sentence is the same as the search space size; inputting the response sentences of the sample question and answer sentences into a sentence coherence classification model, acquiring coherence scores of the response sentences of the sample question and answer sentences based on the sentence coherence classification model, taking the coherence scores of the response sentences of the sample question and answer sentences as reward parameters of reinforcement learning, and adjusting network parameters of the dialogue model based on the reward parameters and the response sentences of the sample question and answer sentences through the initial dialogue model to obtain a reinforced dialogue model; when a target question and answer sentence is acquired, acquiring historical sentences of the target question and answer sentence, wherein the historical sentences are a plurality of question and answer sentences acquired and / or generated before the target question and answer sentence is acquired; inputting the target question and answer sentence and the historical sentences into the dialogue model and acquiring a plurality of candidate response sentences of the target question and answer sentence output by the dialogue model; inputting the plurality of candidate response sentences into the sentence coherence classification model, and acquiring coherence scores of each candidate response sentence and the historical sentences based on the sentence coherence classification model; determining a target response sentence of the target question and answer sentence from the plurality of candidate response sentences based on the coherence scores of the candidate response sentences.

2. A sentence generation method characterized by, The method comprises: acquiring a plurality of sample question and answer sentences and historical sentences of each sample question and answer sentence, determining a baseline dialogue model, determining an initial dialogue model based on the baseline dialogue model, the initial dialogue model having the same network structure and network parameters as the baseline dialogue model, the baseline dialogue model having the ability to output a response sentence of any input sentence based on the any sentence and the historical sentence of the any sentence; determining a search space size of a beam search algorithm, inputting the sample question and answer sentences and their historical sentences into the initial dialogue model, and performing beam search based on the initial dialogue model using a target beam search algorithm of the search space size to obtain response sentences of the sample question and answer sentences, wherein the number of response sentences of any sample question and answer sentence is the same as the search space size; The response sentence of each sample question and answer sentence is input into a sentence coherence classification model, and a coherence score of the response sentence of each sample question and answer sentence is obtained based on the sentence coherence classification model; the sample question and answer sentence and its historical sentence are input into the baseline dialogue model, and a baseline response sentence of the sample question and answer sentence output by the baseline dialogue model is obtained; based on the response sentence of each sample question and answer sentence and the baseline response sentence of each sample question and answer sentence, a coherence correction value of the response sentence of each sample question and answer sentence is obtained; based on the coherence correction value of the response sentence of each sample question and answer sentence and the coherence score, a reward parameter of reinforcement learning is obtained, and reinforcement learning is performed on the initial dialogue model based on the reward parameter and the response sentence of each sample question and answer sentence to adjust the network parameter of the dialogue model, so as to obtain the dialogue model after reinforcement; When the target question and answer sentence is obtained, the historical sentence of the target question and answer sentence is obtained, wherein the historical sentence is a plurality of question and answer sentences obtained and / or generated before the target question and answer sentence is obtained; The target question and answer sentence and the historical sentence are input into the dialogue model, and a plurality of candidate response sentences of the target question and answer sentence output by the dialogue model are obtained; The plurality of candidate response sentences are input into the sentence coherence classification model, and a coherence score of each candidate response sentence and the historical sentence is obtained based on the sentence coherence classification model; Based on the coherence score of each candidate response sentence, a target response sentence of the target question and answer sentence is determined from the plurality of candidate response sentences.

3. The method of claim 2, wherein, The coherence correction value of the response sentence of each sample question and answer sentence based on the response sentence of each sample question and answer sentence and the baseline response sentence of each sample question and answer sentence comprises: A first generation probability of any response sentence of the initial dialogue model generating the sample question and answer sentence is obtained, and a second generation probability of the baseline dialogue model generating the baseline response sentence of the sample question and answer sentence and the baseline response sentence being the same as the any response sentence is obtained; Based on the first generation probability and the second generation probability, a relative entropy of the initial dialogue model and the baseline dialogue model is obtained, and the relative entropy is determined as the coherence correction value.

4. A sentence generation device characterized by comprising: Comprise: The response sentence generation module is configured to obtain a plurality of sample question and answer sentences and historical sentences of each sample question and answer sentence, determine a baseline dialogue model, determine an initial dialogue model based on the baseline dialogue model, the initial dialogue model and the baseline dialogue model have the same network structure and network parameters, and the baseline dialogue model has the ability to output a response sentence of any input sentence based on the any input sentence and the historical sentence of the any input sentence. The response sentence generation module is further configured to determine a search space size of a beam search algorithm, input the sample question and answer sentences and the historical sentences of the sample question and answer sentences into the initial dialogue model, perform beam search based on the initial dialogue model by using a target beam search algorithm with the search space size, and obtain the response sentences of the sample question and answer sentences, wherein the number of response sentences of any sample question and answer sentence is the same as the search space size. The response sentence generation module is further configured to input the response sentences of the sample question and answer sentences into a sentence coherence classification model, and obtain the coherence scores of the response sentences of the sample question and answer sentences based on the sentence coherence classification model. The coherence scores of the response sentences of the sample question and answer sentences are taken as reward parameters of reinforcement learning, and reinforcement learning is performed by the initial dialogue model based on the reward parameters and the response sentences of the sample question and answer sentences to adjust network parameters of the dialogue model, so as to obtain the dialogue model after reinforcement. The obtaining module is configured to, when a target question and answer sentence is obtained, obtain a historical sentence of the target question and answer sentence, wherein the historical sentence is a plurality of question and answer sentences obtained and / or generated before the target question and answer sentence is obtained. The response sentence generation module is further configured to input the target question and answer sentence and the historical sentence obtained by the obtaining module into the dialogue model, and output a plurality of candidate response sentences of the target question and answer sentence based on the dialogue model. The coherence scoring module is configured to input the plurality of candidate response sentences output by the response sentence generation module based on the dialogue model into the sentence coherence classification model, and output the coherence scores of each candidate response sentence and the historical sentence based on the sentence coherence classification model. The response sentence determination module is configured to determine a target response sentence of the target question and answer sentence from the plurality of candidate response sentences based on the coherence scores of the candidate response sentences.

5. A sentence generation apparatus characterized by comprising: The response sentence generation module is configured to obtain a plurality of sample question and answer sentences and historical sentences of each sample question and answer sentence, determine a baseline dialogue model, and determine an initial dialogue model based on the baseline dialogue model, wherein the initial dialogue model has the same network structure and network parameters as the baseline dialogue model, and the baseline dialogue model has the ability to output a response sentence of any input sentence based on the any input sentence and a historical sentence of the any input sentence. The response sentence generation module is further configured to determine a search space size of a beam search algorithm, input the sample question and answer sentences and the historical sentences of the sample question and answer sentences into the initial dialogue model, perform beam search based on the initial dialogue model by using a target beam search algorithm with the search space size, and obtain the response sentences of the sample question and answer sentences, wherein the number of response sentences of any sample question and answer sentence is the same as the search space size. The response sentence generation module is further configured to input the response sentences of the sample question and answer sentences into a sentence coherence classification model, and obtain the coherence scores of the response sentences of the sample question and answer sentences based on the sentence coherence classification model. ​ input the sample question-answer sentences and the historical sentences thereof into the baseline dialogue model to obtain baseline response sentences of the sample question-answer sentences output by the baseline dialogue model; based on the coherence modification values of the response sentences of the sample question-answer sentences and the coherence scores, obtain a reward variable of reinforcement learning, and perform reinforcement learning on the initial dialogue model based on the reward variable and the response sentences of the sample question-answer sentences to adjust network parameters of the dialogue model, to obtain the dialogue model after reinforcement; The obtaining module is configured to, when a target question-answer sentence is obtained, obtain a historical sentence of the target question-answer sentence, wherein the historical sentence is a plurality of question-answer sentences obtained and / or generated before the target question-answer sentence is obtained. The response sentence generation module is further configured to input the target question-answer sentence and the historical sentence obtained by the obtaining module into the dialogue model, and output a plurality of candidate response sentences of the target question-answer sentence based on the dialogue model. The coherence scoring module is configured to input the plurality of candidate response sentences output by the response sentence generation module based on the dialogue model into the sentence coherence classification model, and output a coherence score of each candidate response sentence and the historical sentence based on the sentence coherence classification model. The response sentence determination module is configured to determine a target response sentence of the target question-answer sentence from the plurality of candidate response sentences based on the coherence score of each candidate response sentence.

6. A computer device, comprising: comprising: a processor, a memory, and a network interface; The processor is connected with the memory and the network interface, wherein the network interface is configured to provide a data communication function, the memory is configured to store program code, and the processor is configured to call the program code to execute the method of any one of claims 1-3.

7. A computer readable storage medium characterized in that, The computer-readable storage medium stores a computer program, and the computer program is adapted to be loaded by the processor and execute the method of any one of claims 1-3. The computer-readable storage medium stores a computer program, and the computer program is adapted to be loaded by the processor and execute the method of any one of claims 1-3.

Images