Method, device and computer storage medium for determining reply information

Abstract

The application discloses a method, device and equipment for determining reply information and a computer storage medium. The method comprises the following steps: obtaining question information; inputting the question information into a pre-trained generation model, determining the reply information of the question information through the pre-trained generation model; inputting the reply information into a binary classification discriminator, searching for target question information with a preset similarity to the question information in a corpus in the case that the determination result output by the binary classification discriminator is first indication information; calculating the similarity between the reply information and target reply information corresponding to the target question information; and outputting the reply information with the highest similarity in the case that the similarity is greater than a target value. Thus, the uncontrollability of the reply is improved, and the reliability of the reply is improved.

Description

Technical Field

[0001] This application belongs to the field of artificial intelligence, and in particular relates to a method, apparatus, device and computer storage medium for determining response information. Background Technology

[0002] Existing casual conversation modules typically retrieve responses to questions using either retrieval or generation methods. Retrieval methods generally involve searching an existing corpus database to find questions similar to the user's question and then returning responses to those similar questions. Generation methods typically use generative models (such as GPT3, T5, and Bart) to generate responses to questions. Based on a pre-trained generative model, responses are automatically generated during inference based on the user's input. Responses obtained through retrieving similar questions or generated by generative models are prone to being unintended responses, leading to unpredictability and low reliability. Summary of the Invention

[0003] This application provides a method, apparatus, device, and computer storage medium for determining response information, which can improve the uncontrollability of responses and enhance their reliability.

[0004] In a first aspect, embodiments of this application provide a method for determining response information, the method comprising:

[0005] Obtain problem information;

[0006] The question information is input into a pre-trained generative model, and the pre-trained generative model determines the response information to the question information. The generative model is a model that generates the corresponding response information according to the input question information, and the response information includes at least one item.

[0007] The reply information is input into the binary classification discriminator. If the judgment result output by the binary classification discriminator is the first indication information, the target question information with a preset similarity to the question information is retrieved from the corpus.

[0008] Calculate the similarity between the response information and the target response information corresponding to the target question information;

[0009] If the similarity is greater than the target value, output the response with the highest similarity.

[0010] In one possible implementation embodiment, it further includes:

[0011] If the similarity is not greater than the target value, output the predetermined response information.

[0012] In one possible implementation embodiment, it further includes:

[0013] If the judgment result output by the binary classifier is the second indication information, the predetermined response information is output.

[0014] In one possible implementation, the method further includes, before inputting the problem information into the pre-trained generative model:

[0015] Obtain multi-round corpus information, including question information samples and response information samples;

[0016] Multi-round corpus information is input into the generative model, and the predicted response information is obtained through the generative model;

[0017] A well-trained generative model is obtained when the error between the predicted response and the actual response is within a preset range.

[0018] In one possible implementation, the multi-round corpus information consists of at least two sets of multi-round corpus information, and the method further includes the following steps before inputting the response information into the binary classification discriminator:

[0019] Obtain at least two sets of multi-round corpus information, wherein the corpus information of one round includes a question and a response;

[0020] If the number of rounds exceeds a predetermined threshold, positive samples are constructed using multi-round corpus information from the same group according to a first predetermined rule, and negative samples are constructed using at least two groups of multi-round corpus information according to a second predetermined rule. The initial binary classifier is then trained to obtain the binary classifier. The first predetermined rule is that each sample is supplemented with either a question or a response, and the second predetermined rule is that each sample includes the question and / or response information from the at least two groups of multi-round corpus information.

[0021] In one possible implementation embodiment, it further includes:

[0022] The multi-round corpus information is converted into multiple question-answer pairs;

[0023] The question-answer pairs are saved to a corpus, which contains multiple question-answer pairs. Each question-answer pair includes a question and at least one corresponding response.

[0024] Secondly, embodiments of this application provide a device for determining response information, the device comprising:

[0025] The acquisition module is used to obtain problem information;

[0026] The generation module is used to input question information into a pre-trained generation model and determine the response information for the question information through the pre-trained generation model. The generation model is a model that generates corresponding response information according to the input question information, and the response information includes at least one item.

[0027] The judgment module is used to input the reply information into the binary classification discriminator. If the judgment result output by the binary classification discriminator is the first indication information, the module will search the corpus for target question information that has a preset similarity to the question information.

[0028] The calculation module is used to calculate the similarity between the response information and the target response information corresponding to the target question information;

[0029] The determination module is used to output the response with the highest similarity when the similarity is greater than the target value.

[0030] Thirdly, embodiments of this application provide an electronic device, the device comprising:

[0031] Processor and memory storing computer program instructions;

[0032] The method for determining the response information when the processor executes computer program instructions to implement any of the above.

[0033] Fourthly, embodiments of this application provide a computer storage medium on which computer program instructions are stored, and a method for determining response information that implements any of the above-mentioned items when the computer program instructions are executed by a processor.

[0034] Fifthly, embodiments of this application provide a computer program product, characterized in that, when the instructions in the computer program product are executed by the processor of an electronic device, the electronic device is able to execute the method for determining the response information of any of the above-mentioned items.

[0035] The method, apparatus, device, and computer storage medium for determining response information in this application embodiment acquire question information, input the question information into a pre-trained generative model, and determine the response information of the question information through the pre-trained generative model. The generative model is a model that generates corresponding response information according to the input question information. The response information includes at least one item. The response information is input into a binary classification discriminator. If the judgment result output by the binary classification discriminator is a first indication, target question information with a preset similarity to the question information is retrieved from the corpus. The similarity between the response information and the target response information corresponding to the target question information is calculated. If the similarity is greater than a target value, the response information with the highest similarity is output. In this way, the reasonableness of the response information is judged by the binary classification discriminator. If it is reasonable, target question information similar to the question information is retrieved from the corpus, and the similarity between the response information and the target response information corresponding to the target question information is calculated. The response information with the highest similarity is selected as the final response information. This can improve the uncontrollability of the response information and thus improve the reliability of the response information. Attached Figure Description

[0036] To more clearly illustrate the technical solutions of the embodiments of this application, the accompanying drawings used in the embodiments of this application will be briefly introduced below. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0037] Figure 1 This is a flowchart illustrating a method for determining response information according to an embodiment of this application;

[0038] Figure 2 This is a flowchart illustrating a method for determining response information provided in another embodiment of this application;

[0039] Figure 3 This is a schematic diagram of the structure of a device for determining response information provided in another embodiment of this application;

[0040] Figure 4 This is a schematic diagram of the structure of an electronic device provided in another embodiment of this application. Detailed Implementation

[0041] The features and exemplary embodiments of various aspects of this application will be described in detail below. To make the objectives, technical solutions, and advantages of this application clearer, the application will be further described in detail below with reference to the accompanying drawings and specific embodiments. It should be understood that the specific embodiments described herein are only intended to explain this application and not to limit it. For those skilled in the art, this application can be implemented without some of these specific details. The following description of the embodiments is merely to provide a better understanding of this application by illustrating examples.

[0042] It should be noted that, in this document, relational terms such as "first" and "second" are used merely to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising..." does not exclude the presence of additional identical elements in the process, method, article, or apparatus that includes said element.

[0043] Existing casual conversation modules typically retrieve responses to questions using either retrieval or generation methods. Retrieval methods generally involve searching an existing corpus database to find questions similar to the user's question and then returning responses to those similar questions. Generation methods typically use generative models (such as GPT3, T5, and Bart) to generate responses to questions. Based on a pre-trained generative model, responses are automatically generated during inference based on the user's input. Responses obtained through retrieving similar questions or generated by generative models are prone to being unintended responses, leading to unpredictability and low reliability.

[0044] To address the problems of the prior art, embodiments of this application provide a method, apparatus, device, and computer storage medium for determining response information. The method for determining response information provided in this application embodiment will be described first below.

[0045] Figure 1 A flowchart illustrating a method for determining response information according to an embodiment of this application is shown.

[0046] like Figure 1 As shown, the method for determining response information provided in this application embodiment includes steps S110 to S150.

[0047] S110. Obtain problem information.

[0048] In some embodiments, the problem information can be problem information entered by the user.

[0049] In some embodiments, the problem information can be any of text, audio, and video.

[0050] S120. Input the question information into the pre-trained generative model, and determine the response information of the question information through the pre-trained generative model. The generative model is a model that generates the corresponding response information according to the input question information, and the response information includes at least one item.

[0051] In some embodiments, the generative model can generate response information based on the question information. For example, the generative model can be a generative model such as GPT3, T5, or Bart. Here, the response information is the response information corresponding to the question information, and the response information includes at least one item.

[0052] S130. Input the reply information into the binary classification discriminator. If the judgment result output by the binary classification discriminator is the first indication information, search the corpus for target question information that has a preset similarity to the question information.

[0053] In some embodiments, at least one reply corresponding to the question information is input to a binary classification discriminator, which determines whether the at least one reply is reasonable. If the binary classification discriminator outputs a first indication information as the determination result of the at least one reply, a target question information with a preset similarity to the question information is retrieved from the corpus. The first indication information is information that indicates that the input reply information is a reasonable reply, for example, the first indication information is reasonable.

[0054] In some embodiments, the response information and the corpus from multiple rounds preceding the response information can be concatenated together and input into a binary classification discriminant to determine the reasonableness of the response information.

[0055] In some embodiments, the preset similarity is set by the user in advance. If at least one reply is reasonable, the question information is converted into a sentence vector (for example, the sentence vector can be generated using the penultimate layer of the generative model, which can be a 768-dimensional sentence vector). The target question information with the preset similarity to the sentence vector is retrieved from the corpus. The target question information includes at least one instance. The target reply information corresponding to the target question information is obtained. The target question information corresponds to at least one target reply information. Here, the question information and the reply information corresponding to the question information in the corpus are stored in the form of sentence vectors (for example, the sentence vectors stored in the corpus can also be generated using the penultimate layer of the generative model, which is a 768-dimensional sentence vector).

[0056] As an example, the question information is compared with the question information in the corpus, and the target question information Q that is similar to the question information q is retrieved, where Q = {q1, q2, ... q}. n The size of the set Q is controlled by a preset similarity τ1, and the problem information q is related to q1, q2, ... q. n The similarity values ​​are denoted as μ1, μ2, ..., μ n Here, μ1, μ2…μ n The similarity is greater than the preset similarity τ1.

[0057] In some embodiments, if no target question information with a preset similarity to the question information is retrieved, a predetermined response information is output.

[0058] S140. Calculate the similarity between the response information and the target response information corresponding to the target question information.

[0059] In some embodiments, the response information is converted into sentence vectors, and the similarity between the response information and the target response information corresponding to the target question information in the corpus is calculated. The method of converting the response information into sentence vectors is the same as the method of converting the question information into sentence vectors.

[0060] As an example, the target response information corresponding to the target question information is A, where A = {a 11 ,a 12 ,…a 1k ,a n1 ,a n2 ,…a nm The response information is converted into sentence vectors, and the similarity between the response information and the target response information is calculated. The similarity includes at least μ. 11 μ 12 ,…μ1k,…μ n1 μ n2 ,…μ nm .

[0061] S150. If the similarity is greater than the target value, output the response information with the highest similarity.

[0062] In some embodiments, the similarity scores are sorted, and if the similarity score is greater than the target value, the response with the highest similarity score is output.

[0063] In some embodiments, the similarity between the question information and the target question information is multiplied by the similarity between the response information and the target response information to determine the similarity product. The similarity products are then sorted, and if the similarity product is greater than the target value, the response information with the highest similarity product is output.

[0064] As an example, regarding similarity μ 11 μ12 ,…μ 1k ,…μ n1 μ n2 ,…μ nm Sort the responses and output the responses with the highest similarity if at least one similarity is greater than the target value τ2.

[0065] In another example, calculate μ1, μ2…μ respectively. n and μ 11 μ 12 ,…μ 1k ,…μ n1 μ n2 ,…μ nm The response information with the highest similarity product is output if at least one similarity product is greater than the target value τ2.

[0066] In this way, by using a binary classification discriminator to determine the rationality of the response information, if it is reasonable, the system retrieves target question information that is similar to the question information from the corpus, calculates the similarity between the response information and the target response information corresponding to the target question information, and selects the response information with the highest similarity as the final response information. This can improve the uncontrollability of the response information and thus improve its reliability.

[0067] Based on this, in some embodiments, it may further include:

[0068] If the similarity is not greater than the target value, output the predetermined response information.

[0069] In some embodiments, the predetermined response information is a user-defined response information, which may include, but is not limited to, any one of "error", "unrecognized", "did not understand your meaning", and "I don't quite understand".

[0070] In some embodiments, the similarity scores are sorted, and if the similarity score is not greater than the target value, a predetermined response message is output.

[0071] In some embodiments, the similarity between the question information and the target question information, and the similarity between the response information and the target response information are multiplied to determine the similarity product. The similarity products are then sorted, and if the similarity product is not greater than the target value, the predetermined response information is output.

[0072] In this way, for replies with low similarity, i.e., inaccurate replies, no further replies will be output, reducing the possibility of misleading users.

[0073] Based on this, in some embodiments, it may further include:

[0074] If the judgment result output by the binary classifier is the second indication information, the predetermined response information is output.

[0075] In some embodiments, the second indication information is information that indicates the input response information is unreasonable, for example, the first indication information is unreasonable.

[0076] In this way, unreasonable or inaccurate responses will not be output, reducing the risk of misleading users.

[0077] Based on this, in some embodiments, prior to the above-described S120, the method may further include:

[0078] Obtain multi-round corpus information, including question information samples and response information samples;

[0079] Multi-round corpus information is input into the generative model, and the predicted response information is obtained through the generative model;

[0080] A well-trained generative model is obtained when the error between the predicted response and the actual response is within a preset range.

[0081] The corpus information can be any of the following: text, audio, and video.

[0082] In some embodiments, the generative model may include, but is not limited to, any one of the GPT3 model, T5 model, and Bart model.

[0083] In some embodiments, multi-round corpus information is acquired and input into a generative model. The generative model then obtains predicted response information. Here, the last response information is used as the label. The corpus information from the previous multiple rounds is input into the generative model to predict the last response information and output the predicted response information. The predicted response information is compared with the real response information. The cross-entropy loss function L=-[ylogy′+(1-y)log(1-y′)] is used as the loss function. The final training effect is evaluated through the loss function, where y is the label value of the real response information (positive class value is 1, negative class value is 0), y′ is the probability value of the predicted response information (y′∈(0,1)), and the cross-entropy loss function represents the difference between the label of the real response information and the probability value of the predicted response information.

[0084] As an example, let S be the information from multiple rounds of corpus, where S can include at least two sets of corpus information. i，i≥2 Taking S1 as an example, for the same set of multi-round corpus information S1, the corpus information S from the previous multiple rounds input to the generation model is... 11 S 12 , ...S1q , tagged S 1(q+1) The response information output by the generated model is S′ 1(q+1) , will S′ 1(q+1) and S 1(q+1) Compare them.

[0085] In this way, training the generative model with a large number of samples improves the accuracy of the response information.

[0086] Based on this, in some embodiments, the multi-round corpus information consists of at least two sets of multi-round corpus information. Before S130 above, the method may further include:

[0087] Obtain at least two sets of multi-round corpus information, wherein the corpus information of one round includes a question and a response;

[0088] If the number of rounds exceeds a predetermined threshold, positive samples are constructed using multi-round corpus information from the same group according to a first predetermined rule, and negative samples are constructed using at least two groups of multi-round corpus information according to a second predetermined rule. The initial binary classifier is then trained to obtain the binary classifier. The first predetermined rule is that each sample is supplemented with either a question or a response, and the second predetermined rule is that each sample includes the question and / or response information from the at least two groups of multi-round corpus information.

[0089] As an example, taking S1 as an example, for the same set of multi-round corpus information S1, including S 11 S 12 , ...S 1q S 1(q+1) In multiple rounds, the round number of the multi-round corpus information S1 is (q+1). When (q+1) is greater than a predetermined threshold k, positive samples are constructed according to the first predetermined rule, and the positive samples are S. 11 S 12 …S 1k S 1(k+1) S 11 S 12 …S 1(k+1) S 1(k+2) S 11 S 12 …S 1q S 1(q+1) To construct negative samples, at least two sets of multi-round corpus information are used to construct negative samples according to a second predetermined rule. For example, the negative sample is S. 11 S 12 …S 1k S 2(k+1) S 11 S 12 …S 1u S5n The initial binary classifier is trained using positive and negative samples to obtain the binary classifier.

[0090] In this way, by training the binary classifier with a large number of samples, and using the trained binary classifier to judge the rationality of the response information, the reliability of the response information is improved.

[0091] Based on this, in some embodiments, the method may further include:

[0092] The multi-round corpus information is converted into multiple question-answer pairs;

[0093] The question-answer pairs are saved to a corpus, which contains multiple question-answer pairs. Each question-answer pair includes a question and at least one corresponding response.

[0094] In some embodiments, the acquired multi-round corpus information is converted into multiple question-answer pairs and stored in the corpus in the form of question-answer pairs.

[0095] As an example, taking S1 as an example, if s 1q If it is problem information, then s 1q With s 1(q+1) As a question-and-answer pair, it is saved to the corpus. If s 1q Corresponding to multiple reply messages a 11 ,a 12 ,…a 1j Then combine it with s 1q The questions and answers are saved as question-answer pairs in the corpus, where the question-answer pairs are stored in the corpus in the form of vectors.

[0096] In this way, the question information and the response information corresponding to at least one question information are stored in the corpus in the form of vectors, which makes it easy to retrieve directly based on the question information.

[0097] In the embodiments provided in this application, such as Figure 2As shown, multi-round corpus information is acquired, and a binary classifier is trained using at least two sets of multi-round corpus information to determine whether the generated response information is a reasonable response to the question. The collected multi-round corpus information is converted into question-answer pairs and stored in the corpus for subsequent retrieval. Generative models such as GPT3 are trained using the multi-round corpus information to generate response information. Then, the generated response information is input into the previously trained binary classifier for judging reasonableness. If the response information is judged to be unreasonable, the configured fixed response information is directly returned. If the response information is judged to be reasonable, a corpus retrieval is performed. During the corpus retrieval process, for the question information input by the user, target question information with a similarity higher than a preset similarity in the corpus is recalled. Multiple target response information corresponding to the target question information with high similarity are compared with the response information. If the similarity exceeds the target value, the response information is returned; otherwise, the configured fixed response information is returned. Here, the configured fixed response information is the predetermined response information. The embodiments provided in this application add the retrieval of question information corresponding to reasonable response information on the basis of the generative model, and search in the corpus. In this way, the accuracy of response information is improved while ensuring the diversity of response information.

[0098] Based on the method for determining response information provided in the above embodiments, this application also provides specific implementations of the apparatus for determining response information. Please refer to the following embodiments.

[0099] First see Figure 3 The response information determining device 300 provided in this application embodiment includes:

[0100] Module 310 is used to obtain problem information;

[0101] The generation module 320 is used to input the question information into the pre-trained generation model and determine the response information of the question information through the pre-trained generation model. The generation model is a model that generates the corresponding response information according to the input question information, and the response information includes at least one item.

[0102] The judgment module 330 is used to input the reply information into the binary classification discriminator. When the judgment result output by the binary classification discriminator is the first indication information, the module retrieves target question information with a preset similarity to the question information from the corpus.

[0103] Calculation module 340 is used to calculate the similarity between the response information and the target response information corresponding to the target question information;

[0104] The determination module 350 is used to output the response information with the highest similarity when the similarity is greater than the target value.

[0105] Based on this, in some embodiments, the device 300 may further include:

[0106] The determination module 350 is also used to output predetermined response information when the similarity is not greater than the target value.

[0107] Based on this, in some embodiments, the device 300 may further include:

[0108] The judgment module 330 is also used to output predetermined response information when the judgment result output by the binary classifier is the second indication information.

[0109] Accordingly, in some embodiments, the apparatus may further include:

[0110] The acquisition module 310 is also used to acquire multi-round corpus information before inputting the question information into the pre-trained generative model. The corpus information includes question information samples and response information samples.

[0111] The generation module 320 is also used to input multi-round corpus information into the generation model and obtain predicted response information through the generation model;

[0112] The determination module 350 is also used to obtain the trained generative model when the error between the predicted response information and the actual response information is within a preset range.

[0113] Based on this, in some embodiments, the multi-turn corpus information consists of at least two sets of multi-turn corpus information, and the device 300 may further include:

[0114] The acquisition module 310 is also used to acquire at least two sets of rounds of multi-round corpus information before inputting the reply information into the binary classification discriminator, wherein the corpus information of one round includes a question and a reply.

[0115] The determination module 350 is further configured to construct positive samples using multi-round corpus information from the same group according to a first predetermined rule, and construct negative samples using at least two groups of multi-round corpus information according to a second predetermined rule when the number of rounds exceeds a predetermined threshold, and train the initial binary classifier to obtain the binary classifier. The first predetermined rule is to add a question information followed by a response information or a response information followed by a question information to each sample. The second predetermined rule is that each sample includes the question information and / or response information from the at least two groups of multi-round corpus information.

[0116] Based on this, in some embodiments, the device 300 may further include:

[0117] The conversion module is used to convert multi-round corpus information into multiple question-answer pairs;

[0118] The saving module is used to save question-answer pairs to a corpus. The corpus includes multiple question-answer pairs, where each question-answer pair includes a question and at least one corresponding response.

[0119] The various modules of the response information determination device provided in this application embodiment can achieve... Figure 1 and Figure 2 The system provides functions for each step of the method to determine the response information and achieves the corresponding technical effects. For the sake of brevity, these will not be elaborated here.

[0120] Based on the same inventive concept, embodiments of this application also provide an electronic device.

[0121] Figure 4 A schematic diagram of the hardware structure of the electronic device provided in an embodiment of this application is shown.

[0122] An electronic device may include a processor 401 and a memory 402 storing computer program instructions.

[0123] Specifically, the processor 401 may include a central processing unit (CPU), an application specific integrated circuit (ASIC), or one or more integrated circuits that can be configured to implement the embodiments of this application.

[0124] Memory 402 may include mass storage for data or instructions. For example, and not limitingly, memory 402 may include a hard disk drive (HDD), floppy disk drive, flash memory, optical disk, magneto-optical disk, magnetic tape, or Universal Serial Bus (USB) drive, or a combination of two or more of these. Where appropriate, memory 402 may include removable or non-removable (or fixed) media. Where appropriate, memory 402 may be internal or external to the integrated gateway disaster recovery device. In a particular embodiment, memory 402 is non-volatile solid-state memory.

[0125] Memory may include read-only memory (ROM), random access memory (RAM), disk storage media devices, optical storage media devices, flash memory devices, and electrical, optical, or other physical / tangible memory storage devices. Therefore, typically, memory includes one or more tangible (non-transitory) computer-readable storage media (e.g., memory devices) encoded with software including computer-executable instructions, and when the software is executed (e.g., by one or more processors), it is operable to perform the operations described with reference to the method according to one aspect of this disclosure.

[0126] The processor 401 reads and executes computer program instructions stored in the memory 402 to implement any of the methods for determining response information in the above embodiments.

[0127] In one example, the electronic device may also include a communication interface 403 and a bus 404. For example, Figure 4 As shown, the processor 401, memory 402, and communication interface 403 are connected through bus 404 and complete communication with each other.

[0128] The communication interface 403 is mainly used to realize communication between various modules, devices, units and / or equipment in the embodiments of this application.

[0129] Bus 404 includes hardware, software, or both, that couples components of an electronic device together. For example, and not limitingly, the bus may include an Accelerated Graphics Port (AGP) or other graphics bus, an Extended Industry Standard Architecture (EISA) bus, a Front Side Bus (FSB), a Hyper Transport (HT) interconnect, an Industry Standard Architecture (ISA) bus, an Infinite Bandwidth Interconnect, a Linear Predictive Coding (LPC) bus, a memory bus, a MicroChannel Architecture (MCA) bus, a Peripheral Component Interconnect (PCI) bus, a PCI-Express (Peripheral Component Interconnect-X) bus, a Serial Advanced Technology Attachment (SATA) bus, a Video Electronics Standards Association Local Bus (VESA Local Bus, VLB) bus, or other suitable buses, or a combination of two or more of these. Where appropriate, bus 404 may include one or more buses. Although specific buses are described and illustrated in the embodiments of this application, this application contemplates any suitable bus or interconnection. The electronic device can perform the method for determining response information in the embodiments of this invention, thereby achieving... Figure 1 and Figure 2 The method for determining the described response information.

[0130] Furthermore, in conjunction with the method for determining response information in the above embodiments, this application embodiment can provide a computer storage medium for implementation. This computer storage medium stores computer program instructions; when these computer program instructions are executed by a processor, they implement any of the methods for determining response information in the above embodiments.

[0131] This application also provides a computer program product, wherein the instructions in the computer program product, when executed by the processor of an electronic device, cause the electronic device to perform various processes implementing any of the above-described embodiments of the method for determining response information.

[0132] It should be clarified that this application is not limited to the specific configurations and processes described above and shown in the figures. For the sake of brevity, detailed descriptions of known methods are omitted here. In the above embodiments, several specific steps are described and shown as examples. However, the method process of this application is not limited to the specific steps described and shown. Those skilled in the art can make various changes, modifications, and additions, or change the order of steps, after understanding the spirit of this application.

[0133] The functional blocks shown in the above-described block diagram can be implemented as hardware, software, firmware, or a combination thereof. When implemented in hardware, they can be, for example, electronic circuits, application-specific integrated circuits (ASICs), appropriate firmware, plug-ins, function cards, etc. When implemented in software, the elements of this application are programs or code segments used to perform the required tasks. Programs or code segments can be stored on a machine-readable medium or transmitted over a transmission medium or communication link via data signals carried on a carrier wave. "Machine-readable medium" can include any medium capable of storing or transmitting information. Examples of machine-readable media include electronic circuits, semiconductor memory devices, read-only memory (ROM), flash memory, erasable read-only memory (EROM), floppy disks, compact disc read-only memory (CD-ROM), optical disks, hard disks, fiber optic media, radio frequency (RF) links, etc. Code segments can be downloaded via computer networks such as the Internet, intranets, etc.

[0134] It should also be noted that the exemplary embodiments mentioned in this application describe methods or systems based on a series of steps or apparatus. However, this application is not limited to the order of the above steps; that is, the steps can be performed in the order mentioned in the embodiments, or in a different order, or several steps can be performed simultaneously.

[0135] The aspects of this disclosure have been described above with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this disclosure. It should be understood that each block in the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing apparatus to produce a machine such that these instructions, executable via the processor of the computer or other programmable data processing apparatus, enable the implementation of the functions / actions specified in one or more blocks of the flowchart illustrations and / or block diagrams. Such a processor can be, but is not limited to, a general-purpose processor, a special-purpose processor, a special application processor, or a field-programmable logic circuit. It is also understood that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can also be implemented by special-purpose hardware performing the specified functions or actions, or can be implemented by a combination of special-purpose hardware and computer instructions.

[0136] The above are merely specific embodiments of this application. Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes of the systems, modules, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here. It should be understood that the protection scope of this application is not limited thereto. Any person skilled in the art can easily conceive of various equivalent modifications or substitutions within the technical scope disclosed in this application, and these modifications or substitutions should all be covered within the protection scope of this application.

Claims

1. A method for determining a response message, characterized in that, include: Obtain problem information; The question information is input into a pre-trained generative model, and the response information for the question information is determined by the pre-trained generative model. The generative model is a model that generates corresponding response information according to the input question information, and the response information includes at least one item. The response information and the corpus from the previous rounds are concatenated and input into a binary classification discriminator. If the judgment result output by the binary classification discriminator is the first indication information, a target question information with a preset similarity to the question information is retrieved from the corpus. The target question information includes at least one item. Calculate the similarity between the response information and the target response information corresponding to the at least one target question information, respectively; If the similarity is greater than the target value, output the response information with the highest similarity. Before the input is fed to the binary classifier, the method further includes: Obtain at least two sets of multi-round corpus information. The corpus information of one round includes a question and a response. Positive samples are constructed using multi-round corpus information from the same group according to a first predetermined rule, and negative samples are constructed using at least two groups of multi-round corpus information according to a second predetermined rule. The initial binary classifier is then trained to obtain the binary classifier. The first predetermined rule is that each sample is accompanied by either a question or a response, and the second predetermined rule is that each sample includes the question and / or response information from the at least two groups of multi-round corpus information. When the similarity is greater than the target value, outputting the response information with the highest similarity includes: For each piece of target question information, the similarity between the target question information and the question information is multiplied by the similarity between each target response information and the response information to determine the similarity product. If at least one similarity product is greater than the target value, output the response information corresponding to the highest similarity product among all similarity products.

2. The method for determining response information according to claim 1, characterized in that, Also includes: If the similarity is not greater than the target value, output the predetermined response information.

3. The method for determining response information according to claim 1, characterized in that, Also includes: If the judgment result output by the binary classifier is the second indication information, a predetermined response information is output.

4. The method for determining response information according to claim 1, characterized in that, Before inputting the problem information into the pre-trained generative model, the method further includes: Obtain multi-round corpus information, which includes question information samples and response information samples; The multi-round corpus information is input into the generation model, and the predicted response information is obtained through the generation model; If the error between the predicted response information and the actual response information is within a preset range, a well-trained generative model is obtained.

5. The method for determining response information according to claim 4, characterized in that, The multi-round corpus information comprises at least two sets of multi-round corpus information. Before inputting the response information into the binary classification discriminator, the method further includes: The rounds in which the at least two sets of multi-round corpus information are obtained; If the number of rounds exceeds a predetermined threshold, positive samples are constructed using multi-round corpus information from the same group according to a first predetermined rule, and negative samples are constructed using at least two groups of multi-round corpus information according to a second predetermined rule. The initial binary classifier is then trained to obtain the binary classifier. The first predetermined rule is that each sample is supplemented with either a question or a response, and the second predetermined rule is that each sample includes the question and / or response information from the at least two groups of multi-round corpus information.

6. The method for determining response information according to claim 1, characterized in that, Also includes: The multi-round corpus information is converted into multiple question-answer pairs; The question-answer pairs are saved to the corpus, which includes multiple question-answer pairs, wherein each question-answer pair includes a question and at least one response corresponding to the question.

7. A device for determining response information, characterized in that, The device includes: The acquisition module is used to obtain problem information; A generation module is used to input the question information into a pre-trained generation model, and determine the response information of the question information through the pre-trained generation model. The generation model is a model that generates corresponding response information according to the input question information, and the response information includes at least one item. The judgment module is used to concatenate the response information and the corpus from multiple rounds preceding the response information, input the concatenation into a binary classification discriminator, and, if the judgment result output by the binary classification discriminator is the first indication information, retrieve target question information with a preset similarity to the question information in the corpus, wherein the target question information includes at least one item. The calculation module is used to calculate the similarity between the response information and the target response information corresponding to the at least one target question information, wherein the target question information corresponds to at least one target response information; A determining module is configured to output the response information with the highest similarity when the similarity is greater than a target value; the step of outputting the response information with the highest similarity when the similarity is greater than the target value includes: for each target question information, multiplying the similarity between the target question information and the question information, and the similarity between each target response information and the response information, respectively, to determine a similarity product; and outputting the response information corresponding to the highest similarity product among all similarity products when at least one similarity product is greater than the target value. The acquisition module is also used to acquire at least two sets of multi-round corpus information before the input is sent to the binary classification discriminator. The corpus information of one round includes a question and a response. The determining module is further configured to construct positive samples using multi-round corpus information from the same group according to a first predetermined rule, construct negative samples using at least two groups of multi-round corpus information according to a second predetermined rule, train the initial binary classifier, and obtain the binary classifier. The first predetermined rule is to add a question information followed by a response information or a response information followed by a question information to each sample. The second predetermined rule is that each sample includes the question information and / or response information from the at least two groups of multi-round corpus information.

8. An electronic device, characterized in that, The device includes: a processor and a memory storing computer program instructions; When the processor executes the computer program instructions, it implements the method for determining the response information as described in any one of claims 1-6.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer program instructions, which, when executed by a processor, implement the method for determining response information as described in any one of claims 1-6.

10. A computer program product, characterized in that, When the instructions in the computer program product are executed by the processor of the electronic device, the electronic device is able to perform the method for determining the response information as described in any one of claims 1-6.

Images