Method and apparatus for determining information

Abstract

The present application discloses a method and apparatus for determining information, relating to the field of computer technology. The method includes: obtaining overall context information and a plurality of candidate response information for replying to the overall context information; for each candidate response information among the plurality of candidate response information, determining the total difference information between the candidate response information and other candidate response information except the candidate response information among the plurality of candidate response information; determining the consistency information between the total difference information and the overall context information as the first consistency information between the candidate response information and the overall context information; and determining a target response information from the plurality of candidate response information according to the first consistency information corresponding to each candidate response information. Using this method can improve the accuracy of determining the response information.

Description

Technical Field

[0001] The present disclosure relates to the field of computer technologies, and particularly to a method and an apparatus for determining information. Background Art

[0002] With the development of artificial intelligence technologies, more and more fields adopt artificial intelligence models to implement human-computer dialogue systems. For example, intelligent customer service systems, intelligent chat systems, self-service question-and-answer systems, etc. Existing human-computer dialogue systems usually determine reply / response information based on the semantics of the overall context information in historical dialogues.

[0003] However, the method of determining reply / response information based on the semantics of the overall context information has the problem of inaccurate information determination. Summary of the Invention

[0004] The present disclosure provides a method, an apparatus, an electronic device, and a computer-readable storage medium for determining information.

[0005] According to a first aspect of the present disclosure, there is provided a method for determining information, including: obtaining overall context information and a plurality of candidate reply information for replying to the overall context information; for each candidate reply information among the plurality of candidate reply information, determining total difference information between the candidate reply information and other candidate reply information other than the candidate reply information among the plurality of candidate reply information; determining consistency information between the total difference information and the overall context information as first consistency information between the candidate reply information and the overall context information; and determining a target reply information from the plurality of candidate reply information according to the first consistency information corresponding to each candidate reply information.

[0006] In some embodiments, for each candidate reply information among the plurality of candidate reply information, determining total difference information between the candidate reply information and other candidate reply information other than the candidate reply information among the plurality of candidate reply information includes: obtaining a word vector of each candidate reply information; for each candidate reply information, determining difference information between the candidate reply information and each other candidate reply information among other candidate reply information based on a similarity between the word vector of the candidate reply information and the word vector of each other candidate reply information among other candidate reply information; and determining the difference information between the candidate reply information and all other reply information as the total difference information between the candidate reply information and other candidate reply information.

[0007] In some embodiments, determining the consistency information between the total difference information and the overall previous context information as the first consistency information between the candidate response information and the overall previous context information includes: obtaining the word vectors of the overall previous context information; determining the consistency information between the total difference information and the word vectors of the overall previous context information as the first consistency information between the candidate response information and the overall previous context information.

[0008] In some embodiments, the method includes: determining the consistency information between the total difference information and the target previous context information as the second consistency information between the candidate response information and the target previous context information, where the target previous context information is the information in the overall previous context information that belongs to the same dialogue party as the candidate response information; determining the target response information from multiple candidate response information according to the first consistency information corresponding to each candidate response information, including: determining the target response information from multiple candidate response information according to the first consistency information and the second consistency information corresponding to each candidate response information.

[0009] In some embodiments, the total difference information is determined based on the similarity between the word vectors of the candidate response information and the word vectors of other candidate response information among the multiple candidate response information except the candidate response information itself. Determining the consistency information between the total difference information and the target previous context information as the second consistency information between the candidate response information and the target previous context information includes: obtaining the word vectors of the target previous context information; determining the consistency information between the total difference information and the word vectors of the target previous context information as the second consistency information between the candidate response information and the target previous context information.

[0010] In some embodiments, determining the target response information from multiple candidate response information according to the first consistency information and the second consistency information corresponding to each candidate response information includes: for each candidate response information, determining the score of the candidate response information by using the first consistency information corresponding to the candidate response information, the second consistency information corresponding to the candidate response information, and the third consistency information between the semantics of the candidate response information and the semantics of the overall previous context information; determining the target response information from multiple candidate response information according to the scores of each candidate response information.

[0011] According to a second aspect of the present disclosure, there is provided a device for determining information, including: an acquisition unit configured to acquire overall context information and a plurality of candidate response information for replying to the overall context information; a first determination unit configured to, for each candidate response information among the plurality of candidate response information, determine the total difference information between the candidate response information and other candidate response information other than the candidate response information among the plurality of candidate response information; a second determination unit configured to determine the consistency information between the total difference information and the overall context information as the first consistency information between the candidate response information and the overall context information; a third determination unit configured to determine a target response information from the plurality of candidate response information according to the first consistency information corresponding to each candidate response information.

[0012] In some embodiments, the first determination unit includes: a first acquisition module configured to acquire the word vector of each candidate response information; a first determination module configured to, for each candidate response information, determine the difference information between the candidate response information and each other candidate response information based on the similarity between the word vector of the candidate response information and the word vector of each other candidate response information among the other candidate response information; a second determination module configured to determine the difference information between the candidate response information and all other response information as the total difference information between the candidate response information and the other candidate response information.

[0013] In some embodiments, the second determination unit includes: a second acquisition module configured to acquire the word vector of the overall context information; a third determination module configured to determine the consistency information between the total difference information and the word vector of the overall context information as the first consistency information between the candidate response information and the overall context information.

[0014] In some embodiments, the device includes: a fourth determination unit configured to determine the consistency information between the total difference information and the target context information as the second consistency information between the candidate response information and the target context information, where the target context information is the information in the overall context information that belongs to the same conversation party as the candidate response information; the third determination unit includes: a fourth determination module configured to determine a target response information from the plurality of candidate response information according to the first consistency information and the second consistency information corresponding to each candidate response information.

[0015] In some embodiments, the total difference information is determined based on the similarity between the word vectors of the candidate response information and the word vectors of other candidate response information among the multiple candidate response information except the candidate response information. The fourth determination unit includes: a third acquisition module configured to acquire the word vector of the target previous context information; a fifth determination module configured to determine the consistency information between the total difference information and the word vector of the target previous context information as the second consistency information between the candidate response information and the target previous context information.

[0016] In some embodiments, the fourth determination module includes: a scoring module configured to, for each candidate response information, determine the score of the candidate response information by using the first consistency information corresponding to the candidate response information, the second consistency information corresponding to the candidate response information, and the third consistency information between the semantics of the candidate response information and the semantics of the overall previous context information; a selection module configured to determine the target response information from the multiple candidate response information according to the scores of each candidate response information.

[0017] According to a third aspect of the present disclosure, embodiments of the present disclosure provide an electronic device, including: one or more processors; a storage device for storing one or more programs, and when the one or more programs are executed by the one or more processors, the one or more processors implement the method for determining information provided in the first aspect.

[0018] According to a fourth aspect of the present disclosure, embodiments of the present disclosure provide a computer-readable storage medium, on which a computer program is stored, and when the program is executed by a processor, the method for determining information provided in the first aspect is implemented.

[0019] The method and device for determining information provided by the present disclosure include: acquiring overall previous context information and multiple candidate response information for replying to the overall previous context information; for each candidate response information among the multiple candidate response information, determining the total difference information between the candidate response information and other candidate response information among the multiple candidate response information except the candidate response information; determining the consistency information between the total difference information and the overall previous context information as the first consistency information between the candidate response information and the overall previous context information; and determining the target response information from the multiple candidate response information according to the first consistency information corresponding to each candidate response information, which can improve the accuracy of determining the target response information.

[0020] It should be understood that the content described in this part is not intended to identify the key or important features of the embodiments of the present disclosure, nor is it used to limit the scope of the present disclosure. Other features of the present disclosure will become easily understood through the following description. Description of the Drawings

[0021] The accompanying drawings are used to better understand the present solution and do not limit the present application. Among them:

[0022] Figure 1 is an exemplary system architecture diagram to which the embodiments of the present application can be applied;

[0023] Figure 2 is a flowchart of an embodiment of the method for determining information according to the present application;

[0024] Figure 3 is a flowchart of another embodiment of the method for determining information according to the present application;

[0025] Figure 4 is a flowchart of the application scenario of the method for determining information according to the present application;

[0026] Figure 5 is a schematic diagram of the training device of the model adopted in the application scenario of the method for determining information according to the present application;

[0027] Figure 6 is a schematic structural diagram of an embodiment of the device for determining information according to the present application;

[0028] Figure 7 is a block diagram of an electronic device for implementing the method for determining information in the embodiments of the present application. Detailed implementation manners

[0029] The following describes exemplary embodiments of the present application with reference to the accompanying drawings. Various details of the embodiments of the present application are included to facilitate understanding, and they should be considered merely exemplary. Therefore, those of ordinary skill in the art should recognize that various changes and modifications can be made to the embodiments described herein without departing from the scope and spirit of the present application. Similarly, for the sake of clarity and conciseness, the descriptions of well-known functions and structures are omitted below.

[0030] Figure 1 Illustrates an exemplary system architecture 100 to which the embodiments of the method for determining information or the device for determining information according to the present application can be applied.

[0031] As Figure 1 shown, the system architecture 100 may include terminal devices 101, 102, 103, a network 104, and a server 105. The network 104 is used to provide a medium for communication links between the terminal devices 101, 102, 103 and the server 105. The network 104 may include various connection types, such as wired, wireless communication links, or fiber optic cables, etc.

[0032] Users can use terminal devices 101, 102, and 103 to interact with server 105 via network 104 to receive or send messages, etc. Terminal devices 101, 102, and 103 can be user terminal devices, on which various client applications can be installed, such as image applications, video applications, shopping applications, chat applications, search applications, financial applications, etc.

[0033] Terminal devices 101, 102, and 103 can be various electronic devices with a display screen and supporting receiving server messages, including but not limited to smartphones, tablets, e-book readers, electronic players, laptop computers, and desktop computers, etc.

[0034] Terminal devices 101, 102, and 103 can be hardware or software. When terminal devices 101, 102, and 103 are hardware, they can be various electronic devices. When terminal devices 101, 102, and 103 are software, they can be installed in the above-listed electronic devices. It can be implemented as multiple software or software modules (such as multiple software modules for providing distributed services), or it can be implemented as a single software or software module. No specific limitation is made here.

[0035] After server 105 obtains the overall context information and multiple candidate response information for replying to the overall context information, for each candidate response information among the multiple candidate response information, it determines the total difference information between the candidate response information and the multiple candidate response information, determines the consistency information between the total difference information and the overall context information as the first consistency information between the candidate response information and the overall context information, and determines the target response information from the multiple candidate response information according to the first consistency information corresponding to each candidate response information.

[0036] It should be noted that the method for determining information provided by the embodiments of the present disclosure can be executed by server 105. Correspondingly, the device for determining information can be set in server 105.

[0037] It should be understood that Figure 1 the numbers of terminal devices, networks, and servers in

[0038] Continue to refer to Figure 2 , which shows a flow 200 of an embodiment of the method for determining information according to the present disclosure, including the following steps:

[0039] Step 201, obtain the overall context information and multiple candidate response information for replying to the overall context information.

[0040] In this embodiment, the execution subject of the method for determining information (such as Figure 1 the server 105 shown) may obtain the overall context information and multiple candidate response information for replying to the overall context information. Among them, the overall context information refers to the historical information in the conversation, such as the historical Q&A messages in the intelligent customer service system, the historical conversation records in the intelligent chat system, etc. The response information refers to the response information for replying to the last piece of information or all the information in the overall context information.

[0041] Step 202: For each candidate response information among the multiple candidate response information, determine the total difference information between the candidate response information and the other candidate response information except the candidate response information among the multiple candidate response information.

[0042] In this embodiment, for each candidate response information among the multiple candidate response information, the total difference information between the candidate response information and the other candidate response information except the candidate response information among the multiple candidate response information can be determined. For example, if the candidate response information A is "The weather is sunny today", the candidate response information B is "The weather is sunny today and the humidity is moderate", the candidate response information C is "The weather is sunny today and it is suitable to travel", and the candidate response information D is "The weather is sunny today, it is suitable to travel, and it is suitable to go hiking", then the total difference information between the candidate response information D and the other candidate response information except the candidate response information among the multiple candidate response information is "It is suitable to travel and it is suitable to go hiking".

[0043] Step 203: Determine the consistency information between the total difference information and the overall context information as the first consistency information between the candidate response information and the overall context information.

[0044] In this embodiment, the consistency information between the candidate response information and the overall context information can be determined based on the total difference information between the candidate response information and the other candidate response information except the candidate response information. For the sake of distinction, this consistency information can be called the first consistency information. This consistency information is used to represent the logical consistency degree / similarity degree between the content expressed by the total difference information and the content expressed by the overall context information, or the logical consistency degree / similarity degree between the grammar / semantics of the total difference information and the grammar / semantics of the overall context information, etc.

[0045] Step 204: Determine the target response information from the multiple candidate response information according to the first consistency information corresponding to each candidate response information.

[0046] In this embodiment, the target response information can be determined from multiple candidate response information according to the first consistency information corresponding to each candidate response information, where the first consistency information corresponding to any response information refers to the total difference information between the any candidate response information and other candidate response information, and the consistency information with the overall previous context information.

[0047] The method for determining information provided in this embodiment includes obtaining the overall previous context information and multiple candidate response information for replying to the overall previous context information; for each candidate response information among the multiple candidate response information, determining the total difference information between the candidate response information and other candidate response information other than the candidate response information among the multiple candidate response information; determining the consistency information between the total difference information and the overall previous context information as the first consistency information between the candidate response information and the overall previous context information; and determining the target response information from the multiple candidate response information according to the first consistency information corresponding to each candidate response information, which can enhance the consistency in logic and content between the determined target response information and the overall previous context information in the historical conversation record, thereby improving the accuracy of the pushed information.

[0048] Optionally, for each candidate response information among the multiple candidate response information, determining the total difference information between the candidate response information and other candidate response information other than the candidate response information among the multiple candidate response information includes: obtaining the word vector of each candidate response information; for each candidate response information, determining the difference information between the candidate response information and each other candidate response information based on the similarity between the word vector of the candidate response information and the word vector of each other candidate response information among the other candidate response information; and determining the difference information between the candidate response information and all other response information as the total difference information between the candidate response information and other candidate response information.

[0049] In this embodiment, the word vector of each candidate response information can be obtained based on a pre-trained semantic parsing model, or a pre-trained language representation model, etc., so as to determine the difference information between the candidate response information and each candidate response information based on the similarity between the word vector of the candidate response information and the word vector of each other candidate response information among the multiple candidate responses. The difference information can be the difference content between the semantics expressed by the two pieces of information, or the difference information between the logics shown by the two pieces of information. Then, the difference information between the candidate response information and each other candidate response information is summarized / aggregated to obtain the total difference information between the candidate response information and all other response information, that is, the total difference information between the candidate response information and other candidate response information other than the candidate response information among the multiple candidate response information.

[0050] In this embodiment, the difference information between candidate response messages is determined based on the similarity between the word vectors of the candidate response messages, which can improve the accuracy and efficiency of determining the difference information.

[0051] Optionally, the consistency information between the total difference information and the overall context information is determined as the first consistency information between the candidate response message and the overall context information, including: obtaining the word vector of the overall context information; determining the consistency information between the total difference information and the word vector of the overall context information as the first consistency information between the candidate response message and the overall context information.

[0052] In this embodiment, the word vector of the overall context information can be obtained based on a pre-trained semantic parsing model, or a pre-trained language representation model, etc. Then, based on the word vector of the overall context information and the total difference information between the candidate response message and other candidate response messages, the first consistency information between the candidate response message and the overall context information is determined. It can be understood that since the total difference information is determined based on the difference between the word vectors of the candidate response messages, the data format of the total difference information is in the form of word vectors. The consistency between the word vector of the overall context information and the total difference information can be directly compared, that is, the similarity / identity comparison, to obtain the consistency information between the candidate information and the overall context information.

[0053] In this embodiment, the first consistency information between the candidate response message and the overall context information is determined based on the word vector of the overall context information and the total difference information, which can improve the accuracy and efficiency of determining the first consistency information.

[0054] Continuing to refer to Figure 3 , a flowchart 300 of another embodiment of the method for determining information according to the present disclosure is shown, including the following steps:

[0055] Step 301, obtain the overall context information and multiple candidate response messages for replying to the overall context information.

[0056] Step 302, for each candidate response message among the multiple candidate response messages, determine the total difference information between the candidate response message and other candidate response messages among the multiple candidate response messages except the candidate response message.

[0057] Step 303, determine the consistency information between the total difference information and the overall context information as the first consistency information between the candidate response message and the overall context information.

[0058] In this embodiment, the descriptions of steps 301, 302, and 303 are the same as those of steps 201, 202, and 203, and will not be elaborated here.

[0059] Step 304: Determine the consistency information between the total difference information and the target above-text information as the second consistency information between the candidate response information and the target above-text information, where the target above-text information is the information in the overall above-text information that belongs to the same dialogue party as the candidate response information.

[0060] In this embodiment, the consistency information between the total difference information and the target above-text information can be determined as the consistency information between the candidate response information and the target above-text information. For the sake of distinction, this consistency information can be called the second consistency information. This consistency information is used to represent the logical consistency between the content expressed by the total difference information and the content expressed by the target above-text information, or the logical consistency between the grammar / semantics of the total difference information and the grammar / semantics of the target above-text information, etc. The target above-text information refers to the dialogue record generated by the dialogue party of the candidate response information in the overall above-text information. For example, if the overall above-text information is a dialogue between A and B (where A can be a real user who asks questions and B can be a robot customer service who replies to the user's questions), and if the current dialogue process is in the stage where B needs to reply, that is, when B needs to generate a message currently, then the dialogue party of the candidate response information is B. Therefore, the target above-text information is all the messages from B in the overall above-text information.

[0061] Step 305: Determine the target response information from multiple candidate response information according to the first consistency information and the second consistency information corresponding to each candidate response information.

[0062] In this embodiment, the target response information can be determined from multiple candidate response information according to the first consistency information and the second consistency information corresponding to each candidate response information. Among them, the first consistency information corresponding to any response information refers to the consistency information between the total difference information between this arbitrary candidate response information and other candidate response information and the overall above-text information; the second consistency information corresponding to any response information refers to the consistency information between the total difference information between this arbitrary candidate response information and other candidate response information and the target above-text information.

[0063] The method for determining information provided in this embodiment, compared with Figure 2The method described in the embodiment adds the step of obtaining the second consistency information between the candidate reply information and the target previous context information, and also based on the second consistency information when determining the target reply information, which can enhance the consistency in logic and content between the determined candidate reply information and the target previous context information in the historical conversation record, that is, enhance the consistency of the candidate reply information with the information provided by the dialogue party itself, thereby improving the accuracy of determining the reply information.

[0064] Optionally, the total difference information is determined based on the similarity between the word vector of the candidate reply information and the word vectors of other candidate reply information among the multiple candidate reply information, and the consistency information between the total difference information and the target previous context information is determined as the second consistency information between the candidate reply information and the target previous context information, including: obtaining the word vector of the target previous context information; determining the consistency information between the total difference information and the word vector of the target previous context information as the second consistency information between the candidate reply information and the target previous context information.

[0065] In this embodiment, the word vector of the target previous context information can be obtained based on a pre-trained semantic parsing model, or a pre-trained language representation model, etc. Then, based on the word vector of the target previous context information and the total difference information between the candidate reply information and other candidate reply information, the second consistency information between the candidate reply information and the target previous context information is determined. It can be understood that since the total difference information is determined based on the difference between the word vector of the candidate reply information and the word vectors of other candidate reply information, the data format of the total difference information is in the form of word vectors, and the consistency comparison, that is, the similarity / sameness comparison, can be directly performed between the word vector of the target previous context information and the total difference information to obtain the consistency information between the candidate information and the target previous context information.

[0066] This embodiment determines the second consistency information between the candidate reply information and the overall previous context information based on the word vector of the target previous context information and the total difference information, which can improve the accuracy and efficiency of determining the second consistency information.

[0067] Optionally, according to the first consistency information and the second consistency information corresponding to each candidate reply information, the target reply information is determined from the multiple candidate reply information, including: for each candidate reply information, using the first consistency information corresponding to the candidate reply information, the second consistency information corresponding to the candidate reply information, and the third consistency information between the semantics of the candidate reply information and the semantics of the overall previous context information to determine the score of the candidate reply information; determining the target reply information from the multiple candidate reply information according to the score of each candidate reply information.

[0068] In this embodiment, for each candidate response message, the score of the candidate response message can be determined by using the first consistency information corresponding to the candidate response message, the second consistency information corresponding to the candidate response message, and the third consistency between the semantics of the candidate response message and the semantics of the overall previous context information. Specifically, the score of the candidate response message is made proportional to the semantic relevance of the previous context (the semantic relevance of the previous context refers to the degree of relevance / consistency between the semantics of the candidate response message and the semantics of the overall previous context information), proportional to the first consistency information (the higher the first consistency degree of the candidate response message, the higher the score), and proportional to the second consistency information (the higher the second consistency degree of the candidate response message, the higher the score). After determining the scores of each candidate response message, the candidate response message with the highest score is determined as the target response message.

[0069] In this embodiment, by determining the candidate response message with greater semantic relevance of the previous context, higher first consistency degree, and higher second consistency degree as the target response message, the accuracy of determining the target response message can be improved.

[0070] In some application scenarios, such as Figure 4 shown, the method for determining information includes:

[0071] In the first step, the historical conversation / overall previous context information and multiple candidate response messages for replying to the overall previous context information are input into a BERT (Bidirectional Encoder Representations from Transformer) model to obtain the word vectors of the overall previous context information output by the BERT model and the word vectors of each candidate response message among the multiple candidate response messages.

[0072] Specifically, it includes: obtaining (U; r i ), where U = {u1,..., u N} represents the overall previous context information, represents the i-th previous context sentence, represents the N-th word in the i-th overall previous context sentence, represents the i-th candidate response message, represents the N-th word in the i-th candidate response message. After connecting the overall previous context information U and each candidate response message r i , it is input into a pre-trained BERT model, and the word vectors of the overall previous context information and the word vectors of the candidate response messages output by the BERT model are obtained:

[0073]

[0074] Among them, BERT(·) returns the output of the last layer of the BERT model. <;> represents the concatenation / joining of the two sequences before and after the semicolon, and H U represents the word vector of the overall upstream context information U, represents the candidate response information r i of the word vector. In addition, based on the BERT model, the total vector of the semantic information of the overall upstream context information can also be obtained

[0075] Second step, after obtaining the word vectors of the overall upstream context information and the candidate response information, based on the two, a fine-grained response comparison is performed.

[0076] Specifically, it includes: First, for each candidate response information, calculate the word-level attention between the word vector H U of the overall upstream context information and the word vector of this candidate response information, and obtain the similarity matrix between the two:

[0077]

[0078] Among them,

[0079] Among them, ⊙ represents element-wise multiplication between the two matrices. is the vector representation of the m-th word in r i , is the model parameter. represents the similarity between the m-th word of r i and the n-th word of r j .

[0080] Secondly, based on the above similarity matrix obtain the difference information between r i and r j , and the comparison information between r i and r j is defined as:

[0081]

[0082]

[0083] Among them, represents the similar part between r i and r j , represents the different part between r i and r j , that is, the difference information between r i and r j .

[0084] Finally, after obtaining the difference information between the current r i and each of the multiple candidate response messages r j and synthesizing the above difference information, the r i is obtained as the differential representation from all other candidate response messages i.e., the total difference information:

[0085]

[0086]

[0087]

[0088] wherein, is an intermediate variable of the calculation formula; [·] represents a splicing operation, that is, splicing the sequences in []; σ(·) represents the Dirac δ function; W2, W3, and b2 represent model parameters.

[0089] In the third step, perform the consistency reasoning of the overall context logic: Based on the differential representation of r i from all other candidate response messages determine the consistency information i between the candidate response message r U and the overall context information (specifically, the word vector H (i.e., the first consistency information mentioned above in the context)).

[0090] Specifically, it includes:

[0091]

[0092]

[0093]

[0094] H i_h = Relu(A i_h H U W7)

[0095] E h_i = MaxPolling(H h_i )

[0096] E i_h = MaxPolling(H i_h )

[0097] g hi = σ(E h_i W8 + E i_h W9 + b4)

[0098]

[0099] Among them, W4, W5, W6, W7, W8, W9, and b4 are model parameters; A h_i and A i_h are word-level attention matrices between all the overall previous context information U and the candidate response information r i They focus on different perspectives, that is, the information focused by the attention models used to obtain the two types of data is different. The attention model used to obtain A h_i focuses on the overall previous context information related to the response information, and the attention model used to obtain A i_h focuses on the response information related to the overall previous context information. MaxPooling represents the max pooling operation, SoftMax(·) represents a normalized exponential function, and Relu(·) represents a rectified linear unit function; H h_i is the response-aware previous context representation, and H i_h is the previous context-aware response representation; g hi represents the gate mechanism value obtained by fusing H h_i and H i_h based on the gate mechanism.

[0100] Step 4, enhancing the consistency of the overall information of the dialogue party: Based on the differential representation of r i and all other candidate response information determine the consistency information between the candidate response information r i and the target previous context information (specifically the word vector H S ), where the target previous context information refers to the overall previous context information given by the dialogue party to which the candidate response information belongs (for example, if the candidate response information is the dialogue information that user A needs to conduct, the target previous context information refers to all the information sent by user A in the overall previous context information)

[0101]

[0102]

[0103]

[0104]

[0104] H i_s = Relu(A i_h H U W 13 )

[0105] Among them, A s_i and A i_s are the target previous context information (i.e., the overall previous context information of the dialogue party itself) and the candidate response information r iThe word-level attention matrices between them focus on different perspectives, that is, the information focused by the attention models used to obtain the two types of data is different. To obtain A s_i The attention model used focuses on the information of the target's previous context. To obtain A i_s The attention model used focuses on the response information (reply information) related to the information of the target's previous context; the superscript T represents the transpose of the matrix; H s_i represents the speaker representation perceived by the response, which is the vector representation of the overall previous context information, and this vector representation is related to the response information; H i_s represents the response representation perceived by the speaker, which is the vector representation of the response, and this vector is related to the information of the target's previous context; W 10 、W 11 、W 12 、W 13 are model parameters.

[0106] After that, the candidate reply information r i and the consistency information between the candidate reply information r

[0107] E s_i = MaxPolling(H s_1 )

[0108] E i_s = MaxPolling(H i_s )

[0109] g si = σ(E s_i W 14 + E i_s W 15 + b5)

[0110]

[0111] where W 14 、W 15 、b5、b6 are model parameters; g si represents the gate value obtained by fusing H s_i and H i_s based on the gate mechanism.

[0112] Step 5. First, concatenate the total vector of the semantics of the overall previous context information this candidate reply information r i and the consistency information between this candidate reply information r this candidate reply information r i and the consistency information between the candidate reply information r and the information of the target's previous context to obtain this candidate reply information r iInference information H i :

[0113]

[0114] Secondly, based on the inference information, predict the candidate response information r i The score P(r i |U, R) is

[0115]

[0116] where W 16 and b6 are model parameters, and the set M is the identifier of the candidate response information other than the candidate response information r i among multiple candidate response information; R represents the set of candidate response information.

[0117] In this application scenario, the above model parameters can be obtained based on model training. The system for training the model can be as Figure 5 shown. The steps for training the model include: obtaining sample dialogue data, which includes dialogue history and candidate responses, filtering the sample dialogue data to filter out incomplete data and redundant data, so as to obtain training data; inputting the training data into a knowledge transfer predictor to obtain the word vectors of each piece of training data, inputting the training data processed by the knowledge transfer predictor into a knowledge-aware generator to obtain knowledge-aware information (such as the above first consistency information, second consistency information), and determining the target response information based on the knowledge-aware information. The loss function of this model can be defined as:

[0118]

[0119] where λ is a hyperparameter, θ is all trainable parameters, N is the size of the training data in the dataset, is the actual response information. Optimize the model based on the loss function. When the loss value of the loss function meets the preset threshold, it can be confirmed that the model training is completed, and the model parameters can be obtained based on the trained model.

[0120] In this application scenario, the structure of the system for determining the target response information can include:

[0121] Context encoding module: Given the overall context information and the current candidate response information, use the BERT model to encode the input information, obtain the word-level word vector representation of the given overall context information and the current candidate response information, and obtain the semantic representation of the overall context information and the semantic representation of the candidate response information.

[0122] Fine-grained comparison module: Given the word vectors of the overall context information and the candidate response information, by calculating the similarity matrix of the current candidate response information with other candidates, the total difference information between the current candidate response information and other candidate response information can be obtained. Then, using a bipolar matching mechanism (i.e., the process of obtaining H h_i and H i_h ), the overall context word vectors and the above total difference information are compared in two opposite directions (i.e., calculating the context representation related to the response and the response representation related to the context), obtaining the response-aware context representation and the context-aware response representation, and combining the two through a gate mechanism to obtain the first consistency information between the overall context and the current candidate response information.

[0123] Using another bipolar matching mechanism (i.e., the process of obtaining H s_i and H i_s ), the speaker's context word vectors are compared with the above total difference information to obtain the response-aware speaker context representation and the speaker-aware response representation, and combining the two through a gate mechanism to obtain the second consistency information between the speaker's context and the current candidate response information.

[0124] Prediction module: Given the semantic representations of the overall context information and the current candidate response information, the first consistency information between the overall context information and the candidate response information, and the second consistency information between the speaker's overall context information (i.e., the target context information) and the current candidate response information, by concatenating the three and inputting them into a softmax network, the final score of the candidate response is obtained. The training objective of the model is to maximize the probability value of the correct response.

[0125] In this application scenario, a specific example of the method for determining information is:

[0126] The dialogue history information is "A: It's already half past six. Let's start preparing dinner now. B: But I don't want to cook anymore. I'm so tired of cooking every day. A: Let's go out to eat. There's a new Chinese restaurant on Third Avenue. Xiaoming went there yesterday and said it was very delicious. B: Really? What kind of food do they have? You know, I can't eat spicy food. A: Don't worry. This chef is from Guangdong. I know that Cantonese cuisine is one of your favorite dishes. B: Great. Do you know how to get there? A: I don't know where it is. I only know it's on Third Avenue. Don't worry. I think we can find it. B: But I don't want to walk. It's too hot outside. A: Then how about asking Xiaoming to pick us up? We can invite him to dinner. B: Good idea".

[0127] The candidate response information includes the following options: "1) Since Sichuan cuisine is your favorite dish, why don't we go there after work? 2) Okay, I know where the restaurant is. 3) Okay, let's go! 4) Okay, we can go to eat your favorite Sichuan cuisine after school."

[0128] Taking option 1) as an example, splice the overall above - text information with option 1), use the BERT model to obtain the vector representation of each word, and obtain the vector representation of the same dimension for the entire semantic information.

[0129] The difference between option 1) and option 2) is [Sichuan cuisine is your favorite dish, and we go to eat after work]; the difference between option 1) and option 3) is [Sichuan cuisine is your favorite dish]; the difference between option 1) and option 4) is [go after work, after school]. Then integrate the differential content to obtain the total difference information between option 1) and all other options [Sichuan cuisine is your favorite dish, and we go to eat after work]. Calculate the matching degree between this total difference information and the overall above - text information. It is found that Sichuan cuisine does not match Guangdong cuisine, and there is no word for "after work" in the text. Therefore, the score of option 1) in terms of overall above - text consistency will be relatively low (the degree of difference in consistency included in the first consistency information). Finally, calculate the matching degree between the differential content and the historical above - text of speaker A. It is found that speaker A has said information such as the other person's favorite Guangdong cuisine, etc. Therefore, it will strengthen the comparison between such information and the differential content. It is found that Sichuan cuisine does not match Guangdong cuisine. Therefore, the score of option 1) in terms of speaker's above - text consistency is also relatively low (the degree of difference in consistency included in the second consistency information).

[0130] Similarly, compare the differential content of option 2) with options 1), 3), and 4), and calculate the matching score. Since "I know where the restaurant is" contradicts what A has said before "don't know where but believe can find", the matching degree score is relatively low (the difference in consistency between the semantics of the option and the semantics of the overall above - text). Compare option 3) with options 1), 2), and 4). Since there is no contradictory information, the relative score is relatively high. Compare option 4) with options 1), 2), and 3), and calculate the matching score. Since "favorite Sichuan cuisine" contradicts what A has said before "your favorite Guangdong cuisine", the matching score is relatively low. Finally, sort the scores of the four options to determine that option 3) is the target response information.

[0131] Further reference Figure 6 , as an implementation of the methods shown in the above - mentioned figures, the present disclosure provides an embodiment of a device for determining information. This device embodiment corresponds to Figure 2 and Figure 3 the method embodiments shown, and this device can be specifically applied to various electronic devices.

[0132] AsFigure 6 As shown in Figure 6 , the apparatus for determining information in this embodiment includes: an acquisition unit 601, a first determination unit 602, a second determination unit 603, and a third determination unit 604. Among them, the acquisition unit is configured to acquire the overall context information and multiple candidate response information for replying to the overall context information; the first determination unit is configured to determine, for each candidate response information among the multiple candidate response information, the total difference information between the candidate response information and the other candidate response information except the candidate response information among the multiple candidate response information; the second determination unit is configured to determine the consistency information between the total difference information and the overall context information as the first consistency information between the candidate response information and the overall context information; the third determination unit is configured to determine the target response information from the multiple candidate response information according to the first consistency information corresponding to each candidate response information.

[0133] In some embodiments, the first determination unit includes: a first acquisition module configured to acquire the word vector of each candidate response information; a first determination module configured to determine, for each candidate response information, the difference information between the candidate response information and each other candidate response information based on the similarity between the word vector of the candidate response information and the word vector of each other candidate response information among the other candidate response information; a second determination module configured to determine the difference information between the candidate response information and all other response information as the total difference information between the candidate response information and the other candidate response information.

[0134] In some embodiments, the second determination unit includes: a second acquisition module configured to acquire the word vector of the overall context information; a third determination module configured to determine the consistency information between the total difference information and the word vector of the overall context information as the first consistency information between the candidate response information and the overall context information.

[0135] In some embodiments, the apparatus includes: a fourth determination unit configured to determine the consistency information between the total difference information and the target context information as the second consistency information between the candidate response information and the target context information, where the target context information is the information in the overall context information that belongs to the same conversation party as the candidate response information; the third determination unit includes: a fourth determination module configured to determine the target response information from the multiple candidate response information according to the first consistency information and the second consistency information corresponding to each candidate response information.

[0136] In some embodiments, the total difference information is determined based on the similarity between the word vectors of the candidate response information and the word vectors of other candidate response information among the multiple candidate response information except the candidate response information. The fourth determination unit includes: a third acquisition module configured to acquire the word vector of the target context information; a fifth determination module configured to determine the consistency information between the total difference information and the word vector of the target context information as the second consistency information between the candidate response information and the target context information.

[0137] In some embodiments, the fourth determination module includes: a scoring module configured to, for each candidate response information, determine the score of the candidate response information by using the first consistency information corresponding to the candidate response information, the second consistency information corresponding to the candidate response information, and the third consistency information between the semantics of the candidate response information and the semantics of the overall context information; a selection module configured to determine the target response information from the multiple candidate response information according to the scores of each candidate response information.

[0138] Each unit in the above device 600 corresponds to the steps in the method described with reference Figure 2 and Figure 3 Therefore, the operations, features, and achievable technical effects described above for the method for determining information also apply to the device 600 and the units included therein, and will not be elaborated herein.

[0139] According to an embodiment of the present application, the present application also provides an electronic device and a readable storage medium.

[0140] As Figure 7 shown, it is a block diagram of an electronic device 700 for a method of determining information according to an embodiment of the present application. The electronic device is intended to represent various forms of digital computers, such as, a laptop computer, a desktop computer, a workbench, a personal digital assistant, a server, a blade server, a mainframe computer, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as, a personal digital processor, a cellular phone, a smart phone, a wearable device, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely illustrative and are not intended to limit the implementation of the present application described and / or claimed herein.

[0141] As Figure 7As shown, the electronic device includes: one or more processors 701, a memory 702, and interfaces for connecting the components, including a high-speed interface and a low-speed interface. Each component is interconnected using different buses and can be mounted on a common motherboard or otherwise mounted as required. The processor can process instructions executed within the electronic device, including instructions stored in the memory or on the memory to display graphical information of a GUI on an external input / output device (such as a display device coupled to the interface). In other embodiments, multiple processors and / or multiple buses can be used together with multiple memories and multiple memories if needed. Similarly, multiple electronic devices can be connected, with each device providing part of the necessary operations (such as an array of servers, a set of blade servers, or a multi-processor system). Figure 7 In the figure, a processor 701 is taken as an example.

[0142] The memory 702 is the non-transitory computer-readable storage medium provided in this application. Among them, the memory stores instructions executable by at least one processor, so that the at least one processor executes the method for determining information provided in this application. The non-transitory computer-readable storage medium of this application stores computer instructions, and the computer instructions are used to cause the computer to execute the method for determining information provided in this application.

[0143] The memory 702, as a non-transitory computer-readable storage medium, can be used to store non-transitory software programs, non-transitory computer-executable programs, and modules, such as the program instructions / modules corresponding to the method for determining information in the embodiments of this application (for example, the acquisition unit 601, the first determination unit 602, the second determination unit 603, and the third determination unit 604 shown in the appendix). The processor 701 executes various functional applications and data processing of the server by running the non-transitory software programs, instructions, and modules stored in the memory 702, that is, implements the method for determining information in the above method embodiments. Figure 6 As shown, the acquisition unit 601, the first determination unit 602, the second determination unit 603, and the third determination unit 604). The processor 701 executes various functional applications and data processing of the server by running the non-transitory software programs, instructions, and modules stored in the memory 702, that is, implements the method for determining information in the above method embodiments.

[0144] The memory 702 can include a program storage area and a data storage area. Among them, the program storage area can store an operating system and application programs required for at least one function; the data storage area can store data created according to the use of the electronic device for extracting video segments, etc. In addition, the memory 702 can include high-speed random access memory and can also include non-transitory memory, such as at least one magnetic disk storage device, a flash memory device, or other non-transitory solid-state storage devices. In some embodiments, the memory 702 can optionally include a memory remotely set relative to the processor 701, and these remote memories can be connected to the electronic device for extracting video segments through a network. Examples of the above network include but are not limited to the Internet, an enterprise intranet, a local area network, a mobile communication network, and combinations thereof.

[0145] The electronic device for the method of determining information may further include: an input device 703, an output device 704, and a bus 705. The processor 701, the memory 702, the input device 703, and the output device 704 may be connected through the bus 705 or other means. Figure 7 Taking the connection through the bus 705 as an example.

[0146] The input device 703 can receive input digital or character information, and generate key signal inputs related to the user settings and function controls of the electronic device for extracting video segments, such as input devices like touchscreens, keypads, mice, trackpads, touchpads, pointing sticks, one or more mouse buttons, trackballs, joysticks, etc. The output device 704 may include a display device, an auxiliary lighting device (e.g., LED), and a tactile feedback device (e.g., a vibration motor), etc. The display device may include, but is not limited to, a liquid crystal display (LCD), a light emitting diode (LED) display, and a plasma display. In some embodiments, the display device may be a touchscreen.

[0147] Various embodiments of the systems and techniques described herein can be implemented in digital electronic circuit systems, integrated circuit systems, dedicated ASICs (application specific integrated circuits), computer hardware, firmware, software, and / or combinations thereof. These various embodiments may include: implemented in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which can be a dedicated or general-purpose programmable processor, and can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit the data and instructions to the storage system, the at least one input device, and the at least one output device.

[0148] These computing programs (also referred to as programs, software, software applications, or code) include machine instructions for a programmable processor, and these computing programs can be implemented using high-level procedural and / or object-oriented programming languages, and / or assembly / machine languages. As used herein, the terms “machine-readable medium” and “computer-readable medium” refer to any computer program product, device, and / or apparatus (e.g., a disk, an optical disk, a memory, a programmable logic device (PLD)) for providing machine instructions and / or data to a programmable processor, including a machine-readable medium that receives machine instructions as a machine-readable signal. The term “machine-readable signal” refers to any signal for providing machine instructions and / or data to a programmable processor.

[0149] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and a pointing device (e.g., a mouse or a trackball) by which the user can provide input to the computer. Other kinds of devices can also be used to provide interaction with the user; for example, the feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including acoustic input, voice input, or tactile input).

[0150] The systems and techniques described herein can be implemented in a computing system including backend components (e.g., as a data server), or a computing system including middleware components (e.g., an application server), or a computing system including frontend components (e.g., a user computer having a graphical user interface or a web browser through which the user can interact with an implementation of the systems and techniques described herein), or a computing system including any combination of such backend components, middleware components, or frontend components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: local area network (LAN), wide area network (WAN), and the Internet.

[0151] A computer system can include a client and a server. The client and the server are generally remote from each other and typically interact through a communication network. The relationship of the client and the server is generated by computer programs running on the respective computers and having a client-server relationship with each other.

[0152] It should be understood that various forms of the processes shown above can be used, with steps reordered, added, or deleted. For example, the steps recited in this application can be executed in parallel, sequentially, or in a different order, as long as the desired results of the technical solutions disclosed in this application can be achieved, and no limitation is imposed herein.

[0153] The above specific embodiments do not constitute a limitation on the protection scope of this application. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this application shall be included within the protection scope of this application.

Claims

1. A method for determining information, comprising: Obtaining overall context information and a plurality of candidate response information for replying to the overall context information; For each candidate response information among the plurality of candidate response information, determining the total difference information between the candidate response information and the other candidate response information except the candidate response information among the plurality of candidate response information; Determining the consistency information between the total difference information and the overall context information as the first consistency information between the candidate response information and the overall context information, wherein the first consistency information is used to characterize the logical consistency degree and / or similarity degree between the content expressed by the total difference information and the content expressed by the overall context information, or the logical consistency degree and / or similarity degree corresponding between the grammar and / or semantics of the total difference information and the grammar and / or semantics of the overall context information; Determining the target response information from the plurality of candidate response information according to the first consistency information corresponding to each candidate response information.

2. The method according to claim 1, wherein The step of, for each candidate response information among the plurality of candidate response information, determining the total difference information between the candidate response information and the other candidate response information except the candidate response information among the plurality of candidate response information, includes: Obtaining the word vector of each candidate response information; For each candidate response information, determining the difference information between the candidate response information and each of the other candidate response information based on the similarity between the word vector of the candidate response information and the word vector of each of the other candidate response information; Determining the difference information between the candidate response information and all other response information as the total difference information between the candidate response information and the other candidate response information.

3. The method according to claim 2, wherein, The step of determining the consistency information between the total difference information and the overall context information as the first consistency information between the candidate response information and the overall context information, includes: Obtaining the word vector of the overall context information; Determining the consistency information between the total difference information and the word vector of the overall context information as the first consistency information between the candidate response information and the overall context information.

4. The method according to claim 1, wherein The method further includes: Determining the consistency information between the total difference information and the target context information as the second consistency information between the candidate response information and the target context information, wherein the target context information is the information in the overall context information that belongs to the same conversation party as the candidate response information; The step of determining the target response information from the plurality of candidate response information according to the first consistency information corresponding to each candidate response information, includes: Determining the target response information from the plurality of candidate response information according to the first consistency information and the second consistency information corresponding to each candidate response information.

5. The method according to claim 4, wherein, The total difference information is determined based on the similarity between the word vector of the candidate response information and the word vectors of the other candidate response information except the candidate response information among the plurality of candidate response information; Determining the consistency information between the total difference information and the target above - text information as the second consistency information between the candidate response information and the target above - text information includes: Obtaining the word vectors of the target above - text information; Determining the consistency information between the total difference information and the word vectors of the target above - text information as the second consistency information between the candidate response information and the target above - text information.

6. The method according to claim 4, wherein Determining the target response information from the multiple candidate response information according to the first consistency information and the second consistency information corresponding to each candidate response information includes: For each candidate response information, determining the score of the candidate response information by using the first consistency information corresponding to the candidate response information, the second consistency information corresponding to the candidate response information, and the third consistency information between the semantics of the candidate response information and the semantics of the overall above - text information; Determining the target response information from the multiple candidate response information according to the scores of each candidate response information.

7. A device for determining information, comprising: An acquisition unit configured to acquire overall above - text information and multiple candidate response information for replying to the overall above - text information; A first determination unit configured to, for each candidate response information among the multiple candidate response information, determine the total difference information between the candidate response information and other candidate response information among the multiple candidate response information except the candidate response information; A second determination unit configured to determine the consistency information between the total difference information and the overall above - text information as the first consistency information between the candidate response information and the overall above - text information, where the first consistency information is used to characterize the logical consistency degree and / or similarity degree between the content expressed by the total difference information and the content expressed by the overall above - text information, or the logical consistency degree and / or similarity degree corresponding to the grammar and / or semantics of the total difference information and the grammar and / or semantics of the overall above - text information, etc.; A third determination unit configured to determine the target response information from the multiple candidate response information according to the first consistency information corresponding to each candidate response information.

8. The apparatus according to claim 7, wherein, The first determination unit includes: A first acquisition module configured to acquire the word vectors of each candidate response information; A first determination module configured to, for each candidate response information, determine the difference information between the candidate response information and each other candidate response information based on the similarity between the word vector of the candidate response information and the word vectors of each other candidate response information among the other candidate response information; A second determination module configured to determine the difference information between the candidate response information and all other response information as the total difference information between the candidate response information and the other candidate response information.

9. The apparatus according to claim 7, wherein The second determination unit includes: A second acquisition module configured to acquire the word vectors of the overall above - text information; A third determination module, configured to determine the consistency information between the total difference information and the word vectors of the overall above-text information as the first consistency information between the candidate response information and the overall above-text information.

10. The apparatus according to claim 7, wherein The apparatus includes: A fourth determination unit, configured to determine the consistency information between the total difference information and the target above-text information as the second consistency information between the candidate response information and the target above-text information, where the target above-text information is the information in the overall above-text information that belongs to the same dialogue party as the candidate response information; The third determination unit includes: A fourth determination module, configured to determine the target response information from the multiple candidate response information according to the first consistency information and the second consistency information corresponding to each candidate response information.

11. The apparatus according to claim 10, wherein, The total difference information is determined based on the similarity between the word vectors of the candidate response information and the word vectors of other candidate response information except the candidate response information among the multiple candidate response information. The fourth determination unit includes: A third acquisition module, configured to acquire the word vectors of the target above-text information; A fifth determination module, configured to determine the consistency information between the total difference information and the word vectors of the target above-text information as the second consistency information between the candidate response information and the target above-text information.

12. The device according to claim 10, wherein, The fourth determination module includes: A scoring module, configured to determine the score of each candidate response information by using the first consistency information corresponding to the candidate response information, the second consistency information corresponding to the candidate response information, and the third consistency information between the semantics of the candidate response information and the semantics of the overall above-text information; A selection module, configured to determine the target response information from the multiple candidate response information according to the scores of each candidate response information.

13. An electronic device, including: At least one processor; And A memory communicatively connected to the at least one processor; wherein, The memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the method according to any one of claims 1-6.

14. A non-transitory computer-readable storage medium storing computer instructions, wherein, The computer instructions are used to cause the computer to execute the method according to any one of claims 1-6.

Images