A method, device, equipment and medium for determining a conversation response scheme

By calculating the matching results of the session information to be answered in the intelligent customer service system and the answers to the question, the problem of low text matching accuracy is solved, and the response accuracy of the intelligent customer service system is improved.

CN111753062BActive Publication Date: 2024-11-26BEIJING JINGDONG SHANGKE INFORMATION TECH CO LTD +1
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN201911075722.2
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2019-11-06
Publication Date
2024-11-26
Estimated Expiration
2039-11-06

AI Technical Summary

Technical Problem

The accuracy of text matching in the intelligent customer service system is not high, resulting in inaccurate responses, and it is necessary to improve the accuracy and recall rate of text matching.

Method used

By obtaining the session information to be answered and the question answering pairs in the question answering database, the matching results between the session information to be answered and the question and answers are calculated respectively, and the target matching results are determined based on these matching results, and the response scheme is determined. Specific methods include standardizing the processing of session information, calculating the matching results using cosine similarity algorithm, deep semantic matching model, etc., and determining the final matching results through weighted summing or result fusion model.

Benefits of technology

Improve the accuracy and recall rate of text matching, thereby improving the response accuracy of the intelligent customer service system.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN111753062B_ABST
    Figure CN111753062B_ABST
Patent Text Reader

Abstract

The embodiment of the present invention discloses a method, device, equipment and medium for determining a conversation response scheme. The method comprises: obtaining conversation information to be responded to contained in a detected conversation response request and a question-answer pair in a question-answer library; determining a first matching result between the conversation information to be responded to and the question in the question-answer pair, and determining a second matching result between the conversation information to be responded to and the answer in the question-answer pair; determining a target matching result between the conversation information to be responded to and the question-answer pair according to the first matching result and the second matching result, and determining a response scheme for the conversation response request according to the target matching result, by matching the conversation information to be responded to with the answer and the question in the question-answer pair respectively, determining the target matching result according to the first matching result and the second matching result, and determining the response scheme based on the target matching result, thereby improving the accuracy and recall rate of text matching and improving the response accuracy of the intelligent customer service system.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The embodiments of the present invention relate to the field of information processing, and in particular to a method, device, equipment and medium for determining a conversation response scheme. Background Art

[0002] With the rapid development of the Internet, intelligent customer service systems have also developed rapidly on major Internet platforms. Text matching is a core problem in natural language processing. In intelligent customer service systems, text matching is one of the core technologies, and many businesses can be abstracted into text matching problems to a large extent. In the process of realizing the present invention, the inventors found that there are at least the following technical problems in the prior art: At present, text matching in intelligent customer service systems generally has the problem of low accuracy, which leads to inaccurate responses from intelligent customer service systems. Therefore, how to improve the accuracy and recall rate of text matching is a technical problem that needs to be solved urgently. Summary of the invention

[0003] The embodiments of the present invention provide a method, device, equipment and medium for determining a conversation response scheme, so as to improve the accuracy and recall rate of text matching, thereby improving the response accuracy of an intelligent customer service system.

[0004] In a first aspect, an embodiment of the present invention provides a method for determining a session response scheme, including:

[0005] Obtaining the to-be-answered session information contained in the detected session answer request and the question-answer pair in the question-answer library;

[0006] Determine a first matching result between the unanswered session information and the question in the question-answer pair, and determine a second matching result between the unanswered session information and the answer in the question-answer pair, wherein the first matching result is used to characterize the matching degree between the unanswered session information and the question in the question-answer pair, and the second matching result is used to characterize the matching degree between the unanswered session information and the answer in the question-answer pair;

[0007] A target matching result of the to-be-answered session information and the question-answer pair is determined according to the first matching result and the second matching result, and a response solution for the session response request is determined according to the target matching result.

[0008] In a second aspect, an embodiment of the present invention further provides a session response scheme determination device, including:

[0009] A matching parameter acquisition module is used to obtain the to-be-answered session information contained in the detected session answer request and the question-answer pair in the question-answer library;

[0010] a matching result acquisition module, configured to determine a first matching result between the unanswered session information and the question in the question-answer pair, and to determine a second matching result between the unanswered session information and the answer in the question-answer pair, wherein the first matching result is used to characterize the matching degree between the unanswered session information and the question in the question-answer pair, and the second matching result is used to characterize the matching degree between the unanswered session information and the answer in the question-answer pair;

[0011] A response result determination module is used to determine a target matching result of the to-be-answered session information and the question answer pair according to the first matching result and the second matching result, and to determine a response solution for the session response request according to the target matching result.

[0012] In a third aspect, an embodiment of the present invention further provides a computer device, the device comprising:

[0013] one or more processors;

[0014] A storage device for storing one or more programs;

[0015] When the one or more programs are executed by the one or more processors, the one or more processors implement the method for determining a session response solution as provided in any embodiment of the present invention.

[0016] In a fourth aspect, an embodiment of the present invention further provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements a method for determining a session response solution as provided in any embodiment of the present invention.

[0017] The embodiment of the present invention obtains the unanswered session information contained in the detected session response request and the question-answer pair in the question-answer library; determines a first matching result between the unanswered session information and the question in the question-answer pair, and determines a second matching result between the unanswered session information and the answer in the question-answer pair; determines a target matching result between the unanswered session information and the question-answer pair according to the first matching result and the second matching result, and determines a response plan for the session response request according to the target matching result, matches the unanswered session information with the answer and the question in the question-answer pair respectively, determines a target matching result between the unanswered session information and the question-answer pair according to the first matching result between the unanswered session information and the question and the second matching result between the unanswered session information and the answer, and determines a response plan based on the target matching result, thereby improving the accuracy and recall rate of text matching, and thereby improving the response accuracy of the intelligent customer service system. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] Figure 1It is a flow chart of a method for determining a session response solution provided in Embodiment 1 of the present invention;

[0019] Figure 2a It is a flow chart of a method for determining a session response solution provided by Embodiment 2 of the present invention;

[0020] Figure 2b It is a model diagram of a representational deep text matching model;

[0021] Figure 2c This is a model diagram of an interactive deep text matching model;

[0022] Figure 3a It is a flow chart of a method for determining a session response solution provided by Embodiment 3 of the present invention;

[0023] Figure 3b It is a flowchart of a method for determining a session response solution provided by Embodiment 3 of the present invention;

[0024] Figure 4 It is a structural diagram of a device for determining a session response solution provided by a fourth embodiment of the present invention;

[0025] Figure 5 It is a structural diagram of a computer device provided in Embodiment 5 of the present invention. DETAILED DESCRIPTION

[0026] The present invention will be further described in detail below in conjunction with the accompanying drawings and embodiments. It is to be understood that the specific embodiments described herein are only used to explain the present invention, rather than to limit the present invention. It should also be noted that, for ease of description, only parts related to the present invention, rather than all structures, are shown in the accompanying drawings.

[0027] Embodiment 1

[0028] Figure 1 This is a flow chart of a method for determining a session response solution provided in Embodiment 1 of the present invention. This embodiment can be applied to the situation when determining a response solution corresponding to a response session request initiated by a user. The method can be executed by a session response solution determination device, which can be implemented in software and / or hardware. For example, the session response solution determination device can be configured in a computer device.

[0029] like Figure 1 As shown, the method includes:

[0030] S110: Acquire the to-be-answered session information contained in the detected session answer request and the question-answer pairs in the question-answer library.

[0031] In this embodiment, the conversation response request may be a request initiated by the user for obtaining response information. Exemplarily, when the user sends conversation information to the intelligent customer service through the terminal, the terminal generates a conversation response request containing the conversation information, and sends the conversation response request to the conversation response solution determination device. After receiving the conversation response request, the conversation response solution determination device parses the conversation response request, obtains the conversation information contained in the conversation response request, and uses the obtained conversation information as the conversation information to be answered. Optionally, the question answer library may be a pre-set database containing question-answer pairs. Exemplarily, the question answer library may be a FAQ (Frequently Asked Questions) library.

[0032] In one embodiment of the present invention, the acquiring the to-be-answered session information contained in the detected session answering request includes:

[0033] Obtaining the session information contained in the detected session response request;

[0034] The session information is standardized to obtain the unanswered session information, wherein the standardized processing includes at least one of traditional Chinese character conversion, special symbol conversion, character segmentation, stop word removal, and information completion.

[0035] In order to make the answer scheme corresponding to the matched session information more accurate, the session information obtained from the session answer request can be standardized, and the information obtained after the standardization process is used as the session information to be answered. Optionally, the main operations of the standardization process include: at least one of the following: traditional and simplified Chinese conversion, English uppercase and lowercase conversion, special symbol conversion, character segmentation, stop word removal, and information completion. Exemplarily, after obtaining the session information contained in the session answer request, firstly, all traditional Chinese characters in the session information are converted into simplified Chinese characters through the traditional and simplified dictionary table, and then the abnormal symbols existing therein are filtered out through the special symbol word table, and then the filtered session information is segmented by word segmentation technology, and the stop words in the word segmentation are removed through the stop word word table, and the finally obtained information is used as the session information to be answered.

[0036] S120: Determine a first matching result between the conversation information to be answered and the question in the question-answer pair, and determine a second matching result between the conversation information to be answered and the answer in the question-answer pair.

[0037] In this embodiment, in order to improve the accuracy and recall rate of text matching, the conversation information to be answered is matched with the question and answer in the question-answer pair respectively to obtain a first matching result and a second matching result, and the question-answer pair matching the conversation information to be answered is determined by combining the first matching result and the second matching result. The first matching result is used to characterize the matching degree between the conversation information to be answered and the question in the question-answer pair, and the second matching result is used to characterize the matching degree between the conversation information to be answered and the answer in the question-answer pair.

[0038] Optionally, the first matching result between the unanswered conversation information and the question in the question-answer pair, and the second matching result between the unanswered conversation information and the answer in the question-answer pair can be calculated by a cosine similarity algorithm, a deep semantic matching model (DSSM), a vector space model, a longest common substring, a longest common subsequence, a minimum edit distance method, a Hamming distance, and other algorithms. Among them, the method for calculating the first matching result and the method for calculating the second matching result can be the same or different. Exemplarily, the first matching result between the unanswered conversation information and the question in the question-answer pair can be calculated by a deep semantic matching model, and the second matching result between the unanswered conversation information and the answer in the question-answer pair can be calculated by a cosine similarity algorithm. Exemplarily, the first matching result between the unanswered conversation information and the question in the question-answer pair, and the second matching result between the unanswered conversation information and the answer in the question-answer pair can also be calculated by a deep semantic matching model.

[0039] In one embodiment of the present invention, determining a second matching result between the to-be-answered conversation information and the answer in the question-answer pair includes:

[0040] The unanswered conversation information and the answer in the question-answer pair are input into a trained answer matching model to obtain a second matching result output by the answer matching model.

[0041] In one embodiment, in order to make the calculation of the second matching result more accurate, the answer matching model used to calculate the second matching result can be pre-trained. After obtaining the unanswered session information, the second matching result between the unanswered session information and the answer in the question-answer pair is obtained based on the unanswered session information, the answer in the question-answer pair, and the trained answer matching model. Exemplarily, the unanswered session information sample, the answer sample, and the matching result between the unanswered session information sample and the answer sample are obtained, and the answer matching model training sample pair is generated based on the unanswered session information sample, the answer sample, and the matching result between the unanswered session information sample and the answer sample. The answer matching model training sample pair is used to train the pre-built answer matching model to obtain a trained answer matching model. The answer matching model can be a machine learning model such as a convolutional neural network and a recurrent neural network.

[0042] S130: Determine a target matching result of the pair of the to-be-answered conversation information and the question answer according to the first matching result and the second matching result, and determine a response solution for the conversation response request according to the target matching result.

[0043] In this embodiment, after obtaining the first matching result between the conversation information to be answered and the question and the second matching result between the conversation information to be answered and the answer, the target matching result of the conversation information to be answered and the question-answer pair is determined in combination with the first matching result and the second matching result, and the question-answer pair matching the conversation information to be answered is determined based on the target matching result, and the answer scheme corresponding to the conversation information to be answered is determined according to the determined question-answer pair.

[0044] Optionally, the target matching result may be a matching value, and the matching values ​​between the session information to be answered and each question-answer pair may be arranged in descending order, and the question-answer pair with the largest matching value with the session information to be answered may be taken as the question-answer pair that matches the session information to be answered, and the response plan for the session information to be answered may be determined based on the question-answer pair.

[0045] In one embodiment of the present invention, determining a target matching result of the unanswered conversation information and the question-answer pair according to the first matching result and the second matching result includes:

[0046] The first matching result and the second matching result are weightedly summed to obtain the target matching result.

[0047] In one embodiment, the weights of the first matching result and the second matching result may be preset, and after the first matching result and the second matching result are determined, the first matching result and the second matching result are weighted and summed to obtain the target matching result. 1 +bS 2Calculate the target matching result between the conversation information to be answered and the question-answer pair. Where S is the target matching result between the conversation information to be answered and the question-answer pair, S 1 is the first matching result between the conversation information to be answered and the question answer pair, a is the weight of the first matching result, S 2 is the second matching result between the conversation information to be answered and the question-answer pair, and b is the weight of the second matching result.

[0048] In another embodiment of the present invention, the result fusion model may be pre-trained. After obtaining the first matching result and the second matching result, the target matching result between the to-be-answered conversation information and the question answer pair is obtained based on the first matching result, the second matching result, and the trained result fusion model. Exemplarily, the first matching result sample, the second matching result sample, and the target matching result sample corresponding to the first matching result sample and the second matching result are obtained, and a result fusion model training sample pair is generated based on the first matching result sample, the second matching result sample, and the target matching result sample corresponding to the first matching result sample and the second matching result. The result fusion model training sample pair is used to train the pre-constructed result fusion model to obtain a trained result fusion model. The result fusion model may be a support vector machine model.

[0049] The embodiment of the present invention obtains the unanswered session information contained in the detected session response request and the question-answer pair in the question-answer library; determines a first matching result between the unanswered session information and the question in the question-answer pair, and determines a second matching result between the unanswered session information and the answer in the question-answer pair; determines a target matching result between the unanswered session information and the question-answer pair according to the first matching result and the second matching result, and determines a response plan for the session response request according to the target matching result, and matches the unanswered session information with the answer and the question in the question-answer pair respectively, determines a target matching result between the unanswered session information and the question-answer pair according to the first matching result between the unanswered session information and the question and the second matching result between the unanswered session information and the answer, and determines a response plan based on the target matching result, thereby improving the accuracy and recall rate of text matching, and thereby improving the response accuracy of the intelligent customer service system.

[0050] Embodiment 2

[0051] FIG2 is a flow chart of a method for determining a session response scheme provided by Embodiment 2 of the present invention. This embodiment is optimized on the basis of the above embodiment. As shown in FIG2 , the method includes:

[0052] S210: Acquire the to-be-answered session information contained in the detected session answer request and the question-answer pairs in the question-answer library.

[0053] S220: Input the conversation information to be answered and the question in the question-answer pair into the trained question matching model to obtain a first matching result output by the question matching model.

[0054] In this embodiment, the method of obtaining the first matching result between the unanswered conversation information and the question in the question-answer pair is specific, and the first matching result between the unanswered conversation information and the question is obtained according to the trained question matching model. Optionally, the question matching model can be a representational deep text matching model or an interactive deep text matching model.

[0055] The representational deep text matching model focuses more on the construction of the representation layer, where it converts the text into a unique overall representation vector and determines the feedback result based on the overall representation vector. Typical network structures of representational deep text matching models include DSSM, CDSMM (Convolutional latent semantic model) and ARC-I. Figure 2b It is a model diagram of a representational deep text matching model. Figure 2b The calculation process of a representational deep text matching model is schematically shown in FIG. Figure 2b As shown in the figure, text A is converted into word embedding (Word Embeding) A, and text B is converted into word embedding B. After a series of nonlinear transformations of word embedding A and word embedding B, the overall representation vector A (Query A Representation) of text A and the overall representation vector B (Query B Representation) of text B are obtained. The similarity between the overall representation vector A and the overall representation vector B is calculated by the cosine similarity algorithm (cosine) to obtain the matching degree of text A and text B.

[0056] However, the sentence representation obtained by the representational deep text matching model loses semantic focus, is prone to semantic drift, and the contextual importance of words is difficult to measure. The interactive deep text matching model assumes that the global matching degree depends on the local matching degree, first matches words at the input layer, and uses the matching results as grayscale images for subsequent modeling, so that the semantic focus can be better grasped in the context during the matching process. Typical network structures of representational deep text matching models include ARC-II, DeepMatch, and MatchPyramid. Figure 2c This is a model diagram of an interactive deep text matching model. Figure 2c The calculation process of an interactive deep text matching model is schematically shown in FIG. Figure 2cAs shown, text A is converted into word vector A, text B is converted into word vector B, and then the interaction matrix of word vector A and word vector B is obtained. After a series of nonlinear transformations are performed on the interaction matrix, the matching degree between text A and text B is obtained through a normalized exponential function (softmax). Preferably, an interactive deep text matching model is used as the question matching model, so that the semantic focus can be better grasped during the matching process of the question matching model, thereby making the matching result more accurate.

[0057] In one embodiment of the present invention, the question matching model includes a semantic matching model and a feature fusion model, the semantic matching model is obtained based on attention model training, and the question in the question-answer pair and the to-be-answered conversation information are input into the trained question matching model to obtain a first matching result output by the question matching model, including:

[0058] Inputting the unanswered conversation information and the question into the trained semantic matching model to obtain a semantic matching result output by the semantic matching model;

[0059] The matching features between the to-be-answered conversation information and the question and the semantic matching result are input into a trained feature fusion model, and a matching result output by the feature fusion model is obtained as the first matching result.

[0060] Considering the situation that when two texts are not similar in literal representation but express the same meaning, or when two texts are similar in literal representation but express different meanings, for example, "taxi" and "taxi" are not similar in literal representation but are actually the same means of transportation, and "machine learning" and "learning machine" have completely overlapping vocabulary but express different meanings. In this embodiment, semantic matching is performed between the conversation information to be answered and the question, and the semantic matching result is used as a feature of calculating the first matching result between the conversation information to be answered and the question in the question answer pair, and the first matching result between the conversation information to be answered and the question is determined in combination with the matching feature between the conversation information to be answered and the question.

[0061] Optionally, the semantic matching result between the unanswered conversation information and the question in the question-answer pair can be obtained by a trained semantic matching model. The semantic matching model is constructed based on the attention model and trained by mutual information technology. Exemplarily, samples of unanswered conversation information, question samples, and samples of semantic matching results between the unanswered conversation information samples and the question samples are obtained, and a result semantic matching model training sample pair is generated based on the samples of unanswered conversation information, question samples, and samples of semantic matching results between the unanswered conversation information samples and the question samples. The pre-constructed semantic matching model is trained using the semantic matching model sample pair and mutual information technology to obtain a trained semantic matching model.

[0062] Optionally, in combination with the semantic matching result and matching feature between the unanswered conversation information and the question, it is determined that the first matching result between the unanswered conversation information and the question can be obtained by a trained feature fusion model. Exemplarily, a semantic matching result sample, a matching feature sample, and a matching result sample between the unanswered conversation information sample and the question sample are obtained, and a feature fusion model training sample pair is generated based on the semantic matching result sample, the matching feature sample, and the matching result sample between the unanswered conversation information sample and the question sample, and the feature fusion model training sample pair is used to train the pre-built feature fusion model to obtain a trained feature fusion model. The feature fusion model can be a machine learning model such as a convolutional neural network and a recurrent neural network.

[0063] In this embodiment, the matching features between the conversation information to be answered and the question include: at least one of a length parameter of the conversation information to be answered, a length parameter of the question, a longest common subsequence length of the conversation information to be answered and the question, a cosine similarity between a word vector of the conversation information to be answered and a word vector of the question, a cosine similarity between a word vector of a core word of the conversation information to be answered and a word vector of the core word of the question, an intersection / union size between a segmentation set of the conversation information to be answered and a segmentation combination of the question, a cosine similarity between an average value of the word vector of the conversation information to be answered and an average value of the word vector of the question, and a cosine similarity between a word vector of a feature word of the conversation information to be answered and a word vector of a feature word of the question.

[0064] The length parameter of the conversation information to be answered may be a ratio of the length of the conversation information to be answered to a set value; the question length parameter may be a ratio of the question length to a set value, wherein the set value may be set according to actual needs; the cosine similarity between the word vector of the conversation information to be answered and the word vector of the question may be a cosine similarity between the sum of all the word vectors of the conversation information to be answered and the sum of all the word vectors of the question; the cosine similarity between the word vector of the core word of the conversation information to be answered and the word vector of the core word of the question may be a cosine similarity between the sum of the word vectors of all the core words of the conversation information to be answered and the sum of the word vectors of all the core words of the answer after the core words of the conversation information to be answered and the core words of the question are respectively determined by inverse document frequency (IDF); the cosine similarity between the word vector of the feature word of the conversation information to be answered and the word vector of the feature word of the question may be a cosine similarity between the sum of the word vectors of nouns and verbs in the conversation information to be answered and the sum of the word vectors of nouns and verbs in the question.

[0065] S230: Determine a second matching result between the to-be-answered conversation information and the answer in the question-answer pair.

[0066] S240: Determine a target matching result of the pair of the to-be-answered conversation information and the question answer according to the first matching result and the second matching result, and determine a response solution for the conversation response request according to the target matching result.

[0067] The technical solution of the embodiment of the present invention concretizes the method of obtaining the first matching result between the conversation information to be answered and the question in the question-answer pair, by inputting the conversation information to be answered and the question in the question-answer pair into a trained question matching model, obtaining the first matching result output by the question matching model, and determining the first matching result between the conversation information to be answered and the question by combining the semantic matching result and matching features between the conversation information to be answered and the question, so that the determination of the first matching result is more accurate.

[0068] Embodiment 3

[0069] Figure 3a 1 is a flow chart of a method for determining a session response solution provided by Embodiment 3 of the present invention. This embodiment is optimized on the basis of the above embodiment. Figure 3a As shown, the method includes:

[0070] S310: Acquire the to-be-answered session information contained in the detected session answer request and the question-answer pairs in the question-answer library.

[0071] S320: Determine a first matching result between the conversation information to be answered and the question in the question-answer pair, and determine a second matching result between the conversation information to be answered and the answer in the question-answer pair.

[0072] S330: Obtain a standard matching result of the question and the answer in the question-answer pair, and calibrate the second matching result according to the standard matching result to obtain a second calibrated matching result.

[0073] Since the matching semantic space of the conversation information to be answered and the answer is not the same space, there may be a correct match but the second matching result score is very low. In this embodiment, the second matching result between the conversation information to be answered and the answer is corrected by the standard matching result between the question and the answer. The standard matching result between the question and the answer can be calculated by a text similarity algorithm or by a machine learning model.

[0074] In one embodiment of the present invention, obtaining the standard matching result of the question and answer in the question-answer pair includes: inputting the question and answer into a trained question-answer matching model to obtain the standard matching result output by the question-answer matching model.

[0075] In one embodiment, a trained question-answer matching model can be used to obtain a standard matching result between a question and an answer. Exemplarily, a question sample, an answer sample, and a matching result sample between the question sample and the answer sample are obtained, and a question-answer matching model training sample pair is generated based on the question sample, the answer sample, and the matching result sample between the question sample and the answer sample. The pre-built question-answer matching model is trained using the question-answer matching model sample pair and mutual information technology to obtain a trained question-answer matching model. The question-answer matching model can be a machine learning model such as a convolutional neural network or a recurrent neural network.

[0076] In one embodiment of the present invention, calibrating the second matching result according to the standard matching result to obtain a second calibrated matching result includes:

[0077] determining a ratio between the second matching result and the standard matching result;

[0078] The ratio is compared with a preset threshold, and the smaller value between the ratio and the threshold is taken as the second calibration matching result.

[0079] Optionally, you can use S 2 =min(S 2 / S 3 , A) calibrate the second matching result. Where S 2is the second matching result between the conversation information to be answered and the answer, S 3 is the standard matching result between the question and the answer, and A is the set threshold. Optionally, the value of A can be 1.

[0080] S340: Determine a target matching result of the to-be-answered conversation information and the question answer pair according to the first matching result and the second calibration matching result.

[0081] Figure 3b FIG. 1 is a flow chart of a method for determining a session response solution provided by Embodiment 3 of the present invention. Figure 3b As shown, the conversation information input by the user is preprocessed by word segmentation, simplified and traditional Chinese conversion, stop word removal, information completion, etc. to obtain the conversation information to be answered, and the conversation information to be answered is input into the question matching model and the answer matching model respectively. The question matching model obtains the question-answer pair from the FAQ library, and calculates the first matching result between the conversation information to be answered and the question in the question-answer pair. The answer matching model obtains the question-answer pair from the FAQ library, and calculates the second matching result between the conversation information to be answered and the answer in the question-answer pair. The question-answer matching model obtains the question-answer pair from the FAQ library, and calculates the standard matching result between the question and the answer in the question-answer pair. After calibrating the second matching result with the standard matching result, the first matching result and the second calibrated matching result are passed through the matching result fusion model to obtain the target matching result between the conversation information to be answered and the question-answer pair, and the answer scheme for the conversation information to be answered is determined based on the target matching result and output.

[0082] The technical solution of the embodiment of the present invention adds an operation of calibrating the second matching result after obtaining the second matching result. By calibrating the second matching result, the target matching result between the to-be-answered conversation information and the question-answer pair is determined based on the second calibration matching result obtained after the calibration and the first matching result, so that the determination of the target matching result is more accurate.

[0083] Embodiment 4

[0084] Figure 4 1 is a schematic diagram of the structure of a session response scheme determination device provided by Embodiment 4 of the present invention. The session response scheme determination device can be implemented in software and / or hardware, for example, the session response scheme determination device can be configured in a computer device. Figure 4 As shown, the device includes a matching parameter acquisition module 410, a matching result acquisition module 420 and a response result determination module 430, wherein:

[0085] A matching parameter acquisition module 410 is used to acquire the to-be-answered session information contained in the detected session answer request and the question-answer pair in the question-answer library;

[0086] A matching result acquisition module 420 is used to determine a first matching result between the unanswered session information and the question in the question-answer pair, and to determine a second matching result between the unanswered session information and the answer in the question-answer pair, wherein the first matching result is used to characterize the matching degree between the unanswered session information and the question in the question-answer pair, and the second matching result is used to characterize the matching degree between the unanswered session information and the answer in the question-answer pair;

[0087] The response result determination module 430 is used to determine a target matching result of the to-be-answered session information and the question-answer pair according to the first matching result and the second matching result, and determine a response solution for the session response request according to the target matching result.

[0088] In the embodiment of the present invention, a matching parameter acquisition module is used to acquire the unanswered session information contained in the detected session response request and the question-answer pair in the question-answer library; a matching result acquisition module is used to determine a first matching result between the unanswered session information and the question in the question-answer pair, and a second matching result between the unanswered session information and the answer in the question-answer pair; a response result determination module is used to determine a target matching result between the unanswered session information and the question-answer pair according to the first matching result and the second matching result, and a response scheme for the session response request is determined according to the target matching result. By matching the unanswered session information with the answer and the question in the question-answer pair respectively, determining the target matching result between the unanswered session information and the question-answer pair according to the first matching result between the unanswered session information and the question and the second matching result between the unanswered session information and the answer, and determining the response scheme based on the target matching result, the accuracy and recall rate of text matching are improved, thereby improving the response accuracy of the intelligent customer service system.

[0089] Based on the above solution, the matching result acquisition module 410 includes:

[0090] The first result acquisition unit is used to input the unanswered conversation information and the question in the question-answer pair into the trained question matching model to obtain a first matching result output by the question matching model.

[0091] Based on the above solution, the question matching model includes a semantic matching model and a feature fusion model, the semantic matching model is obtained based on attention model training, and the first result acquisition unit is specifically used for:

[0092] Inputting the unanswered conversation information and the question into the trained semantic matching model to obtain a semantic matching result output by the semantic matching model;

[0093] The matching features between the to-be-answered conversation information and the question and the semantic matching result are input into a trained feature fusion model, and a matching result output by the feature fusion model is obtained as the first matching result.

[0094] Based on the above solution, the device further includes a second result calibration module, which is used to:

[0095] After obtaining the second matching result output by the answer matching model, obtaining a standard matching result of the question and the answer in the question-answer pair, and calibrating the second matching result according to the standard matching result to obtain a second calibrated matching result;

[0096] Accordingly, the response result determination module 430 is used to:

[0097] A target matching result between the to-be-answered conversation information and the question-answer pair is determined according to the first matching result and the second calibration matching result.

[0098] Based on the above solution, the second result calibration module includes:

[0099] The standard result acquisition unit is used to input the question and answer into the trained question-answer matching model to obtain the standard matching result output by the question-answer matching model.

[0100] Based on the above solution, the second result calibration module includes a second result calibration unit, which is used to:

[0101] determining a ratio between the second matching result and the standard matching result;

[0102] The ratio is compared with a preset threshold, and the smaller value between the ratio and the threshold is taken as the second calibration matching result.

[0103] Based on the above solution, the response result determination module 430 is specifically used to:

[0104] The first matching result and the second matching result are weightedly summed to obtain the target matching result.

[0105] Based on the above solution, the matching parameter acquisition module 410 is specifically used for:

[0106] Obtaining the session information contained in the detected session response request;

[0107] The session information is standardized to obtain the unanswered session information, wherein the standardized processing includes at least one of traditional Chinese character conversion, special symbol conversion, character segmentation, stop word removal, and information completion.

[0108] Based on the above solution, the matching result acquisition module 410 includes:

[0109] The second result acquisition unit is used to input the unanswered conversation information and the answer in the question-answer pair into the trained answer matching model to obtain a second matching result output by the answer matching model.

[0110] The session response solution determination device provided in the embodiment of the present invention can execute the session response solution determination method provided in any embodiment, and has the corresponding functional modules and beneficial effects of the execution method.

[0111] Embodiment 5

[0112] Figure 5 It is a structural diagram of a computer device provided in Embodiment 5 of the present invention. Figure 5 A block diagram of an exemplary computer device 512 suitable for use in implementing embodiments of the present invention is shown. Figure 5 The computer device 512 shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present invention.

[0113] like Figure 5 As shown, the computer device 512 is in the form of a general-purpose computing device. The components of the computer device 512 may include, but are not limited to: one or more processors 516, a system memory 528, and a bus 518 that connects various system components (including the system memory 528 and the processor 516).

[0114] Bus 518 represents one or more of several types of bus structures, including a memory bus or memory controller, a peripheral bus, an accelerated graphics port, a local bus to processor 516, or a bus structure using any of a variety of bus architectures. These architectures include, by way of example, but are not limited to, an Industry Standard Architecture (ISA) bus, a Micro Channel Architecture (MAC) bus, an Enhanced ISA bus, a Video Electronics Standards Association (VESA) local bus, and a Peripheral Component Interconnect (PCI) bus.

[0115] The computer device 512 typically includes a variety of computer system readable media. These media can be any available media that can be accessed by the computer device 512, including volatile and non-volatile media, removable and non-removable media.

[0116] System memory 528 may include computer system readable media in the form of volatile memory, such as random access memory (RAM) 530 and / or cache memory 532. Computer device 512 may further include other removable / non-removable, volatile / non-volatile computer system storage media. By way of example only, storage device 534 may be used to read and write non-removable, non-volatile magnetic media ( Figure 5 not shown, usually called a "hard drive"). Although Figure 5 Not shown, a disk drive for reading and writing to a removable non-volatile disk (e.g., a "floppy disk"), and an optical disk drive for reading and writing to a removable non-volatile optical disk (e.g., a CD-ROM, a DVD-ROM, or other optical media) may be provided. In these cases, each drive may be connected to the bus 518 via one or more data medium interfaces. The memory 528 may include at least one program product having a set (e.g., at least one) of program modules that are configured to perform the functions of the various embodiments of the present invention.

[0117] A program / utility 540 having a set (at least one) of program modules 542 may be stored, for example, in the memory 528, such program modules 542 including, but not limited to, an operating system, one or more application programs, other program modules, and program data, each of which or some combination may include an implementation of a network environment. The program modules 542 generally perform the functions and / or methods of the embodiments described herein.

[0118] The computer device 512 may also communicate with one or more external devices 514 (e.g., keyboard, pointing device, display 524, etc.), may also communicate with one or more devices that enable a user to interact with the computer device 512, and / or communicate with any device that enables the computer device 512 to communicate with one or more other computing devices (e.g., network card, modem, etc.). Such communication may be performed via an input / output (I / O) interface 522. In addition, the computer device 512 may also communicate with one or more networks (e.g., local area network (LAN), wide area network (WAN) and / or public network, such as the Internet) via a network adapter 520. As shown, the network adapter 520 communicates with other modules of the computer device 512 via a bus 518. It should be understood that, although not shown in the figure, other hardware and / or software modules may be used in conjunction with the computer device 512, including but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data backup storage systems.

[0119] The processor 516 executes various functional applications and data processing by running the program stored in the system memory 528, for example, implementing the method for determining a session response solution provided in an embodiment of the present invention, which includes:

[0120] Obtaining the to-be-answered session information contained in the detected session answer request and the question-answer pair in the question-answer library;

[0121] Determine a first matching result between the unanswered session information and the question in the question-answer pair, and determine a second matching result between the unanswered session information and the answer in the question-answer pair, wherein the first matching result is used to characterize the matching degree between the unanswered session information and the question in the question-answer pair, and the second matching result is used to characterize the matching degree between the unanswered session information and the answer in the question-answer pair;

[0122] A target matching result of the to-be-answered session information and the question-answer pair is determined according to the first matching result and the second matching result, and a response solution for the session response request is determined according to the target matching result.

[0123] Of course, those skilled in the art can understand that the processor can also implement the technical solution of the method for determining a session response solution provided by any embodiment of the present invention.

[0124] Embodiment 6

[0125] Embodiment 6 of the present invention further provides a computer-readable storage medium on which a computer program is stored. When the program is executed by a processor, the method for determining a session response scheme provided in the embodiment of the present invention is implemented. The method includes:

[0126] Obtaining the to-be-answered session information contained in the detected session answer request and the question-answer pair in the question-answer library;

[0127] Determine a first matching result between the unanswered session information and the question in the question-answer pair, and determine a second matching result between the unanswered session information and the answer in the question-answer pair, wherein the first matching result is used to characterize the matching degree between the unanswered session information and the question in the question-answer pair, and the second matching result is used to characterize the matching degree between the unanswered session information and the answer in the question-answer pair;

[0128] A target matching result of the to-be-answered session information and the question-answer pair is determined according to the first matching result and the second matching result, and a response solution for the session response request is determined according to the target matching result.

[0129] Of course, the computer program stored in a computer-readable storage medium provided by an embodiment of the present invention is not limited to the method operations described above, but can also execute related operations in the method for determining a session response solution provided by any embodiment of the present invention.

[0130] The computer storage medium of the embodiment of the present invention can adopt any combination of one or more computer-readable media. The computer-readable medium can be a computer-readable signal medium or a computer-readable storage medium. The computer-readable storage medium can be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples (non-exhaustive list) of computer-readable storage media include: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In this document, a computer-readable storage medium can be any tangible medium containing or storing a program, which can be used by an instruction execution system, device or device or used in combination with it.

[0131] Computer-readable signal media may include data signals propagated in baseband or as part of a carrier wave, which carry computer-readable program code. Such propagated data signals may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. Computer-readable signal media may also be any computer-readable medium other than a computer-readable storage medium, which may send, propagate, or transmit a program for use by or in conjunction with an instruction execution system, apparatus, or device.

[0132] Program code embodied on a computer readable medium may be transmitted using any appropriate medium, including but not limited to wireless, wireline, optical fiber cable, RF, etc., or any suitable combination of the foregoing.

[0133] Computer program code for performing the operations of the present invention may be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, C++, and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a separate software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0134] Note that the above are only preferred embodiments of the present invention and the technical principles used. Those skilled in the art will understand that the present invention is not limited to the specific embodiments described herein, and that various obvious changes, readjustments and substitutions can be made by those skilled in the art without departing from the scope of protection of the present invention. Therefore, although the present invention has been described in more detail through the above embodiments, the present invention is not limited to the above embodiments, and may include more other equivalent embodiments without departing from the concept of the present invention, and the scope of the present invention is determined by the scope of the appended claims.

Claims

1. A method for determining a conversation response scheme, It is characterized in that include: Obtaining the to-be-answered session information contained in the detected session answer request and the question-answer pair in the question-answer library; Determine a first matching result between the unanswered session information and the question in the question-answer pair, and determine a second matching result between the unanswered session information and the answer in the question-answer pair, wherein the first matching result is used to characterize the matching degree between the unanswered session information and the question in the question-answer pair, and the second matching result is used to characterize the matching degree between the unanswered session information and the answer in the question-answer pair; Determine a target matching result of the to-be-answered session information and the question-answer pair according to the first matching result and the second matching result, and determine a response solution for the session response request according to the target matching result; After determining a second matching result between the to-be-answered conversation information and the answer in the question-answer pair, the method further includes: Obtaining a standard matching result of the question and the answer in the question-answer pair, and calibrating the second matching result according to the standard matching result to obtain a second calibrated matching result; Correspondingly, determining a target matching result of the unanswered conversation information and the question-answer pair according to the first matching result and the second matching result includes: A target matching result between the to-be-answered conversation information and the question-answer pair is determined according to the first matching result and the second calibration matching result.

2. The method according to claim 1, It is characterized in that The determining a first matching result between the to-be-answered conversation information and the question in the question-answer pair includes: The to-be-answered conversation information and the question in the question-answer pair are input into a trained question matching model to obtain a first matching result output by the question matching model.

3. The method according to claim 2, It is characterized in that The question matching model includes a semantic matching model and a feature fusion model, the semantic matching model is obtained by training based on the attention model, and the question in the question-answer pair and the conversation information to be answered are input into the trained question matching model to obtain a first matching result output by the question matching model, including: Inputting the unanswered conversation information and the question into the trained semantic matching model to obtain a semantic matching result output by the semantic matching model; the semantic matching result is a result of semantic matching between the unanswered conversation information and the question; The matching features between the to-be-answered conversation information and the question and the semantic matching result are input into a trained feature fusion model to obtain a first matching result output by the feature fusion model.

4. The method according to claim 1, It is characterized in that The obtaining of the standard matching result of the question and the answer in the question-answer pair includes: The question and answer are input into the trained question-answer matching model to obtain a standard matching result output by the question-answer matching model.

5. The method according to claim 1, It is characterized in that The step of calibrating the second matching result according to the standard matching result to obtain a second calibrated matching result includes: determining a ratio between the second matching result and the standard matching result; The ratio is compared with a preset threshold, and the smaller value between the ratio and the threshold is taken as the second calibration matching result.

6. The method according to claim 1, It is characterized in that The determining a target matching result between the unanswered conversation information and the question answer pair according to the first matching result and the second matching result includes: The first matching result and the second matching result are weightedly summed to obtain the target matching result.

7. The method according to claim 1, It is characterized in that The step of obtaining the to-be-answered session information contained in the detected session answer request includes: Obtaining the session information contained in the detected session response request; The session information is standardized to obtain the unanswered session information, wherein the standardized processing includes at least one of traditional Chinese character conversion, special symbol conversion, character segmentation, stop word removal, and information completion.

8. The method according to claim 1, It is characterized in that The determining a second matching result between the to-be-answered conversation information and the answer in the question-answer pair includes: The unanswered conversation information and the answer in the question-answer pair are input into a trained answer matching model to obtain a second matching result output by the answer matching model.

9. A device for determining a conversation response scheme, It is characterized in that include: A matching parameter acquisition module is used to obtain the to-be-answered session information contained in the detected session answer request and the question-answer pair in the question-answer library; a matching result acquisition module, configured to determine a first matching result between the unanswered session information and the question in the question-answer pair, and to determine a second matching result between the unanswered session information and the answer in the question-answer pair, wherein the first matching result is used to characterize the matching degree between the unanswered session information and the question in the question-answer pair, and the second matching result is used to characterize the matching degree between the unanswered session information and the answer in the question-answer pair; a response result determination module, configured to determine a target matching result of the to-be-answered session information and the question-answer pair according to the first matching result and the second matching result, and to determine a response solution for the session response request according to the target matching result; The device also includes a second result calibration module, which is used to: After determining a second matching result between the to-be-answered conversation information and the answer in the question-answer pair, obtaining a standard matching result of the question and the answer in the question-answer pair, and calibrating the second matching result according to the standard matching result to obtain a second calibrated matching result; Accordingly, the response result determination module is used to: A target matching result between the to-be-answered conversation information and the question-answer pair is determined according to the first matching result and the second calibration matching result.

10. A computer device, It is characterized in that The device comprises: one or more processors; A storage device for storing one or more programs; When the one or more programs are executed by the one or more processors, the one or more processors implement the conversation response solution determination method as described in any one of claims 1-8.

11. A computer-readable storage medium having a computer program stored thereon, It is characterized in that When the program is executed by a processor, the method for determining a conversation response scheme as described in any one of claims 1 to 8 is implemented.