A method for achieving accurate and fast scoring of questions and answers in intelligent customer service

Abstract

The embodiment of the present invention provides a method for achieving accurate and fast scoring of questions and answers on intelligent customer service, including the following steps: establishing a knowledge route associated with a knowledge base, matching the corresponding knowledge route according to the characteristics of the question, the knowledge route including a question and answer template, a search engine, and a semantic analysis; segmenting the question through a word segmenter hanlp to obtain segmented words, and matching the segmented words with the routing keywords of the target knowledge base according to the corresponding knowledge route; obtaining the target answer corresponding to the question from the target knowledge base based on the final score obtained by the full-text index score, the semantic similarity score, the word class weighted score, and the matching frequency score, and returning the target answer to the user. The embodiment of the present invention obtains the final score through the full-text index score, the semantic similarity score, the word class weighted score, and the matching frequency score to match the most appropriate answer, thereby achieving accurate answers and fast scoring.

Description

Technical Field

[0001] The present invention relates to the technical field of intelligent customer service, and more specifically, to a method for implementing accurate and rapid scoring of questions and answers on intelligent customer service. Background Art

[0002] With the continuous development of Internet financial technology, more and more technologies are being applied in the financial field, among which intelligent customer service in the financial field involves many technological applications.

[0003] At present, there are three main modes for implementing the question-and-answer function of intelligent customer service. One is to find the corresponding matching question-and-answer template according to the format of the questioner's question, and then return the question. This method is efficient, but it is necessary to maintain this question format template all the time. Once the sentence of the question is not within the template range, the answer cannot be matched. The second method is to use the search engine to first segment the sentence, and then save the keywords. When the user asks a question, the user's question is segmented and then matched through the search engine. This method is efficient, but the accuracy is low. Especially when the knowledge base expands, it is easy to give irrelevant answers. The third method is to use semantic analysis to answer questions. This method is much more flexible and can match the answer to the question through semantic analysis and vocabulary training. However, this method has high performance requirements, low efficiency, and is easy to "answer irrelevant questions". For intelligent customer service, it is more important to match the answer the user needs. Accurately and quickly returning the answer the user needs through matching scoring is the most important thing for intelligent customer service.

[0004] The three question-and-answer modes each have their own shortcomings. If they are combined, they will appear more humane. Therefore, based on the existing technology, it is necessary to achieve accurate and fast scoring of questions and answers in intelligent customer service, which is the trend of IoT modernization.

[0005] SUMMARY OF THE INVENTION

[0006] In order to overcome the shortcomings of the prior art, the present invention provides a method for realizing accurate and fast scoring of questions and answers in intelligent customer service, so as to improve the ability of users to quickly match answers to questions raised in intelligent customer service and accurately match answers to questions.

[0007] The technical solution adopted by the present invention to solve its technical problems is: a method for realizing accurate and fast scoring of questions and answers on intelligent customer service, comprising the following steps: establishing a knowledge route associated with a knowledge base, matching the corresponding knowledge route according to the characteristics of the question, the knowledge route including a question and answer template, a search engine and a semantic analysis; segmenting the question through a word segmenter hanlp to obtain segmented words, and matching the segmented words with the routing keywords of the target knowledge base according to the corresponding knowledge route; obtaining a target answer corresponding to the question from the target knowledge base based on the final score obtained by the full-text index score, the semantic similarity score, the word class weighted score and the matching frequency score, and returning the target answer to the user.

[0008] Preferably, the step of obtaining a target answer corresponding to the question from the target knowledge base based on the final score obtained by the full-text index score, the semantic similarity score, the word class weighted score and the matching frequency score includes:

[0009] The segmented words are matched with the 6 highest-scoring matching results through the BM25 algorithm of the ElastcSearch full-text index;

[0010] Get the full-text index scores corresponding to the top 6 matching results (out of 20 points);

[0011] Score(Q,d)=SUM(Wi*R(qi,d))Wi=IDF(qi)=log((Nn(qi)+0.5) / (n(qi)+0.5))

[0012] R(qi,d)=fi(k1+1) / (fi+K)K=k1*(1-b+b*(dl / avg(dl)))

[0013] Where: suppose the string after word segmentation is qi = q1, q2, q3, ..., qn, N is the total number of documents in the index, n(qi) is the number of documents containing the word segmentation qi, d is the search result, Wi is the correlation weight of the match between a qi word segmentation and the index document, k1 and b are adjustable parameters of the algorithm, dl is the length of the index document d, and avgdl is the average length of all texts in the index text set D.

[0014] Preferably, the step of obtaining a target answer corresponding to the question from the target knowledge base based on the final score obtained by the full-text index score, the semantic similarity score, the word class weighted score and the matching frequency score includes:

[0015] Use the Python synonyms framework to calculate the semantic similarity of the segmented words and find the 6 matching results with the highest matching scores;

[0016] Get the semantic similarity matching scores corresponding to the top 6 matching results (maximum score is 20 points);

[0017] d1=max(compare(a1,b1),compare(a1,b2),…compare(a1,bm));

[0018] d2=max(compare(a2,b1),compare(a2,b2),…compare(a2,bm));

[0019] …

[0020] dn=max(compare(an,b1),compare(an,b2),…compare(an,bm));

[0021] The semantic similarity matching score = avg (d1, d2, ..., dn);

[0022] Among them: the word set after word segmentation: Wi = {a1, a2, ..., an}, the word set of results matching the search engine: Wj = {b1, b2, ..., bm}, compare (a, b) represents the distance between word a and word b, the value range is [0, 1], d represents the distance between words.

[0023] Preferably, the step of obtaining a target answer corresponding to the question from the target knowledge base based on the final score obtained by the full-text index score, the semantic similarity score, the word class weighted score and the matching frequency score includes:

[0024] Use the special(a) function to calculate the 6 matching results with the highest matching scores for the segmented words;

[0025] Get the word class weighted scores of the top 6 matching results;

[0026]

[0027] The semantic similarity matching score = avg(s1, s2, ..., sn);

[0028] Among them: s is the weighted score of the word class, the word set after word segmentation: Wi = {a1, a2, ..., an}, special(a) is a specific noun scoring function, which is managed by the background management system. A score will be set for a specific noun, and the score will be obtained through the special(a) function.

[0029] Preferably, the step of obtaining a target answer corresponding to the question from the target knowledge base based on the final score obtained by the full-text index score, the semantic similarity score, the word class weighted score and the matching frequency score includes:

[0030] According to 20*(matching frequency of the question in the database / highest matching frequency of the current database), the 6 matching results with the highest matching frequency scores are obtained;

[0031] The matching frequency scores of the top 6 matching results are obtained.

[0032] Preferably, the final score calculation method is:

[0033] Final score = w1*full-text index score + w2*semantic similarity score + w3*word class weighted score + w4*matching frequency score;

[0034] Among them: w1, w2, w3, w4 are the weighted values ​​of the four types of scores, the initialization value is 0.5, the value range is [0, 1], it is a configurable value, and the system administrator can adjust it according to the actual question-answer matching results.

[0035] Preferably, obtaining a target answer corresponding to the question from the target knowledge base based on the final score obtained by the full-text index score, the semantic similarity score, the word class weighted score and the matching frequency score includes:

[0036] When a user asks a question, if the greeting database is matched and there is no matching result in the greeting database, the matching result is obtained through a third-party API interface; or

[0037] When a user asks a question, if the greeting database is matched and there is no matching result in the greeting database, the matching result is calculated through the neural network algorithm.

[0038] Preferably, matching the corresponding knowledge routing according to the characteristics of the problem includes:

[0039] The number of identical elements between the routing keyword set Ki = {k1, k2, ... kn} and the word set W = {a1, a2, ..., am} is found to be di.

[0040] Preferably, matching the corresponding knowledge routing according to the characteristics of the problem includes:

[0041] Get the maximum value max(di) and determine the knowledge base route with the highest matching degree.

[0042] Preferably, the matching of the segmented words with the routing keywords of the target knowledge base according to the corresponding knowledge routing includes:

[0043] According to the corresponding knowledge routing, the corresponding word segmentation strategy is set so that the segmented words are matched with the routing keywords of the target knowledge base.

[0044] The beneficial effects of the present invention are: a final score is obtained through corresponding full-text index score, semantic similarity score, word class weighted score and matching frequency score, thereby improving the accuracy and efficiency of question matching answers. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] Figure 1 The present invention is a flowchart of a method for realizing accurate and fast scoring of questions and answers in intelligent customer service.

[0046] Figure 2 The present invention is a scoring diagram of a method for achieving accurate and fast scoring of questions and answers in intelligent customer service. DETAILED DESCRIPTION

[0047] The present invention is further described below in conjunction with the accompanying drawings and embodiments.

[0048] The following will clearly and completely describe the concept, specific structure and technical effects of the present invention in combination with the embodiments and drawings, so as to fully understand the purpose, characteristics and effects of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of them. Based on the embodiments of the present invention, other embodiments obtained by technicians in this field without creative work are all within the scope of protection of the present invention. In addition, all the connection / connection relationships involved in the patent do not refer to the direct connection of components, but refer to the formation of a better connection structure by adding or reducing connection accessories according to the specific implementation situation. The various technical features in the invention can be combined interchangeably without conflicting with each other.

[0049] Reference Figure 1

[0050] S101, establishing a knowledge route associated with a knowledge base, matching a corresponding knowledge route according to the characteristics of a question, wherein the knowledge route includes a question-answering template, a search engine, and semantic analysis;

[0051] Establish knowledge routing associated with the knowledge base, which includes a greeting base, a question-and-answer base, a business base, etc. Match the corresponding knowledge routing according to the characteristics of the question. The knowledge routing includes a question-and-answer template, a search engine, and semantic analysis. When a user asks a question, for example, what are the faults of the smart POS machine? If the question is in the question-and-answer template, the answer is matched through the knowledge routing path; if the question is retrieved through a search engine, the answer is matched through the word segmentation strategy; if the question is retrieved through semantic analysis, it is necessary to perform semantic analysis on the question, find the answer corresponding to the semantics similar to or the same as the question, and match the answer with the knowledge base, providing multiple ways to find the corresponding answer to improve the retrieval efficiency.

[0052] S102, segmenting the question by a word segmenter hanlp to obtain segmented words, and matching the segmented words with routing keywords of a target knowledge base according to corresponding knowledge routing;

[0053] When a user asks "What faults have occurred with the smart POS?", when the question-answer template and semantic analysis knowledge routes cannot retrieve the answer, the question is retrieved through a search engine, and then the question is segmented through the word segmenter hanlp to obtain the segmented words, the question is divided into words of different parts of speech, and the corresponding knowledge base is searched and matched. The system administrator summarizes the business questions and answers into question-answer mapping relationships, builds the knowledge base in advance, and trains the knowledge base to obtain the trained knowledge base, and matches the corresponding knowledge base according to the attributes of the question, so as to quickly match the corresponding knowledge base and improve the efficiency of searching the knowledge base.

[0054] S103, obtaining a target answer corresponding to the question from the target knowledge base based on the final score obtained by the full-text index score, the semantic similarity score, the word class weighted score and the matching frequency score, and returning the target answer to the user.

[0055] Scoring is performed through full-text indexing, semantic similarity, word class weighting and matching frequency, and the final score is obtained. The target answer corresponding to the question is obtained from the target knowledge base and the target answer is returned to the user. When the user asks a question, if it is matched to the greeting library and there is no matching result in the greeting library, the matching result is obtained through the third-party API interface; or when the user asks a question, if it is matched to the greeting library and there is no matching result in the greeting library, the matching result is calculated through the neural network algorithm, thereby achieving accuracy and efficiency in matching scoring.

[0056] Reference Figure 2

[0057] S201, full-text index scoring

[0058] The segmented words are matched with the 6 highest-scoring matching results through the BM25 algorithm of the ElastcSearch full-text index;

[0059] Get the search engine matching scores corresponding to the top 6 matching results (out of 20 points);

[0060] Score(Q,d)=SUM(Wi*R(qi,d))Wi=IDF(qi)=log((Nn(qi)+0.5) / (n(qi)+0.5))R(qi,d)=fi(k1+1) / (fi+K)K=k1*(1-b+b*(dl / avg(dl)))

[0061] Among them: Let the string after word segmentation be qi = q1, q2, q3, ..., qn, N is the total number of documents in the index, n(qi) is the number of documents containing the word segmentation qi, d is the search result, Wi is the relevance weight of the match between a qi word segmentation and an index document, k1 and b are adjustable parameters of the algorithm, dl, avgdl are the length of the index document d and the average length of all texts in the index text set D respectively. The search engine scoring method eliminates most of the problems that are basically impossible to be related in semantics, and improves the efficiency of retrieving words after word segmentation.

[0062] S202, semantic similarity scoring: the words after segmentation are matched with the 6 matching results with the highest semantic similarity calculation using the Python synonyms framework;

[0063] Get the semantic similarity matching scores corresponding to the top 6 matching results (maximum score is 20 points).

[0064] d1=max(compare(a1,b1),compare(a1,b2),…compare(a1,bm));

[0065] d2=max(compare(a2,b1),compare(a2,b2),…compare(a2,bm));

[0066] …

[0067] dn=max(compare(an,b1),compare(an,b2),…compare(an,bm));

[0068] Semantic similarity matching score = avg(d1, d2, …, dn);

[0069] Among them: the word set after word segmentation: Wi = {a1, a2, ..., an}, the word set of the results matching the search engine:

[0070] Wj={b1,b2,…,bm}, compare(a,b) represents the distance between word a and word b, the value range is [0,1], d represents the distance between words.

[0071] S203, weighted scoring of word classes

[0072] Use the special(a) function to calculate the 6 matching results with the highest matching scores for the segmented words;

[0073] Get the word-part-of-speech weighted scores of the top 6 matching results.

[0074] Get the word segmentation results of the 6 matching results and the word segmentation results of the question, and perform weighted scoring of word classes according to specific nouns (16-20 points) > common nouns (15 points) > verbs (10 points) > other words (5 points). Finally, get an average weighted word class score. Calculate the distance between Wi and Wj through the synonym framework synonyms. The weighted word class score calculation method is:

[0075]

[0076] Semantic similarity matching score = avg(s1,s2,…,sn);

[0077] Among them: s is the weighted score of the word class, the word set after word segmentation: Wi = {a1, a2, ..., an}, special(a) is a specific noun scoring function, which is managed by the background management system. A score will be set for a specific noun, and the score will be obtained through the special(a) function.

[0078] S204, Matching frequency score

[0079] Get the matching frequency of the 6 matching results in the database, and calculate the matching frequency score by 20*(matching frequency of the question in the database / highest matching frequency of the current database). The higher the matching frequency of the corresponding question, the higher the score. This way, the probability of matching the answer the user needs is increased.

[0080] S205, Final Rating

[0081] Final score = w1*full-text index score + w2*semantic similarity score + w3*word class weighted score + w4*matching frequency score.

[0082] Among them: w1, w2, w3, w4 are the weighted values ​​of the four types of scores, and the initialization value is 0.5. The value range is [0, 1]. It is a configurable value and can be adjusted by the system administrator according to the actual question-answer matching results. For example, if the system administrator thinks that the search engine matching score is more important for answer matching, then the value of w1 is adjusted, or the values ​​of w2, w3, and w4 are lowered accordingly; or if the system administrator thinks that the semantic similarity score is more important for answer matching, then the value of w2 is adjusted, or the values ​​of w1, w3, and w4 are lowered accordingly, and so on.

[0083] In the embodiment of the present application, the final score is obtained through full-text index score, semantic similarity score, word class weighted score and matching frequency score to match the most appropriate answer, thereby improving the accuracy and efficiency of question matching answers.

[0084] The principles of knowledge base routing and word segmentation:

[0085] Suppose the routing keyword set of knowledge base i is Ki = {k1, k2, ... kn}, and the word set after word segmentation is: W = {a1, a2, ..., am}. The number of identical elements between the routing keyword set Ki and the word set W is calculated as di, and the maximum value max(di) is obtained to determine the knowledge base route with the highest matching degree. If multiple knowledge base routes have the same score, the final selected knowledge base route is determined according to the weight value of the knowledge base route configured in the background. Generally, the problem is first partially matched with keywords, and then the first score is performed. The knowledge base with the highest score is selected for subsequent matching. On the one hand, the number of subsequent matching is reduced and the retrieval efficiency is improved. On the other hand, the matching of knowledge irrelevant to the problem is reduced, thereby improving the matching accuracy.

[0086] Different word segmentation strategies are adopted according to the corresponding knowledge routing.

[0087] For example, the greeting database strategy has the widest word segmentation range. In addition to nouns, verbs, directional words, etc., other words such as modal particles are also involved in the word segmentation, while word groups, articles, and punctuation marks are not involved in the word segmentation, for example: Are you there, hello?; the question and answer strategy only segments nouns and verbs, for example: the smart POS machine has a problem; the business strategy only segments nouns, for example: the smart POS machine; the system adopts different word segmentation strategies according to the different types of knowledge bases, one part is part of speech, and the other part is weighted words. Different word segmentation strategies can provide greater flexibility and accuracy for subsequent scoring.

[0088] The above is a specific description of the preferred implementation of the present invention, but the invention is not limited to the embodiments. Those skilled in the art can make various equivalent modifications or substitutions without violating the spirit of the present invention. These equivalent modifications or substitutions are all included in the scope defined by the claims of this application.

Claims

1. A method for achieving accurate and rapid scoring of questions and answers on intelligent customer service, characterized in that: The following steps are involved: Establishing a knowledge route associated with the knowledge base, matching the corresponding knowledge route according to the characteristics of the question, wherein the knowledge route includes a question-answering template, a search engine, and semantic analysis; The question is segmented by the word segmenter hanlp to obtain segmented words, and the segmented words are matched with the routing keywords of the target knowledge base according to the corresponding knowledge routing; The final score obtained based on the full-text index score, semantic similarity score, word class weighted score and matching frequency score obtains the target answer corresponding to the question from the target knowledge base and returns the target answer to the user.

2. A method for achieving accurate and rapid scoring of questions and answers on intelligent customer service according to claim 1, characterized in that: The step of obtaining a target answer corresponding to the question from the target knowledge base based on the final score obtained by the full-text index score, the semantic similarity score, the word class weighted score and the matching frequency score includes: The segmented words are matched with the 6 highest-scoring matching results through the BM25 algorithm of the ElastcSearch full-text index; Get the full-text index scores corresponding to the top 6 matching results, where the full score is 20 points; Score(Q,d)=SUM(Wi*R(qi,d))Wi=IDF(qi)=log((Nn(qi)+0.5) / (n(qi)+0.5)) R(qi,d)=fi(k1+1) / (fi+K)K=k1*(1-b+b*(dl / avg(dl))) Where: suppose the string after word segmentation is qi = q1, q2, q3, ..., qn, N is the total number of documents in the index, n(qi) is the number of documents containing the word segmentation qi, d is the search result, Wi is the correlation weight of the match between a qi word segmentation and the index document, k1 and b are adjustable parameters of the algorithm, dl is the length of the index document d, and avg(dl) is the average length of all texts in the index text set D.

3. A method for achieving accurate and rapid scoring of questions and answers on intelligent customer service according to claim 1, characterized in that: The step of obtaining a target answer corresponding to the question from the target knowledge base based on the final score obtained by the full-text index score, the semantic similarity score, the word class weighted score and the matching frequency score includes: Use the Python synonyms framework to calculate the semantic similarity of the segmented words and find the 6 matching results with the highest matching scores; Get the semantic similarity matching scores corresponding to the top 6 matching results, with a full score of 20 points; d1=max(compare(a1,b1),compare(a1,b2),…compare(a1,bm)); d2=max(compare(a2,b1),compare(a2,b2),…compare(a2,bm)); … dn=max(compare(an,b1),compare(an,b2),…compare(an,bm)); The semantic similarity matching score = avg (d1, d2, ..., dn); Among them: the word set after word segmentation: Wi = {a1, a2, ..., an}, the word set of results matching the search engine: Wj = {b1, b2, ..., bm}, compare (a, b) represents the distance between word a and word b, the value range is [0, 1], d represents the distance between words.

4. A method for achieving accurate and rapid scoring of questions and answers on intelligent customer service according to claim 1, characterized in that: The step of obtaining a target answer corresponding to the question from the target knowledge base based on the final score obtained by the full-text index score, the semantic similarity score, the word class weighted score and the matching frequency score includes: Use the special(a) function to calculate the 6 matching results with the highest matching scores for the segmented words; Get the word class weighted scores of the top 6 matching results; The semantic similarity matching score = avg(s1, s2, ..., sn); Among them: s is the weighted score of the word class, the word set after word segmentation: Wi = {a1, a2, ..., an}, special(a) is a specific noun scoring function, which is managed by the background management system. A score will be set for a specific noun, and the score will be obtained through the special(a) function.

5. A method for achieving accurate and rapid scoring of questions and answers on intelligent customer service according to claim 1, characterized in that: The step of obtaining a target answer corresponding to the question from the target knowledge base based on the final score obtained by the full-text index score, the semantic similarity score, the word class weighted score and the matching frequency score includes: According to 20*(matching frequency of the question in the database / highest matching frequency of the current database), the 6 matching results with the highest matching frequency scores are obtained; The matching frequency scores of the top 6 matching results are obtained.

6. A method for implementing accurate and rapid scoring of questions and answers on intelligent customer service according to any one of claims 2 to 5, characterized in that: Final score calculation method: Final score = w1*full-text index score + w2*semantic similarity score + w3*word class weighted score + w4*matching frequency score; Among them: w1, w2, w3, w4 are the weighted values ​​of the four types of scores, the initialization value is 0.5, the value range is [0, 1], it is a configurable value, and the system administrator can adjust it according to the actual question-answer matching results.

7. A method for achieving accurate and rapid scoring of questions and answers on intelligent customer service according to claim 1, characterized in that: The final score based on the full-text index score, semantic similarity score, word class weighted score and matching frequency score is used to obtain the target answer corresponding to the question from the target knowledge base, including: When a user asks a question, if the greeting database is matched and there is no matching result in the greeting database, the matching result is obtained through a third-party API interface; or When a user asks a question, if the greeting database is matched and there is no matching result in the greeting database, the matching result is calculated through the neural network algorithm.

8. A method for achieving accurate and rapid scoring of questions and answers on intelligent customer service according to claim 1, characterized in that: The knowledge routing corresponding to the problem feature matching includes: The number of identical elements between the routing keyword set Ki = {k1, k2, ... kn} and the word set W = {a1, a2, ..., am} is found to be di.

9. A method for achieving accurate and rapid scoring of questions and answers on intelligent customer service according to claim 1, characterized in that: The knowledge routing corresponding to the problem feature matching includes: Get the maximum value max(di) and determine the knowledge base route with the highest matching degree.

10. A method for achieving accurate and rapid scoring of questions and answers on intelligent customer service according to claim 1, characterized in that: The step of matching the segmented words with the routing keywords of the target knowledge base according to the corresponding knowledge routing includes: According to the corresponding knowledge routing, the corresponding word segmentation strategy is set so that the segmented words are matched with the routing keywords of the target knowledge base.

Images