Question and answer processing method and device, electronic equipment, storage medium and computer product

Abstract

The invention relates to the technical field of Internet, and provides a question and answer processing method and device, electronic equipment, a storage medium and a computer product, and the method comprises the steps: determining a keyword list based on an input text; according to the keyword list and a keyword set corresponding to each preset question in a preset knowledge base, determining a matching score between the input text and each preset question; respectively inputting the input text and each preset question into a word embedding model to obtain text vectors respectively output by the word embedding model; determining a similarity score between a text vector of the input text and a text vector corresponding to each preset question; based on each matching score and each similarity score, determining a target question from each preset question; and determining reply content of the input text based on the target question. According to the method and the device, the semantic complexity can be captured, the retrieval accuracy is improved, and the session processing accuracy of the intelligent question and answer is further improved.

Description

Technical Field

[0001] The present application relates to the field of Internet technology, and in particular to a question-answering processing method, device, electronic device, storage medium, and computer product. Background Art

[0002] Intelligent question-answering mainly refers to automatically returning a reasonable answer in the form of a dialogue based on a user's question. Existing intelligent question-answering methods often use knowledge base retrieval plus language model summarization to generate answers. The key to this type of method lies in the accuracy of retrieval between the user's question and the standard answer in the knowledge base. At present, the retrieval method in the intelligent question-answering process mainly relies on converting the user's question into an encoding, calculating the similarity with the answer vector in the pre-encoded knowledge base, and returning the most similar answer (with the highest similarity score).

[0003] However, the most similar answer obtained by calculating the similarity between the encoding converted from the user's question and the answer vector in the pre-encoded knowledge base cannot fully capture the complexity of semantics, resulting in insufficient retrieval accuracy, which in turn leads to insufficient accuracy in the current intelligent question and answer session processing. Summary of the invention

[0004] The present application aims to solve at least one of the technical problems existing in the related art. To this end, the present application proposes a question-answering processing method, device, electronic device, storage medium and computer product to solve the problem that the current answer retrieval for question-answering cannot fully capture the complexity of semantics, resulting in insufficient retrieval accuracy, and to improve the accuracy of conversation processing of intelligent question-answering.

[0005] The question-answer processing method according to the first aspect of the present application includes: Determining a list of keywords based on the input text; Determine the matching scores between the input text and each preset question according to the keyword list and the keyword sets corresponding to each preset question in the preset knowledge base; Inputting the input text and each of the preset questions into a word embedding model respectively, to obtain text vectors output by the word embedding model respectively; Determine the similarity scores between the text vector of the input text and the text vector corresponding to each preset question; Determining a target question from among the preset questions based on the matching scores and the similarity scores; The answer content of the input text is determined based on the target question.

[0006] According to an embodiment of the present application, determining the answer content of the input text based on the target question includes: If there are multiple target questions, determine the common ancestor node of each target question from a concept tree; wherein the concept tree is a tree structure constructed based on multiple preset questions, keywords corresponding to each preset question, and weights of the keywords corresponding to each preset question; Pruning the concept tree based on the common ancestor node to obtain a common ancestor child node list; Determine and output a follow-up question based on the common ancestor child node list; receiving supplementary text returned based on the follow-up question, and determining a target child node from the common ancestor child node list based on the supplementary text; The answer content of the target sub-node corresponding to the preset question is used as the answer content of the input text.

[0007] According to one embodiment of the present application, determining the target child node from the common ancestor child node list based on the supplementary text includes: determining a keyword set for the supplementary text; Perform keyword similarity matching between the supplementary text and each common ancestor subnode in the common ancestor subnode list; If the common ancestor child node with the highest similarity is a leaf node in the concept tree, the common ancestor child node with the highest similarity is determined as the target child node.

[0008] According to one embodiment of the present application, the concept tree is constructed based on the following steps: Perform hierarchical clustering based on multiple preset questions to obtain a cluster tree; Optimizing the clustering tree based on the large language model and the preset prompt words to obtain an optimized clustering tree; The keywords of the preset questions corresponding to each node in the optimized clustering tree and the weights of the keywords of the preset questions corresponding to each node are added to the corresponding nodes to obtain a concept tree.

[0009] According to an embodiment of the present application, when determining the matching scores between the input text and each preset question respectively according to the keyword list and the keyword set corresponding to each preset question in the preset knowledge base, the following steps are performed for each preset question respectively: Determine a keyword matching result of each keyword in the keyword list with a keyword set corresponding to the current preset question in the preset knowledge base; Determine the matching score of each keyword matching result respectively; Multiply each matching score by the weight of the corresponding keyword to obtain a weighted score; The weighted scores of the keywords are summed up to obtain a matching score between the input text and the current preset question.

[0010] According to an embodiment of the present application, determining a target question from each preset question based on each matching score and each similarity score includes: Multiply the matching score of each preset question by the first weight to obtain the first score corresponding to each preset question; Multiply the similarity score of each preset question by the second weight to obtain a second score corresponding to each preset question; Add the first score and the second score of each preset question to obtain the comprehensive score corresponding to each preset question; Based on the comprehensive scores, target questions are determined from among the preset questions.

[0011] According to the second aspect of the present application, a question and answer processing device includes: A first determination module, configured to determine a keyword list based on an input text; A second determination module is used to determine the matching scores between the input text and each preset question according to the keyword list and the keyword sets corresponding to each preset question in the preset knowledge base; An input module, used to input the input text and each of the preset questions into a word embedding model respectively, to obtain text vectors output by the word embedding model respectively; A third determination module is used to determine the similarity scores between the text vector of the input text and the text vector corresponding to each preset question; A fourth determination module, configured to determine a target question from among the preset questions based on the matching scores and the similarity scores; The fifth determination module is used to determine the answer content of the input text based on each target question.

[0012] According to an electronic device of an embodiment of the third aspect of the present application, the electronic device includes a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, the question and answer processing method described above is implemented.

[0013] According to the storage medium of the fourth aspect of the present application, the storage medium is a non-transitory computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, it implements any of the question and answer processing methods described above.

[0014] According to the computer program product of the fifth aspect of the present application, the computer program includes a computer program, which, when executed by a processor, implements any of the question and answer processing methods described above.

[0015] The above one or more technical solutions in the embodiments of the present application have at least the following technical effects: Based on the input text, a keyword list is determined, so that the matching scores between the input text and each preset question can be determined according to the keyword list and the keyword set corresponding to each preset question in the preset knowledge base; and the input text and each preset question are respectively input into the word embedding model to obtain the text vectors respectively output by the word embedding model, so that the similarity scores between the text vector of the input text and the text vector corresponding to each preset question can be determined; further, based on each matching score and each similarity score, the target question can be accurately determined from each preset question, and then the answer content of the input text can be accurately determined based on the target question. Due to the use of keyword matching scoring combined with artificial intelligence matching scoring, a more accurate standard question for the input text can be selected from multiple preset questions as standard questions. The multi-dimensional semantic similarity judgment standard makes the result more accurate, so that the complexity of semantics can be captured, the accuracy of retrieval can be improved, and then the accuracy of conversation processing of intelligent question and answer can be improved.

[0016] Additional aspects and advantages of the present application will be given in part in the description below, and in part will become apparent from the description below, or will be learned through the practice of the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] In order to more clearly illustrate the technical solutions in the present invention or the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0018] Figure 1 It is a flowchart of the question and answer processing method provided in an embodiment of the present application.

[0019] Figure 2 It is a schematic diagram of the overall flow of the question and answer processing method provided in the embodiment of the present application.

[0020] Figure 3 It is a structural schematic diagram of the electronic device provided by this application. DETAILED DESCRIPTION

[0021] The following is a further detailed description of the implementation of the present application in conjunction with the accompanying drawings and examples. The following examples are used to illustrate the present application but cannot be used to limit the scope of the present application.

[0022] In the description of the embodiments of the present application, it should be noted that the terms "center", "longitudinal", "lateral", "up", "down", "front", "back", "left", "right", "vertical", "horizontal", "top", "bottom", "inside", "outside" and the like indicate positions or positional relationships based on the positions or positional relationships shown in the accompanying drawings, which are only for the convenience of describing the embodiments of the present application and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore cannot be understood as limiting the embodiments of the present application. In addition, the terms "first", "second", and "third" are used for descriptive purposes only and cannot be understood as indicating or implying relative importance.

[0023] In the description of the embodiments of the present application, it should be noted that, unless otherwise clearly specified and limited, the terms "connected" and "connection" should be understood in a broad sense, for example, it can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be directly connected or indirectly connected through an intermediate medium. For ordinary technicians in this field, the specific meanings of the above terms in the embodiments of the present application can be understood according to specific circumstances.

[0024] In the embodiments of the present application, unless otherwise clearly specified and limited, a first feature being "above" or "below" a second feature may mean that the first and second features are in direct contact, or the first and second features are in indirect contact through an intermediate medium. Moreover, a first feature being "above", "above" or "above" a second feature may mean that the first feature is directly above or obliquely above the second feature, or simply means that the first feature is higher in level than the second feature. A first feature being "below", "below" or "below" a second feature may mean that the first feature is directly below or obliquely below the second feature, or simply means that the first feature is lower in level than the second feature.

[0025] In the description of this specification, the description with reference to the terms "one embodiment", "some embodiments", "example", "specific example", or "some examples" etc. means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the embodiments of the present application. In this specification, the schematic representations of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or more embodiments or examples in a suitable manner. In addition, those skilled in the art may combine and combine the different embodiments or examples described in this specification and the features of the different embodiments or examples, without contradiction.

[0026] The present application proposes a question-answering processing method, device, electronic device, storage medium and computer product.

[0027] Figure 1 is a flowchart of the question-answering processing method provided in the embodiment of the present application, such as Figure 1 As shown, the question-answering processing method includes: Step 110, determining a keyword list based on the input text.

[0028] Step 120, determining the matching scores between the input text and each preset question respectively according to the keyword list and the keyword sets corresponding to each preset question in the preset knowledge base.

[0029] Step 130, input the input text and each preset question into the word embedding model respectively, and obtain the text vectors output by the word embedding model respectively.

[0030] Step 140: Determine the similarity scores between the text vector of the input text and the text vector corresponding to each preset question.

[0031] Step 150 : determining a target question from the preset questions based on the matching scores and the similarity scores.

[0032] Step 160, determining the answer content of the input text based on each target question.

[0033] It should be noted that the execution subject of the question-answer processing method provided in the embodiment of the present application may be a server, a computer device, etc. The computer device may be, for example, a mobile phone, a tablet computer, a laptop computer, a PDA, an in-vehicle electronic device, a wearable device, an ultra-mobile personal computer (UMPC), a netbook, or a personal digital assistant (PDA), etc. It should be noted that the data required to be obtained in this application are all obtained through formal channels after being authorized by the relevant users.

[0034] A question and answer processing device may be provided or connected in the server or computer device of the present application, thereby controlling the question and answer processing device to execute the question and answer processing method of the present application.

[0035] This application can obtain original data and extract standard questions (which can be referred to as standard questions for short) from the original data as preset questions.

[0036] Specifically, the original data files are generally stored in Excel files, and the main information contained in them includes a standard question and the corresponding answer. First, delete the redundant columns in the file to leave the useful information. Then clean the standard questions. Cleaning mainly includes the splitting of standard questions: the semantics of some standard questions contain multiple questions, which need to be split into two questions and configured with answers. The second is the reconstruction of standard questions: some standard questions are too colloquial and need to be manually corrected, and some keywords are written down. The last cleaning is to remove duplicate standard questions. Since the original data may come from different regions, there will be questions that are very similar in semantics. In this case, only one standard question can be retained.

[0037] And, extract the standard answers to the standard questions as the answer content of the preset questions, and some standard questions need to be manually reviewed. Thus, a preset knowledge base can be constructed based on the extracted preset questions and their answer content. The present application can also generate a keyword set for each preset question in the preset knowledge base.

[0038] The question-answering processing method in this application can be applied to intelligent question-answering scenarios and can be deployed in intelligent question-answering systems, intelligent question-answering platforms, etc.

[0039] In the intelligent question-and-answer scenario, this application can obtain the user's input text.

[0040] Furthermore, the present application can extract keywords from the input text and form a keyword list from the extracted keywords.

[0041] Furthermore, the present application can perform text similarity calculation based on keyword matching for the keyword list and the keyword sets corresponding to each preset question in the preset knowledge base, and then obtain the matching score between the input text and each preset question.

[0042] In addition, the present application can obtain a word embedding model. The word embedding model in the present application can specifically be a pre-trained text encoding model BGE based on a transformer encoder (encoder), and Transformer is a deep learning model architecture.

[0043] BGE can map natural language into high-dimensional vectors for calculation. BGE training tasks include various text similarity matching calculations. In this application, there is no need to fine-tune the BGE model, and encoding and similarity calculations are performed directly.

[0044] Therefore, the application can input the input text and each preset question into the word embedding model respectively, encode them through the word embedding model, and then obtain the text vectors output by the word embedding model respectively. For example, the text vector of the input text is represented as v1, and the text vector corresponding to each preset question is represented as v2, and there are: , where the dimension of the vector can be 1024.

[0045] Furthermore, cosine similarity can be used to calculate the similarity of the feature vectors between the text vector of the input text and the text vectors corresponding to each preset question. The formula for cosine similarity is: .

[0046] Furthermore, each similarity score may be normalized, specifically, normalized to a range of [0, 1], to obtain a similarity score between the text vector of the input text and the text vector corresponding to each preset question.

[0047] Existing systems based on retrieval question-answering often use a method of pre-trained language model encoding plus vector similarity calculation to measure semantic similarity. This method is based on fine-tuning training on a specific domain knowledge set, but the encoding model is uninterpretable. If you want to optimize the encoding results, you must add new data and retrain, which is time-consuming and labor-intensive.

[0048] In this application, for each preset question, a comprehensive score can be determined by weighted summation of the matching score and similarity score corresponding to the preset question, and the preset questions (i.e., standard questions) are screened according to the comprehensive scores. Finally, the target question is determined from all the preset questions and a set of candidate standard questions is formed.

[0049] In terms of semantic similarity evaluation, this application adopts a dual evaluation strategy of artificial keyword features and pre-trained language model encoding. Without fine-tuning the model, only the keyword features can be modified to make the system suitable for tasks in different fields.

[0050] In the existing method, the interaction between the system and the user is that the user asks a question and the system gives an answer. If the question asked by the user is very vague, the system will not judge whether the answer with the highest similarity is the standard answer, but will directly return it. In this way, it is likely to return the wrong answer. Based on this, the application pre-constructs a tree structure based on multiple preset questions, the keywords corresponding to each preset question, and the weights of the keywords corresponding to each preset question, and determines it as a concept tree.

[0051] Furthermore, if there is only one set element in the candidate standard question set, and the standard question corresponding to the set element only exists in the leaf node of the concept tree, the standard question and the answer content are used as the answer content of the input text and returned to the user. If the candidate standard question set is an empty set, the manual question answering is transferred. In other cases, the questioning method based on the concept tree is used to lock the user's desired question.

[0052] According to the question-answer processing method of the embodiment of the present application, a keyword list is determined based on the input text, so that the matching scores between the input text and each preset question can be determined according to the keyword list and the keyword set corresponding to each preset question in the preset knowledge base; and the input text and each preset question are respectively input into the word embedding model to obtain the text vectors respectively output by the word embedding model, so that the similarity scores between the text vector of the input text and the text vector corresponding to each preset question can be determined; further, based on each matching score and each similarity score, the target question can be accurately determined from each preset question, and then the reply content of the input text can be accurately determined based on the target question. Due to the use of keyword matching scoring combined with artificial intelligence matching scoring, a more accurate standard question for the input text can be selected from multiple preset questions as standard questions, and the multi-dimensional semantic similarity evaluation criteria make the result more accurate, so that the complexity of semantics can be captured, the accuracy of retrieval can be improved, and then the accuracy of intelligent question-answering conversation processing can be improved.

[0053] Based on the above embodiment, when determining the matching scores between the input text and each preset question respectively according to the keyword list and the keyword set corresponding to each preset question in the preset knowledge base, the following steps are performed for each preset question respectively: Determine, for each keyword in the keyword list, a keyword matching result of a keyword set corresponding to a current preset question in a preset knowledge base; Determine the matching score of each keyword matching result respectively; Multiply each matching score by the weight of the corresponding keyword to obtain a weighted score; The weighted scores of each keyword are added together to obtain the matching score between the input text and the current preset question.

[0054] Specifically, the present application may record the keyword list as A, and record the keyword sets corresponding to each preset question in the preset knowledge base as Bn.

[0055] Based on this, a similarity score variable can be pre-specified for each standard question. . Loop through all standard questions. If the element a in A matches the element in the keyword set Bn of the standard question currently traversed, If the word itself or its corresponding synonym is the same, then 1 point is scored and multiplied by the weight of the corresponding keyword, and accumulated in S. If it is the same as a related word, then 0.5 points are scored and multiplied by the weight of the corresponding keyword, and accumulated in S. If no identical word is found, then 0 points are scored. The final S is the keyword similarity score between this standard question and the user input text and is used as the matching score, thereby obtaining the matching score between the input text and each preset question.

[0056] The present application performs score calculation based on a text similarity calculation method of keyword matching. Since it intuitively reflects the matching degree of keywords in the text, it is easy to understand and explain, and keyword matching can be combined with other more complex semantic understanding methods to improve the accuracy and robustness of similarity calculation.

[0057] Based on the above embodiment, determining a target question from each preset question based on each matching score and each similarity score includes: Multiply the matching score of each preset question by the first weight to obtain the first score corresponding to each preset question; Multiply the similarity score of each preset question by the second weight to obtain a second score corresponding to each preset question; Add the first score and the second score of each preset question to obtain the comprehensive score corresponding to each preset question; Based on the comprehensive scores, target questions are determined from among the preset questions.

[0058] Specifically, after obtaining the matching score S for each preset question 1 And the similarity score S 2 After that, the present application can determine the weight of the matching score as the first weight w 1 , and determine the weight of the similarity score as the second weight w 2 In one embodiment, the first weight may be 0.7 and the second weight may be 0.3.

[0059] Therefore, the comprehensive scores between the input text and each preset question can be calculated using the following formulas: .

[0060] Furthermore, the present application can set a threshold , delete the preset questions corresponding to the scores less than the threshold. Then set the threshold . Sort all remaining candidate preset questions in descending order If it exists , then discard And all the elements after it, finally get the candidate standard question set.

[0061] If there is only one element in the candidate standard question set, and the standard question corresponding to the element only exists in the leaf node of the concept tree, then the standard question is used as the target question, and the standard question and the answer content are used as the answer content of the input text and returned to the user. If the candidate standard question set is an empty set, then the manual question answering is transferred. In other cases, the questioning method based on the concept tree is used to lock the user's expected question.

[0062] This application uses keyword matching scoring combined with artificial intelligence matching scoring, and can select more accurate standard questions for the input text from multiple preset questions as standard questions. The multi-dimensional semantic similarity evaluation criteria make the results more accurate, so it can capture the complexity of semantics, improve the accuracy of retrieval, and thus improve the accuracy of intelligent question and answer conversation processing.

[0063] At the same time, this application introduces a series of customized scoring criteria and threshold criteria to determine the trade-offs in similarity scores and improve retrieval accuracy.

[0064] Based on the above embodiment, the concept tree is constructed based on the following steps: Perform hierarchical clustering based on multiple preset questions to obtain a cluster tree; The clustering tree is optimized based on the large language model and the preset prompt words to obtain an optimized clustering tree; The keywords of the preset questions corresponding to each node in the optimized clustering tree and the weights of the keywords of the preset questions corresponding to each node are added to the corresponding nodes to obtain a concept tree.

[0065] Specifically, the present application can obtain a clustering tree structure through hierarchical clustering, thereby constructing multiple preset questions into a clustering tree.

[0066] More specifically, hierarchical clustering is divided into two methods: agglomerative hierarchical clustering and divisive hierarchical clustering. Among them, agglomerative hierarchical clustering is a bottom-up clustering method that initially regards each data point as a separate cluster. As the algorithm runs, the nearest clusters are gradually merged until all clusters are merged into a large cluster or the predetermined number of clusters is reached. In this process, a hierarchical nested clustering tree is naturally formed. In the clustering tree, the original data points of different categories are the lowest level of the tree, and the top level of the tree is the root node of a cluster. Due to the large number of standard questions in the knowledge base, directly using a large model to generate a concept tree will result in the "large model illusion" problem, and other clustering methods cannot obtain the hierarchical relationship between clusters. Therefore, agglomerative hierarchical clustering is used to obtain the clustering tree structure, which will serve as the basic framework of the concept tree.

[0067] The process of hierarchical clustering based on multiple preset questions in this application can be shown as follows: Step (1), in the initial stage. Treat each standard question (i.e., preset question) as an independent cluster, assuming that data, then in the initial state there is clusters, each containing only one standard question; Step (2), calculate the distance between all clusters and get A symmetric matrix, each element in the matrix Representation Cluster and Cluster The distance between clusters is , where A and B represent the vector representation of the farthest sample point in the corresponding cluster. The distance between clusters is measured using cosine similarity; ; ; Step (3), find the two clusters with the smallest distance in the symmetric matrix and merge them into a new cluster. This new cluster contains all the standard questions in these two clusters; Step (4), use the average distance measurement method to recalculate the distance between the new cluster and other clusters. Use the Complete Linkage aggregation method to update the distance between clusters; Step (5), repeat steps (3) and (4), continue to find the two clusters with the smallest distance in the symmetric matrix, merge them into a new cluster, and update the symmetric matrix. As the clustering process proceeds, the number of clusters gradually decreases, and eventually all clusters are merged into one cluster; Step (6): During each cluster merging process, the cluster merging process is recorded, and finally a clustering tree structure is formed. Each leaf node represents a standard question (initial cluster), and each non-leaf node represents a cluster merging. The upper nodes represent the later the merging.

[0068] It should be noted that the clustering tree obtained by the above process has the following two problems: (1) It only has a basic tree structure, especially for non-leaf nodes, which should have human-understandable labels to facilitate experts to evaluate the quality of the generated concept tree. (2) The clustering tree hierarchy is too complex and contains a large number of redundant structures.

[0069] Therefore, the present application can combine the obtained clustering trees in batches with the input of prompt words into a large language model, generate the content of non-leaf nodes and optimize the tree structure, thereby obtaining a preliminary concept tree.

[0070] Specifically, non-leaf node content generation and tree structure optimization can be performed as shown in the following example: 1) Generate non-leaf node content: You can input the following prompt example 1 into the large model to generate non-leaf node content: My input is a hierarchical clustering tree, and I want you to complete the following tasks: Now we hope that you can summarize the specific content of the leaf nodes (cluster) and give all the branch nodes (parent cluster) a name. The name should be a summary and generalization of all the child nodes of the node, and it should be as concise as possible, preferably within 10 words; Leaf nodes, i.e. clusters, are not allowed to be branch nodes! It is not allowed to modify the original tree structure or the content of the original leaf node Cluster; When returning, only the original indentation format and branch node name are included; 2) Optimize tree structure: The following prompt example 2 can be input into the large model to optimize the tree structure; My input is a hierarchical clustering tree, and I want you to complete the following tasks: Now I hope you can optimize the tree structure, merge similar nodes and delete redundant non-leaf nodes. The final tree structure needs to be within 5 layers; Leaf nodes are not allowed to be branch nodes; It is not allowed to modify the content of the original leaf node Cluster! It is not allowed to delete any leaf node; Preserve original indentation format on return.

[0071] Thus, an optimized clustering tree is obtained.

[0072] Furthermore, keyword extraction and keyword weight setting can be performed: the purpose of setting keywords and their weights is to add a multi-dimensional evaluation index that is different from model scoring, thereby improving the accuracy of the system. At the same time, when the system is used in different fields, there is no need to fine-tune the model, only fine-tune the keyword configuration, which greatly improves the application efficiency of the system.

[0073] Specifically, for each node of the initial concept tree (i.e., optimized clustering tree), a keyword list and corresponding weight corresponding to the standard question of the node need to be generated. The keyword list is generated using the TextRank algorithm plus manual review.

[0074] The TextRank algorithm is a graph-based sorting algorithm. , perform word segmentation and part-of-speech tagging, retain words with specific parts of speech, such as nouns, verbs, and adjectives, to form a word set Then construct the candidate relationship graph The node set V consists of candidate keywords. The construction of edges is based on co-occurrence relationships. If two nodes (words) appear at the same time in a window (such as a sentence or a fixed-length text), it is considered that there is an edge between them. The next step is to iteratively calculate the weight of each node using the TextRank formula until convergence. The formula is as follows: ; in It is the set of in-link nodes of node V. It is the set of out-link nodes of node V. yes arrive The edge weights can be the number of co-occurrences or the similarity between words. Sort the nodes in descending order according to the calculated TextRank values ​​and select the first T nodes as keywords.

[0075] For keywords generated by the algorithm, manual review is required for re-correction. The standard for manual review is to ensure that the keywords are atomized (indivisible). And to ensure that nouns and verbs rich in semantic information are included in the candidate keywords. The candidate keyword list is guaranteed to be within four words.

[0076] For the obtained keyword list, their weights need to be generated. The less a keyword appears in the system, the more important it is for the sentence, so the corresponding weight is larger. Therefore, this application uses IDF as the weight of each keyword. The formula is: , Where t represents the word to be calculated, N represents the total number of all keywords, Indicates the number of times t appears.

[0077] The calculated weights are also stored in the form of a list, corresponding to the keyword list, and stored in the concept tree file.

[0078] For each keyword, the weight calculated by the TextRank formula Weights calculated with the IDF formula Perform weighted average and use the weight after weighted average as the final weight of the keyword.

[0079] When processing weights, you will get a count of all keywords. Only the names and weights of all keywords are retained and stored in another corresponding file. For each keyword, you need to create a synonym list and a related word list, that is, for each keyword, design words with the same meaning and add them to the synonym list, and design related words and add them to the related word list. For example, the synonym of the keyword "unit" can be "institution", and the related word of "company" can be "employee", etc.

[0080] Thus, the keywords of the preset questions corresponding to each node in the optimized clustering tree and the weights of the keywords of the preset questions corresponding to each node can be added to the corresponding nodes to obtain a concept tree.

[0081] This application uses the concept tree knowledge base to store data and generate follow-up questions. When the user's question is vague or there are multiple similar search results, the user can be quickly guided to supplement the information to get the answer, which helps to improve the accuracy of the intelligent question-answering conversation processing.

[0082] Based on the above embodiment, determining the answer content of the input text based on the target question includes: If there are multiple target questions, determine the common ancestor node of each target question from a concept tree; wherein the concept tree is a tree structure constructed based on multiple preset questions, keywords corresponding to each preset question, and weights of the keywords corresponding to each preset question; Prune the concept tree based on the common ancestor node to obtain a list of common ancestor child nodes; Determine and output follow-up questions based on the list of common ancestor child nodes; receiving supplementary text returned based on the follow-up question, and determining a target child node from a list of common ancestor child nodes based on the supplementary text; The answer content of the target sub-node corresponding to the preset question is used as the answer content of the input text.

[0083] Further, based on the supplementary text, determining the target child node from the common ancestor child node list includes: Determine a set of keywords to supplement the text; Perform keyword similarity matching between the supplementary text and each common ancestor subnode in the common ancestor subnode list; If the common ancestor child node with the highest similarity is a leaf node in the concept tree, the common ancestor child node with the highest similarity is determined as the target child node.

[0084] Specifically, the concept tree questioning option is a further summary of its nodes, which is used by the system to ask questions to the user in a more concise manner.

[0085] For branch nodes, their content is generated through large model prompts. During the generation process, their length is constrained to be within 10 characters and they are a simplification of the leaf node content. Therefore, the branch node content is directly used as its follow-up option.

[0086] For leaf nodes, each leaf node has been generated in the above process Keyword list , concatenate the keyword list in the order in which the original sentence appears . Already included in The important information in the query is used as the query option of the leaf node.

[0087] If the number of target questions is determined to be multiple and multiple standard questions or branch nodes are returned during the search phase, the concept tree tracing phase will be entered. Its purpose is to prune the concept tree through continuous tracing and user answers, and finally determine the only standard question. The specific tracing process includes: Step 1: Get multiple standard questions or branch nodes returned in the search phase and add them to the candidate node list; Step 2: Find the nearest common ancestor of the candidate node list; Step 3: Filter all child nodes of the common ancestor. If the candidate node is not in the subtree with a child node as the root, delete the child node and the subtree. Finally, get the filtered list of child nodes of the common ancestor; Step 4, obtaining the concept tree query options of the common ancestor child node list processed in step 3, and forming the query questions of the concept tree; Step 5: Return to ask follow-up questions and wait for the user to respond; Step 6: Obtain the user's answer as supplementary text, perform similarity matching between the user's answer and the list of common ancestor child nodes, and only retain the common ancestor child nodes with the highest similarity; Step 7: Update the candidate list. If the candidate node is not in the subtree with the child node obtained in step (6) as the root, delete the candidate node. After the deletion operation, if there is only one branch node in the candidate list, add all the child nodes under the branch node to the candidate list; Step 8: Repeat the above steps 2-7 until the child node corresponding to the unique standard question is determined as the target child node; Step 9: Return the unique standard question and its answer.

[0088] This application adds a special follow-up question mechanism to the question-answering system. In complex domain question-answering tasks, a slight omission in a user's question may result in the retrieval of an incorrect answer or multiple candidate answers with similar similarity scores that are difficult to distinguish. The traditional intent understanding method to solve this type of problem requires a lot of training expectations to train the slot extraction model, while this method only needs to use a special tree structure when building the knowledge base and generate follow-up question prompts based on it, which can quickly and efficiently guide users to supplement information and then locate accurate answers.

[0089] Figure 2 is a schematic diagram of the overall flow of the question-answering processing method provided in the embodiment of the present application, such as Figure 2As shown, the present application can obtain intelligent question-answering corpus in a specific field and perform data cleaning. Furthermore, a concept tree knowledge base can be constructed by clustering and the like.

[0090] Furthermore, keyword extraction operations can be performed on the data in the knowledge base, and a keyword list and corresponding weights can be configured and added to the knowledge base file.

[0091] Furthermore, the user's target query is obtained, and the keywords of the user's query are extracted. The similarity score between the knowledge base data and the user's query is calculated by combining speech model encoding with keyword matching.

[0092] Furthermore, the scores of the data to be queried in the knowledge base are sorted and filtered, and if the result after filtering is unique, the result is returned as the answer.

[0093] If the result after screening is not unique, the module will enter the follow-up question module. Specifically, based on the current candidate knowledge base data, find their parent nodes in the knowledge base, and generate follow-up questions based on the parent nodes. After the user supplements the information based on the follow-up question, the candidate data will be screened again based on the supplementary information. Repeat until the number of candidate data is unique, and return the result as the answer.

[0094] In order to facilitate the understanding of this application, the following uses the field of internal intelligent communication of an enterprise as an example to illustrate this application: Example 1 (no need to ask further questions): 1. The user enters the query statement "I want to find the phone number of Manager Z in the Finance Department"; 2. Perform keyword extraction on the query statement, and the keywords proposed include "finance department, z, telephone", etc. At the same time, generate a vector representation of the query; 3. Traverse each node of the knowledge base and calculate the keyword similarity score and vector similarity score of the node for the user query; 4. Sort and filter the scores of all nodes obtained in step 3 to obtain a candidate standard question sequence; 5. In the candidate standard question sequence, "What is the telephone number of Manager zXX of the Finance Department?" has the highest score and is the only standard question in the sequence; 6. Return the only standard question and its corresponding answer.

[0095] Example 2 (situation where further questions are needed): 1. The user enters the query "I want to find the product manager's phone number"; 2. Perform keyword extraction on the query statement, and the keywords proposed include "product, manager, phone number", etc. At the same time, generate a vector representation of the query; 3. Traverse each node of the knowledge base and calculate the keyword similarity score and vector similarity score of the node for the user query; 4. Sort and filter the scores of all nodes obtained in step 3 to obtain a candidate standard question sequence; 5. In the candidate standard question sequence, "What is the phone number of manager w in the product development department?" and "What is the phone number of manager l in the product development department?" have very similar scores and are the only two standard questions in the sequence; 6. The system finds the common ancestor node of two similar questions in the concept tree structure, and generates a follow-up question based on this: "Is it the R&D manager or the development manager?" 7. The user answers the follow-up question “I need the phone number of the product development manager”; 8. The system extracts the keywords "R&D, manager" based on the user's supplementary information. Prune the keywords from the standard candidate sequence in step 4; 9. The pruning result is the only standard question, "What is the phone number of the manager of the product development department?"; 10. Return the only standard question and its corresponding answer.

[0096] The question and answer processing device provided in the present application is described below. The question and answer processing device described below and the question and answer processing method described above can be referenced to each other.

[0097] Furthermore, the present application also provides a question and answer processing device.

[0098] The question-answer processing device comprises: A first determination module, configured to determine a keyword list based on an input text; A second determination module is used to determine the matching scores between the input text and each preset question according to the keyword list and the keyword sets corresponding to each preset question in the preset knowledge base; An input module, used to input the input text and each of the preset questions into a word embedding model respectively, to obtain text vectors output by the word embedding model respectively; A third determination module is used to determine the similarity scores between the text vector of the input text and the text vector corresponding to each preset question; A fourth determination module, configured to determine a target question from among the preset questions based on the matching scores and the similarity scores; The fifth determination module is used to determine the answer content of the input text based on the target question.

[0099] The question-answer processing device of the present application determines a keyword list based on the input text, so that the matching scores between the input text and each preset question can be determined according to the keyword list and the keyword set corresponding to each preset question in the preset knowledge base; and the input text and each preset question are respectively input into the word embedding model to obtain the text vectors respectively output by the word embedding model, so that the similarity scores between the text vector of the input text and the text vector corresponding to each preset question can be determined; further, based on each matching score and each similarity score, the target question can be accurately determined from each preset question, and then the answer content of the input text can be accurately determined based on the target question. Due to the use of keyword matching scoring combined with artificial intelligence matching scoring, a more accurate standard question for the input text can be selected from multiple preset questions as standard questions, and the multi-dimensional semantic similarity evaluation criteria make the result more accurate, so that the complexity of semantics can be captured, the accuracy of retrieval can be improved, and then the accuracy of intelligent question-answering conversation processing can be improved.

[0100] In one embodiment, the second determination module is specifically configured to, when determining the matching scores between the input text and each preset question respectively according to the keyword list and the keyword sets corresponding to each preset question in the preset knowledge base, perform the following steps for each preset question: Determine a keyword matching result of each keyword in the keyword list with a keyword set corresponding to the current preset question in the preset knowledge base; Determine the matching score of each keyword matching result respectively; Multiply each matching score by the weight of the corresponding keyword to obtain a weighted score; The weighted scores of the keywords are summed up to obtain a matching score between the input text and the current preset question.

[0101] In one embodiment, the fourth determining module is specifically configured to: Multiply the matching score of each preset question by the first weight to obtain the first score corresponding to each preset question; Multiply the similarity score of each preset question by the second weight to obtain a second score corresponding to each preset question; Add the first score and the second score of each preset question to obtain the comprehensive score corresponding to each preset question; Based on the comprehensive scores, target questions are determined from among the preset questions.

[0102] In one embodiment, the fifth determining module is specifically configured to: If there are multiple target questions, determine the common ancestor node of each target question from a concept tree; wherein the concept tree is a tree structure constructed based on multiple preset questions, keywords corresponding to each preset question, and weights of the keywords corresponding to each preset question; Pruning the concept tree based on the common ancestor node to obtain a common ancestor child node list; Determine and output a follow-up question based on the common ancestor child node list; receiving supplementary text returned based on the follow-up question, and determining a target child node from the common ancestor child node list based on the supplementary text; The answer content of the target sub-node corresponding to the preset question is used as the answer content of the input text.

[0103] In one embodiment, the fifth determining module includes a determining unit, and the determining unit is used to: determining a keyword set for the supplementary text; Perform keyword similarity matching between the supplementary text and each common ancestor subnode in the common ancestor subnode list; If the common ancestor child node with the highest similarity is a leaf node in the concept tree, the common ancestor child node with the highest similarity is determined as the target child node.

[0104] Figure 3 An example of a physical structure diagram of an electronic device is shown in FIG. Figure 3 As shown, the electronic device may include: a processor 310, a communication interface 320, a memory 330 and a communication bus 340, wherein the processor 310, the communication interface 320 and the memory 330 communicate with each other through the communication bus 340. The processor 310 may call the logic instructions in the memory 330 to execute the following method: determine a keyword list based on input text; Determine the matching scores between the input text and each preset question according to the keyword list and the keyword sets corresponding to each preset question in the preset knowledge base; Inputting the input text and each of the preset questions into a word embedding model respectively, to obtain text vectors output by the word embedding model respectively; Determine the similarity scores between the text vector of the input text and the text vector corresponding to each preset question; Determining a target question from among the preset questions based on the matching scores and the similarity scores; The answer content of the input text is determined based on the target question.

[0105] In addition, the logic instructions in the above-mentioned memory 330 can be implemented in the form of a software functional unit and can be stored in a computer-readable storage medium when it is sold or used as an independent product. Based on this understanding, the technical solution of the present application can be essentially or partly embodied in the form of a software product that contributes to the relevant technology. The computer software product is stored in a storage medium, including several instructions to enable a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk, etc. Various media that can store program codes.

[0106] In another aspect, an embodiment of the present application further provides a non-transitory computer-readable storage medium having a computer program stored thereon, which is implemented when the computer program is executed by a processor to perform the methods provided in the above embodiments, for example, including: determining a keyword list based on input text; Determine the matching scores between the input text and each preset question according to the keyword list and the keyword sets corresponding to each preset question in the preset knowledge base; Inputting the input text and each of the preset questions into a word embedding model respectively, to obtain text vectors output by the word embedding model respectively; Determine the similarity scores between the text vector of the input text and the text vector corresponding to each preset question; Determining a target question from among the preset questions based on the matching scores and the similarity scores; The answer content of the input text is determined based on the target question.

[0107] In another aspect, an embodiment of the present application further provides a computer program product having a computer program stored thereon, which is implemented when the computer program is executed by a processor to perform the methods provided in the above embodiments, for example, including: determining a keyword list based on input text; Determine the matching scores between the input text and each preset question according to the keyword list and the keyword sets corresponding to each preset question in the preset knowledge base; Inputting the input text and each of the preset questions into a word embedding model respectively, to obtain text vectors output by the word embedding model respectively; Determine the similarity scores between the text vector of the input text and the text vector corresponding to each preset question; Determining a target question from among the preset questions based on the matching scores and the similarity scores; The answer content of the input text is determined based on the target question.

[0108] The device embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. Ordinary technicians in this field can understand and implement it without paying creative labor.

[0109] Through the description of the above implementation methods, those skilled in the art can clearly understand that each implementation method can be implemented by means of software plus a necessary general hardware platform, and of course, can also be implemented by hardware. Based on this understanding, the above technical solution is essentially or the part that contributes to the relevant technology can be embodied in the form of a software product, and the computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a disk, an optical disk, etc., including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or some parts of the embodiment.

[0110] Finally, it should be noted that the above implementation modes are only used to illustrate the present application, rather than to limit the present application. Although the present application is described in detail with reference to the embodiments, a person skilled in the art should understand that various combinations, modifications or equivalent substitutions of the technical solutions of the present application do not depart from the spirit and scope of the technical solutions of the present application.

Claims

1. A question-answering processing method, characterized in that: include: Determining a list of keywords based on the input text; Determine the matching scores between the input text and each preset question according to the keyword list and the keyword sets corresponding to each preset question in the preset knowledge base; Inputting the input text and each of the preset questions into a word embedding model respectively, to obtain text vectors output by the word embedding model respectively; Determine the similarity scores between the text vector of the input text and the text vector corresponding to each preset question; Determining a target question from among the preset questions based on the matching scores and the similarity scores; The answer content of the input text is determined based on the target question.

2. The question-answer processing method according to claim 1, characterized in that: The step of determining the answer content of the input text based on the target question includes: If there are multiple target questions, determine the common ancestor node of each target question from a concept tree; wherein the concept tree is a tree structure constructed based on multiple preset questions, keywords corresponding to each preset question, and weights of the keywords corresponding to each preset question; Pruning the concept tree based on the common ancestor node to obtain a common ancestor child node list; Determine and output a follow-up question based on the common ancestor child node list; receiving supplementary text returned based on the follow-up question, and determining a target child node from the common ancestor child node list based on the supplementary text; The answer content of the target sub-node corresponding to the preset question is used as the answer content of the input text.

3. The question-answer processing method according to claim 2, characterized in that: The step of determining the target child node from the common ancestor child node list based on the supplementary text comprises: determining a keyword set for the supplementary text; Perform keyword similarity matching between the supplementary text and each common ancestor subnode in the common ancestor subnode list; If the common ancestor child node with the highest similarity is a leaf node in the concept tree, the common ancestor child node with the highest similarity is determined as the target child node.

4. The question-answer processing method according to claim 2, characterized in that: The concept tree is constructed based on the following steps: Perform hierarchical clustering based on multiple preset questions to obtain a cluster tree; Optimizing the clustering tree based on the large language model and the preset prompt words to obtain an optimized clustering tree; The keywords of the preset questions corresponding to each node in the optimized clustering tree and the weights of the keywords of the preset questions corresponding to each node are added to the corresponding nodes to obtain a concept tree.

5. The question-answer processing method according to claim 1, characterized in that: When determining the matching scores between the input text and each preset question respectively according to the keyword list and the keyword set corresponding to each preset question in the preset knowledge base, the following steps are performed for each preset question respectively: Determine a keyword matching result of each keyword in the keyword list with a keyword set corresponding to a current preset question in a preset knowledge base; Determine the matching score of each keyword matching result respectively; Multiply each matching score by the weight of the corresponding keyword to obtain a weighted score; The weighted scores of the keywords are summed up to obtain a matching score between the input text and the current preset question.

6. The question-answer processing method according to claim 1, characterized in that: The determining a target question from each preset question based on each matching score and each similarity score includes: Multiply the matching score of each preset question by the first weight to obtain the first score corresponding to each preset question; Multiply the similarity score of each preset question by the second weight to obtain a second score corresponding to each preset question; Add the first score and the second score of each preset question to obtain the comprehensive score corresponding to each preset question; Based on the comprehensive scores, target questions are determined from among the preset questions.

7. A question-answer processing device, characterized in that: include: A first determination module, configured to determine a keyword list based on an input text; A second determination module is used to determine the matching scores between the input text and each preset question according to the keyword list and the keyword sets corresponding to each preset question in the preset knowledge base; An input module, used to input the input text and each of the preset questions into a word embedding model respectively, to obtain text vectors output by the word embedding model respectively; A third determination module is used to determine the similarity scores between the text vector of the input text and the text vector corresponding to each preset question; A fourth determination module, configured to determine a target question from among the preset questions based on the matching scores and the similarity scores; The fifth determination module is used to determine the answer content of the input text based on the target question.

8. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the computer program, the question and answer processing method according to any one of claims 1 to 6 is implemented.

9. A storage medium, the storage medium being a non-transitory computer-readable storage medium, on which a computer program is stored, characterized in that: When the computer program is executed by a processor, the question and answer processing method as described in any one of claims 1 to 6 is implemented.

10. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the question and answer processing method according to any one of claims 1 to 6 is implemented.

Images