Question and answer processing method and apparatus, electronic device, storage medium, and computer product
By constructing a concept tree and combining keyword matching and word embedding models, the problem of insufficient semantic capture in intelligent question answering is solved, achieving more accurate question answering and improving the accuracy of conversation processing.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- CHINA MOBILE ZHIJIE TECHNOLOGY (BEIJING) CO LTD
- Filing Date
- 2024-12-17
- Publication Date
- 2026-06-09
Smart Images
Figure CN119938825B_ABST
Abstract
Description
Technical Field
[0001] This application relates to the field of Internet technology, and in particular to a question-and-answer processing method, apparatus, electronic device, storage medium, and computer product. Background Technology
[0002] Intelligent question answering primarily refers to automatically returning a reasonable answer in a dialogue format based on a user's question. Existing intelligent question answering methods often generate answers by combining knowledge base retrieval with language model summarization. The key to these methods lies in the accuracy of the retrieval between the user's question and the standard answers in the knowledge base. Currently, the retrieval method in the intelligent question answering process mainly relies on converting the user's question into an encoding, calculating the similarity with the pre-encoded answer vectors in the knowledge base, and returning the most similar answer (the one with the highest similarity score).
[0003] However, calculating the most similar answer by comparing the encoded user question with the pre-encoded answer vector in the knowledge base cannot fully capture the semantic complexity, resulting in insufficient retrieval accuracy, which in turn leads to insufficient accuracy in the current intelligent question-answering session processing. Summary of the Invention
[0004] This application aims to address at least one of the technical problems existing in related technologies. To this end, this application proposes a question-answering processing method, apparatus, electronic device, storage medium, and computer product to solve the problem that current question-answering answer retrieval cannot fully capture the semantic complexity, resulting in insufficient retrieval accuracy, thereby improving the accuracy of intelligent question-answering conversation processing.
[0005] The question-and-answer processing method according to the first aspect of this application includes:
[0006] Determine the keyword list based on the input text;
[0007] Based on the keyword list and the keyword set corresponding to each preset question in the preset knowledge base, the matching score between the input text and each preset question is determined.
[0008] The input text and each of the preset questions are respectively input into the word embedding model to obtain the text vectors output by the word embedding model respectively;
[0009] Determine the similarity score between the text vector of the input text and the text vector corresponding to each preset question;
[0010] Based on the matching scores and similarity scores, the target question is determined from each preset question;
[0011] The response content of the input text is determined based on the target question.
[0012] According to one embodiment of this application, determining the response content of the input text based on the target question includes:
[0013] If there are multiple target problems, the common ancestor node of each target problem is determined from the concept tree; wherein, the concept tree is a tree structure constructed based on multiple preset problems, the keywords corresponding to each preset problem, and the weights of the keywords corresponding to each preset problem.
[0014] Based on the common ancestor node, the concept tree is pruned to obtain a list of common ancestor child nodes;
[0015] Based on the list of common ancestor child nodes, determine and output the follow-up questions;
[0016] Upon receiving supplementary text returned based on the follow-up question, the target child node is determined from the list of common ancestor child nodes based on the supplementary text;
[0017] The response content of the preset question corresponding to the target sub-node is used as the response content of the input text.
[0018] According to one embodiment of this application, determining the target child node from the list of common ancestor child nodes based on the supplementary text includes:
[0019] Determine the set of keywords for the supplementary text;
[0020] The supplementary text is matched with each common ancestor child node in the list of common ancestor child nodes by keyword similarity.
[0021] If the common ancestor child node with the highest similarity is a leaf node in the concept tree, then the common ancestor child node with the highest similarity is determined as the target child node.
[0022] According to one embodiment of this application, the concept tree is constructed based on the following steps:
[0023] Hierarchical clustering is performed based on multiple pre-defined questions to obtain a clustering tree;
[0024] The clustering tree is optimized based on a large language model and preset prompt words to obtain an optimized clustering tree;
[0025] The keywords of the preset question corresponding to each node in the optimized clustering tree and the weights of the keywords of the preset question corresponding to each node are added to the corresponding nodes to obtain the concept tree.
[0026] According to one embodiment of this application, when determining the matching score between the input text and each preset question based on the keyword list and the keyword set corresponding to each preset question in the preset knowledge base, the following steps are performed for each preset question:
[0027] Determine the keyword matching result of each keyword in the keyword list in the keyword set corresponding to the current preset question in the preset knowledge base;
[0028] Determine the matching score for each keyword matching result;
[0029] Each matching score is multiplied by the weight of the corresponding keyword to obtain the weight score;
[0030] The weighted scores of each keyword are summed to obtain the matching score between the input text and the current preset question.
[0031] According to one embodiment of this application, determining the target question from each preset question based on each matching score and each similarity score includes:
[0032] The matching score for each preset question is multiplied by the first weight to obtain the first score corresponding to each preset question.
[0033] The similarity score of each preset question is multiplied by the second weight to obtain the second score corresponding to each preset question.
[0034] Add the first score and the second score for each preset question to obtain the comprehensive score for each preset question.
[0035] Based on the comprehensive scores, the target question is determined from the preset questions.
[0036] A question-and-answer processing apparatus according to a second aspect embodiment of this application includes:
[0037] The first determining module is used to determine a list of keywords based on the input text;
[0038] The second determining module is used to determine the matching score between the input text and each preset question based on the keyword list and the keyword set corresponding to each preset question in the preset knowledge base.
[0039] The input module is used to input the input text and each of the preset questions into the word embedding model respectively, and obtain the text vectors output by the word embedding model respectively;
[0040] The third determining module is used to determine the similarity score between the text vector of the input text and the text vector corresponding to each preset question;
[0041] The fourth determining module is used to determine the target question from each preset question based on each matching score and each similarity score;
[0042] The fifth determining module is used to determine the response content of the input text based on each target question.
[0043] An electronic device according to a third aspect of this application includes a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement any of the question-and-answer processing methods described above.
[0044] According to a fourth aspect of this application, the storage medium is a non-transitory computer-readable storage medium storing a computer program thereon, which, when executed by a processor, implements the question-and-answer processing method as described above.
[0045] A computer program product according to a fifth aspect of this application includes a computer program that, when executed by a processor, implements the question-and-answer processing method as described above.
[0046] The above-described one or more technical solutions in the embodiments of this application have at least the following technical effects:
[0047] Based on the input text, a keyword list is determined, allowing for the calculation of matching scores between the input text and each preset question, using the keyword list and the keyword sets corresponding to each preset question in the preset knowledge base. Furthermore, the input text and each preset question are fed into a word embedding model, yielding text vectors output by the model. This allows for the determination of similarity scores between the input text's text vector and the text vector corresponding to each preset question. Based on these matching and similarity scores, a target question can be accurately identified from the preset questions, leading to a precise answer to the input text. By combining keyword matching scoring with AI-based matching scoring, a more accurate standard question can be selected from multiple preset questions serving as standard questions. The multi-dimensional semantic similarity evaluation criteria result in more accurate results, capturing semantic complexity, improving retrieval accuracy, and ultimately enhancing the accuracy of intelligent question-answering conversation processing.
[0048] Additional aspects and advantages of this application will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of this application. Attached Figure Description
[0049] To more clearly illustrate the technical solutions in this invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.
[0050] Figure 1 This is a flowchart illustrating the question-and-answer processing method provided in the embodiments of this application.
[0051] Figure 2 This is a schematic diagram of the overall process of the question-and-answer processing method provided in the embodiments of this application.
[0052] Figure 3 This is a schematic diagram of the structure of the electronic device provided in this application. Detailed Implementation
[0053] The embodiments of this application will be described in further detail below with reference to the accompanying drawings and examples. The following examples are used to illustrate this application, but should not be used to limit the scope of this application.
[0054] In the description of the embodiments of this application, it should be noted that the terms "center," "longitudinal," "lateral," "upper," "lower," "front," "rear," "left," "right," "vertical," "horizontal," "top," "bottom," "inner," and "outer," etc., indicate the orientation or positional relationship based on the orientation or positional relationship shown in the accompanying drawings. They are only for the convenience of describing the embodiments of this application and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation. Therefore, they should not be construed as limitations on the embodiments of this application. In addition, the terms "first," "second," and "third" are used for descriptive purposes only and should not be construed as indicating or implying relative importance.
[0055] In the description of the embodiments of this application, it should be noted that, unless otherwise explicitly specified and limited, the terms "connected" and "linked" should be interpreted broadly. For example, they can refer to a fixed connection, a detachable connection, or an integral connection; they can refer to a mechanical connection or an electrical connection; they can refer to a direct connection or an indirect connection through an intermediate medium. Those skilled in the art can understand the specific meaning of the above terms in the embodiments of this application based on the specific circumstances.
[0056] In the embodiments of this application, unless otherwise expressly specified and limited, "above" or "below" the second feature can mean that the first feature is in direct contact with the second feature, or that the first feature is in indirect contact with the second feature through an intermediate medium. Furthermore, "above," "on top of," and "over" the second feature can mean that the first feature is directly above or diagonally above the second feature, or simply that the first feature is at a higher horizontal level than the second feature. "Below," "below," and "under" the second feature can mean that the first feature is directly below or diagonally below the second feature, or simply that the first feature is at a lower horizontal level than the second feature.
[0057] In the description of this specification, the references to terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., refer to specific features, structures, materials, or characteristics described in connection with that embodiment or example, which are included in at least one embodiment or example of the embodiments of this application. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples. Moreover, without contradiction, those skilled in the art can combine and integrate the different embodiments or examples described in this specification, as well as the features of different embodiments or examples.
[0058] This application discloses a question-and-answer processing method, apparatus, electronic device, storage medium, and computer product.
[0059] Figure 1 This is a flowchart illustrating the question-and-answer processing method provided in the embodiments of this application, such as... Figure 1 As shown, the question-and-answer processing method includes:
[0060] Step 110: Determine the keyword list based on the input text.
[0061] Step 120: Determine the matching score between the input text and each preset question based on the keyword list and the keyword set corresponding to each preset question in the preset knowledge base.
[0062] Step 130: Input the input text and each preset question into the word embedding model to obtain the text vectors output by the word embedding model.
[0063] Step 140: Determine the similarity score between the text vector of the input text and the text vector corresponding to each preset question.
[0064] Step 150: Based on each matching score and each similarity score, determine the target question from each preset question.
[0065] Step 160: Determine the response content of the input text based on each target question.
[0066] It should be noted that the execution entity of the question-and-answer processing method provided in this application embodiment can be a server, computer equipment, etc. Computer equipment can be, for example, a mobile phone, tablet computer, laptop computer, PDA, in-vehicle electronic equipment, wearable device, ultra-mobile personal computer (UMPC), netbook, or personal digital assistant (PDA), etc. It should also be noted that all data required in this application has been obtained through legitimate channels after authorization from the relevant users.
[0067] The server or computer equipment of this application may be equipped with or connected to a question-and-answer processing device, thereby enabling the question-and-answer processing device to execute the question-and-answer processing method of this application.
[0068] This application can obtain raw data and extract standard questions (which can be simply referred to as standard questions) from the raw data as preset questions.
[0069] Specifically, the raw data is typically stored in an Excel file, containing a standard question and its corresponding answer. First, redundant columns are removed, leaving only the useful information. Next, the standard questions are cleaned. Cleaning primarily involves splitting the standard questions: some standard questions have multiple semantic layers and need to be split into two questions with corresponding answers. Secondly, the standard questions are reconstructed: some standard questions are too colloquial and require manual correction, making some keywords more formal. Finally, the standard questions are deduplicated. Since the raw data may come from different regions, there may be semantically very similar questions. In this case, only one standard question can be retained.
[0070] Furthermore, standard answers to standard questions are extracted as responses to preset questions. Some standard questions require manual review. Therefore, a preset knowledge base can be constructed based on the extracted preset questions and their responses. This application can also generate a set of keywords for each preset question in the preset knowledge base.
[0071] The question-answering processing method in this application can be applied to intelligent question-answering scenarios and can be deployed in intelligent question-answering systems, intelligent question-answering platforms, etc.
[0072] In intelligent question-answering scenarios, this application can obtain the user's input text.
[0073] Furthermore, this application can extract keywords from the input text and form a keyword list from the extracted keywords.
[0074] Furthermore, this application can perform text similarity calculation based on keyword matching for the keyword list and the keyword set corresponding to each preset question in the preset knowledge base, thereby obtaining the matching score between the input text and each preset question.
[0075] Furthermore, this application can obtain a word embedding model, which can specifically be a pre-trained text encoding model BGE based on a transformer encoder. Transformer is a deep learning model architecture.
[0076] BGE can map natural language into high-dimensional vectors for computation. The training tasks for BGE include various text similarity matching calculations. In this application, the BGE model is not fine-tuned; encoding and similarity calculation are performed directly.
[0077] Therefore, this application allows the input text and each preset question to be input into a word embedding model, encoded by the word embedding model, and then the text vectors output by the word embedding model are obtained respectively. For example, the text vector of the input text is represented as v1, and the text vector corresponding to each preset question is represented as v2, and so on: The dimension of the vector can be 1024.
[0078] Furthermore, cosine similarity can be used to calculate the similarity between the feature vectors of the input text and the corresponding text vectors of each preset question. The formula for cosine similarity is:
[0079] .
[0080] Furthermore, the scores of each similarity can be normalized, specifically to the range of [0, 1], to obtain the similarity scores between the text vector of the input text and the text vector corresponding to each preset question.
[0081] Existing retrieval-based question-answering systems often use a method of pre-trained language model encoding plus vector similarity calculation to measure semantic similarity. This method is based on fine-tuning training on a specific domain knowledge set. However, the encoding model is not interpretable. If you want to optimize the encoding results, you must add new data and retrain, which is time-consuming and labor-intensive.
[0082] In this application, a comprehensive score can be determined by weighted summation for the matching score and similarity score corresponding to each preset question. Preset questions (i.e. standard questions) are then filtered based on the comprehensive scores, and finally the target question is determined from all preset questions to form a set of candidate standard questions.
[0083] This application employs a dual evaluation strategy for semantic similarity assessment, combining manually generated keyword features with pre-trained language model encoding. This allows the system to be adapted to tasks in different domains without requiring fine-tuning of the model, by modifying only the keyword features.
[0084] In existing methods, the interaction between the system and the user is simply that the user asks a question, and the system provides an answer. If the user's question is very vague, the system will likely return an incorrect answer because it does not determine whether the answer with the highest similarity is the standard answer. Therefore, this application pre-constructs a tree structure, which is defined as a concept tree, based on multiple preset questions, the keywords corresponding to each preset question, and the weights of the keywords for each preset question.
[0085] Furthermore, if the candidate standard question set contains exactly one element, and the standard question corresponding to this element exists only in the leaf nodes of the concept tree, then this standard question and its answer are used as the response content of the input text and returned to the user. If the candidate standard question set is empty, then the process is transferred to a human interviewer. In other cases, a follow-up questioning method based on the concept tree is used to pinpoint the user's desired question.
[0086] According to the question-answering method of this application embodiment, a keyword list is determined based on the input text, enabling the matching score between the input text and each preset question to be determined based on the keyword list and the keyword set corresponding to each preset question in the preset knowledge base; and the input text and each preset question are respectively input into a word embedding model to obtain the text vectors output by the word embedding model, enabling the determination of the similarity score between the text vector of the input text and the text vector corresponding to each preset question; furthermore, based on each matching score and each similarity score, the target question can be accurately determined from each preset question, and then the answer content of the input text can be accurately determined based on the target question. Since the scoring of keyword matching is combined with the scoring of artificial intelligence matching, a more accurate standard question for the input text can be selected from multiple preset questions serving as standard questions. The multi-dimensional semantic similarity evaluation criteria make the results more accurate, thus capturing the complexity of semantics, improving the accuracy of retrieval, and thereby improving the accuracy of intelligent question-answering conversation processing.
[0087] Based on the above embodiments, when determining the matching score between the input text and each preset question according to the keyword list and the keyword set corresponding to each preset question in the preset knowledge base, the following steps are performed for each preset question:
[0088] Determine the keyword matching results for each keyword in the keyword list within the keyword set corresponding to the current preset question in the preset knowledge base;
[0089] Determine the matching score for each keyword matching result;
[0090] Each matching score is multiplied by the weight of the corresponding keyword to obtain the weight score;
[0091] The weighted scores of each keyword are summed to obtain the matching score between the input text and the current preset question.
[0092] Specifically, this application may denote the keyword list as A, and the keyword sets corresponding to each preset question in the preset knowledge base as Bn.
[0093] Based on this, a similarity score variable can be pre-specified for each standard question. Iterate through all standard questions. If an element 'a' in A matches an element in the current standard question keyword set Bn, then... If a word or its corresponding synonym is the same, it scores 1 point, multiplied by the weight of the corresponding keyword, and added to S. If it is the same as a related word, it scores 0.5 points, multiplied by the weight of the corresponding keyword, and added to S. If no matching word is found, it scores 0 points. The final S is the keyword similarity score between this standard question and the user's input text, which is used as the matching score. From this, the matching scores between the input text and each preset question can be obtained.
[0094] This application uses a text similarity calculation method based on keyword matching to calculate scores. Since it intuitively reflects the degree of keyword matching in the text, it is easy to understand and interpret. Furthermore, keyword matching can be combined with other more complex semantic understanding methods to improve the accuracy and robustness of similarity calculation.
[0095] Based on the above embodiments, the target question is determined from each preset question based on each matching score and each similarity score, including:
[0096] The matching score for each preset question is multiplied by the first weight to obtain the first score corresponding to each preset question.
[0097] The similarity score of each preset question is multiplied by the second weight to obtain the second score corresponding to each preset question.
[0098] Add the first score and the second score for each preset question to obtain the comprehensive score for each preset question.
[0099] Based on the comprehensive scores, the target question is determined from the preset questions.
[0100] Specifically, after obtaining the matching score S1 and similarity score S2 for each preset question, this application can determine the weight of the matching score as a first weight w1 and the weight of the similarity score as a second weight w2. In one embodiment, the first weight can be 0.7 and the second weight can be 0.3.
[0101] Therefore, the overall score between the input text and each preset question can be calculated using the following formulas:
[0102] .
[0103] Furthermore, this application can set a threshold. Delete the preset questions corresponding to scores below the threshold. Then set the threshold. Sort all remaining candidate preset questions in descending order. If it exists Then discard And all subsequent elements, ultimately yielding the candidate standard question set.
[0104] If the candidate standard question set contains exactly one element, and the standard question corresponding to this element exists only in the leaf nodes of the concept tree, then this standard question is taken as the target question, and the standard question and its answer are used as the input text response and returned to the user. If the candidate standard question set is empty, then the process is transferred to a human interviewer. In other cases, a follow-up questioning method based on the concept tree is used to pinpoint the user's desired question.
[0105] This application employs a combination of keyword matching scoring and AI matching scoring, which can select more accurate standard questions for the input text from multiple preset questions that serve as standard questions. The multi-dimensional semantic similarity evaluation criteria make the results more accurate, thus capturing the complexity of semantics, improving the accuracy of retrieval, and consequently improving the accuracy of intelligent question answering conversation processing.
[0106] Meanwhile, this application introduces a series of custom scoring criteria and threshold criteria to determine the acceptance or rejection of similarity scores and improve retrieval accuracy.
[0107] Based on the above embodiments, the concept tree is constructed based on the following steps:
[0108] Hierarchical clustering is performed based on multiple pre-defined questions to obtain a clustering tree;
[0109] The clustering tree is optimized based on a large language model and preset prompt words to obtain an optimized clustering tree;
[0110] The keywords corresponding to the preset questions of each node in the optimized clustering tree and the weights of the keywords corresponding to the preset questions of each node are added to the corresponding nodes to obtain the concept tree.
[0111] Specifically, this application can obtain a clustering tree structure through hierarchical clustering, thereby constructing multiple predefined problems into a clustering tree.
[0112] More specifically, hierarchical clustering is divided into two types: agglomerative hierarchical clustering and splitting hierarchical clustering. Agglomerative hierarchical clustering is a bottom-up clustering method that initially treats each data point as a separate cluster. As the algorithm runs, it gradually merges the nearest clusters until all clusters are merged into one large cluster, or until a predetermined number of clusters is reached. In this process, a hierarchical nested clustering tree is naturally formed. In the clustering tree, the original data points of different categories are at the lowest level, and the top level is the root node of a cluster. Because the number of standard questions in the knowledge base is too large, directly using a large model to generate a concept tree will lead to the "large model illusion" problem. Other clustering methods cannot obtain the hierarchical relationships between clusters; therefore, agglomerative hierarchical clustering is used to obtain the clustering tree structure, which will serve as the basic framework of the concept tree.
[0113] The hierarchical clustering process based on multiple pre-defined questions in this application can be illustrated as follows:
[0114] Step (1), in the initial stage. Treat each standard question (i.e., the preset question) as an independent cluster, assuming there are If there are 10 data points, then in the initial state there exists 10 data points. Each cluster contains only one standard question;
[0115] Step (2): Calculate the distance between all clusters to obtain... A symmetric matrix, where each element of the matrix... Cluster and cluster The distance between clusters is denoted by A and B, where A and B represent the vector representations of the farthest sample points in their respective clusters. The distance between clusters is measured using cosine similarity.
[0116] ;
[0117] ;
[0118] Step (3): Find the two clusters with the smallest distance in the symmetric matrix and merge them into a new cluster. This new cluster contains all the standard variables from the two clusters.
[0119] Step (4): Recalculate the distances between the new cluster and other clusters using the average distance metric. Update the distances between clusters using the Complete Linkage aggregation method;
[0120] Step (5): Repeat steps (3) and (4) to continue searching for the two clusters with the smallest distance in the symmetric matrix, merge them into a new cluster, and update the symmetric matrix. As the clustering process progresses, the number of clusters gradually decreases, and eventually all clusters are merged into one cluster;
[0121] Step (6): During each cluster merging process, record the cluster merging process to ultimately form a clustering tree structure. Each leaf node represents a standard cluster (initial cluster), and each non-leaf node represents a cluster merging. The higher the node is, the later the merging occurs.
[0122] It should be noted that the clustering tree obtained by the above process has the following two problems: (1) It only has a basic tree structure. In particular, for non-leaf nodes, they should have human-understandable identifiers to facilitate experts in evaluating the quality of the generated concept tree. (2) The clustering tree hierarchy is too complex and has a large number of redundant structures.
[0123] Therefore, this application can combine the obtained clustering tree with the input of prompt words in batches into a large language model to generate the content of non-leaf nodes and optimize the tree structure, thereby obtaining a preliminary concept tree.
[0124] Specifically, non-leaf node content generation and tree structure optimization can be performed as shown in the following example:
[0125] 1) Generate non-leaf node content: You can input the following prompt (Example 1) into the large model to generate non-leaf node content:
[0126] My input is a hierarchical clustering tree. I hope you will complete the following tasks:
[0127] Now, I hope you can summarize the specific content of the leaf nodes (cluster) and give all the branch nodes (parent cluster) a name. The name should be a summary and overview of all the child nodes of that node, and should be as concise as possible, preferably within 10 characters.
[0128] Leaf nodes, i.e., clusters, are not allowed to be branch nodes!
[0129] Modification of the original tree structure and the content of the original leaf nodes (Cluster) are not allowed.
[0130] When returning, only the original indentation format and the branch node name are required;
[0131] 2) Optimize the tree structure: You can input the following hint (Example 2) into the large model to optimize the tree structure;
[0132] My input is a hierarchical clustering tree. I hope you will complete the following tasks:
[0133] Now you need to optimize the tree structure by merging similar nodes and removing redundant non-leaf nodes. The final tree structure should be no more than 5 levels.
[0134] Leaf nodes are not allowed to be branch nodes;
[0135] Modifying the contents of existing leaf nodes (Clusters) is not allowed! Deleting any leaf nodes is not allowed;
[0136] Preserve the original indentation format when returning.
[0137] Thus, the optimized clustering tree is obtained.
[0138] Furthermore, keyword extraction and weight settings can be performed: setting keywords and their weights aims to add a multi-dimensional evaluation metric that differs from model scoring, thereby improving the system's accuracy. Simultaneously, when applying the system to different fields, no further model fine-tuning is required; only keyword configuration adjustments are needed for applicability, significantly improving system application efficiency.
[0139] Specifically, for each node of the initial concept tree (i.e., the optimized clustering tree), a list of keywords and corresponding weights for the standard question corresponding to that node needs to be generated. The generation of the keyword list is accomplished using the TextRank algorithm combined with manual review.
[0140] TextRank is a graph-based ranking algorithm. For the sentence to be processed... The words are segmented and tagged with parts of speech, retaining words of specific parts of speech, such as nouns, verbs, and adjectives, to form a word set. Then construct a candidate relationship graph. The node set V consists of candidate keywords. Edges are constructed based on co-occurrence relationships; if two nodes (words) appear simultaneously within a window (such as a sentence or a fixed-length text), an edge is considered to exist between them. The next step is to iteratively calculate the weight of each node using the TextRank formula until convergence. The formula is as follows:
[0141] ;
[0142] in It is the set of incoming nodes of node V. It is the set of outgoing nodes of node V. yes arrive The edge weights (edge weights can be the number of times they co-occur or the similarity between words) are used to rank the nodes in descending order based on their calculated TextRank values, and the top T nodes are selected as keywords.
[0143] Keywords generated by the algorithm need to be manually reviewed and revised. The criteria for manual review are: first, ensuring the keywords are atomized (making them indivisible); second, ensuring that semantically rich nouns and verbs are included in the candidate keywords; and third, ensuring the candidate keyword list contains no more than four words.
[0144] For the obtained keyword list, their weights need to be generated. The less frequently a keyword appears in the system, the more important it is for identifying the sentence, and therefore the higher its weight. Therefore, this application uses IDF (Independent Keyword Function) as the weight for each keyword. The formula is:
[0145] ,
[0146] Where t represents the word to be calculated, and N represents the total number of all keywords. This indicates the number of times t appears.
[0147] The calculated weights are also stored in the form of a list, corresponding to the keyword list, and stored in the concept tree file.
[0148] For each keyword, the weights calculated using the TextRank formula will be... Weights calculated with the IDF formula Perform a weighted average, and use the weighted average as the final weight of the keyword.
[0149] During the weighting process, a count of all keywords is obtained. Only the names and weights of all keywords are stored in a separate file. For each keyword, a thesaurus and a related thesaurus need to be created. That is, for each keyword, words with the same meaning are added to the thesaurus, and related words are added to the related thesaurus. For example, the synonym for the keyword "unit" could be "organization," and the related words for "company" could be "employee," etc.
[0150] Therefore, the keywords corresponding to the preset questions of each node in the optimized clustering tree and the weights of the keywords corresponding to the preset questions of each node can be added to the corresponding nodes to obtain the concept tree.
[0151] This application utilizes a concept tree knowledge base to store data and generate follow-up questions. This allows for quick guidance of users to supplement information and obtain answers when user questions are ambiguous or when multiple similar search results exist, thus improving the accuracy of conversational processing in intelligent question answering.
[0152] Based on the above embodiments, determining the response content of the input text based on the target question includes:
[0153] If there are multiple target questions, determine the common ancestor node of each target question from the concept tree; wherein, the concept tree is a tree structure constructed based on multiple preset questions, the keywords corresponding to each preset question, and the weights of the keywords corresponding to each preset question;
[0154] Pruning the concept tree based on the common ancestor node yields a list of common ancestor child nodes;
[0155] Determine and output the follow-up questions based on the list of common ancestor child nodes;
[0156] Upon receiving supplementary text returned based on follow-up questions, the target child node is determined from the list of common ancestor child nodes based on the supplementary text;
[0157] Use the answer to the preset question corresponding to the target sub-node as the answer content of the input text.
[0158] Furthermore, based on the supplementary text, the target child node is determined from the list of common ancestor child nodes, including:
[0159] Determine the set of keywords for the supplementary text;
[0160] The supplementary text is matched with each common ancestor child node in the list of common ancestor child nodes by keyword similarity.
[0161] If the common ancestor child node with the highest similarity is a leaf node in the concept tree, then the common ancestor child node with the highest similarity is determined as the target child node.
[0162] Specifically, the concept tree follow-up question options are a further summary of its nodes, used by the system to ask users more concise follow-up questions.
[0163] For branch nodes, their content is generated through prompts from the large model. During the generation process, their length is constrained to within 10 characters and is a simplification of the leaf node content. Therefore, the branch node content is directly used as its follow-up question options.
[0164] For leaf nodes, each leaf node has been generated in the above process. Keyword list The keyword list is obtained by concatenating the keywords in the original sentence order. . It already includes The important information in the document is used as a follow-up question option for the leaf node.
[0165] If, during the retrieval phase, the number of target questions is determined to be multiple, and multiple standard questions or branch nodes are returned, the concept tree follow-up questioning phase will begin. The purpose of this phase is to prune the concept tree through continuous follow-up questions and user responses, ultimately determining a unique standard question. The specific follow-up questioning process includes:
[0166] Step 1: Obtain multiple standard questions or branch nodes returned during the retrieval phase and add them to the candidate node list;
[0167] Step 2: Find the lowest common ancestor of the candidate node list;
[0168] Step 3: Filter all child nodes of the common ancestor. If a candidate node is not in the subtree rooted at a certain child node, delete that child node and the subtree. Finally, obtain the filtered list of child nodes of the common ancestor.
[0169] Step 4: Obtain the concept tree follow-up options from the list of common ancestor child nodes processed in Step 3, and form the follow-up questions of the concept tree;
[0170] Step 5: Return to the follow-up question and wait for the user's response;
[0171] Step 6: Obtain the user's answer as supplementary text, perform similarity matching between the user's answer and the list of common ancestor child nodes, and retain only the common ancestor child node with the highest similarity.
[0172] Step 7: Update the candidate list. If a candidate node is not in the subtree rooted at the child node obtained in step (6), delete the candidate node. After the deletion operation, if there is only one branch node in the candidate list, add all child nodes under that branch node to the candidate list;
[0173] Step 8: Repeat steps 2-7 above until a unique criterion is determined and the corresponding child node is selected as the target child node.
[0174] Step 9: Return to the uniquely identified standard question and its answer.
[0175] This application incorporates a unique follow-up question mechanism into the question-answering system. In complex domain-specific question-answering tasks, even slight omissions in a user's question can lead to incorrect answers or difficulty in distinguishing between multiple candidate answers with similar scores. Traditional intent understanding methods require extensive training materials to develop the slot extraction model, while this method only needs to employ a special tree structure when constructing the knowledge base and generate follow-up question prompts based on it. This allows for rapid and efficient guidance of users to supplement information, thereby locating the accurate answer.
[0176] Figure 2 This is a schematic diagram of the overall flow of the question-and-answer processing method provided in the embodiments of this application, such as... Figure 2As shown, this application can acquire intelligent question-answering corpora in a specific domain and perform data cleaning. Furthermore, a concept tree knowledge base can be constructed through methods such as clustering.
[0177] Furthermore, keyword extraction can be performed on the data in the knowledge base, and a keyword list with corresponding weights can be configured and added to the knowledge base file.
[0178] Furthermore, the user's target query is obtained, and the keywords of the user's query are extracted. A similarity score between the knowledge base data and the user's query is calculated by combining speech model encoding with keyword matching.
[0179] Furthermore, the scores of the data to be queried in the knowledge base are sorted and filtered. If the result after filtering is unique, the result is returned as the answer.
[0180] If the filtered results are not unique, the system proceeds to the follow-up question module. Specifically, based on the current candidate knowledge base data, the system finds their parent nodes in the knowledge base and generates follow-up questions based on these parent nodes. After the user provides additional information based on the follow-up questions, the candidate data is filtered again based on this additional information. This process is repeated until the number of candidate data is unique, and the result is returned as the answer.
[0181] To facilitate understanding of this application, the following explanation uses the field of enterprise internal intelligent communication as an example:
[0182] Example 1 (situation where follow-up questions are not required):
[0183] 1. The user enters the query "I want to find the phone number of Manager Z in the finance department";
[0184] 2. The query statement is processed to extract keywords, which include "finance department, z, telephone," etc. A vector representation of the query is also generated.
[0185] 3. Traverse each node in the knowledge base and calculate the keyword similarity score and vector similarity score of that node for the user query.
[0186] 4. Sort and filter the scores obtained by all nodes in step 3 to obtain a candidate standard question sequence;
[0187] 5. Among the candidate standard questions, "What is the phone number of Manager ZXX in the Finance Department?" received the highest score and was the only standard question in the sequence.
[0188] 6. Return the unique standard question and its corresponding answer.
[0189] Example 2 (situations requiring follow-up questions):
[0190] 1. The user enters the query "I want to find the product manager's phone number";
[0191] 2. Perform keyword extraction on the query statement, extracting keywords containing "product," "manager," and "telephone," etc. Simultaneously, generate a vector representation of the query;
[0192] 3. Traverse each node in the knowledge base and calculate the keyword similarity score and vector similarity score of that node for the user query.
[0193] 4. Sort and filter the scores obtained by all nodes in step 3 to obtain a candidate standard question sequence;
[0194] 5. In the candidate standard question sequence, "What is the phone number of Manager W in the Product R&D Department?" and "What is the phone number of Manager L in the Product Development Department?" have very similar scores and are the only two standard questions in the sequence;
[0195] 6. The system finds the common ancestor node of two similar questions in the concept tree structure, and generates a follow-up question based on this: "Is this the R&D manager or the development manager?"
[0196] 7. The user answers the follow-up question by saying, "I need the phone number of the product development manager."
[0197] 8. The system extracts the keywords "R&D, manager" based on the user's supplementary information. It then prunes the candidate sequence from the standard question in step 4 based on these keywords.
[0198] 9. The pruning result is the only criterion. Ask, "What is the phone number of the Product Development Department Manager?"
[0199] 10. Return to the unique standard question and its corresponding answer.
[0200] The question-and-answer processing apparatus provided in this application is described below. The question-and-answer processing apparatus described below can be referred to in correspondence with the question-and-answer processing method described above.
[0201] Furthermore, this application also provides a question-and-answer processing device.
[0202] The question-and-answer processing device includes:
[0203] The first determining module is used to determine a list of keywords based on the input text;
[0204] The second determining module is used to determine the matching score between the input text and each preset question based on the keyword list and the keyword set corresponding to each preset question in the preset knowledge base.
[0205] The input module is used to input the input text and each of the preset questions into the word embedding model respectively, and obtain the text vectors output by the word embedding model respectively;
[0206] The third determining module is used to determine the similarity score between the text vector of the input text and the text vector corresponding to each preset question;
[0207] The fourth determining module is used to determine the target question from each preset question based on each matching score and each similarity score;
[0208] The fifth determining module is used to determine the response content of the input text based on the target question.
[0209] The question-answering processing device of this application determines a keyword list based on the input text, enabling the determination of a matching score between the input text and each preset question based on the keyword list and the keyword sets corresponding to each preset question in a preset knowledge base. Furthermore, the input text and each preset question are input into a word embedding model to obtain text vectors output by the word embedding model, allowing the determination of a similarity score between the text vector of the input text and the text vector corresponding to each preset question. Further, based on the matching and similarity scores, a target question can be accurately determined from the preset questions, and the response content of the input text can be accurately determined based on the target question. Because it combines keyword matching scoring with artificial intelligence matching scoring, a more accurate standard question for the input text can be selected from multiple preset questions serving as standard questions. The multi-dimensional semantic similarity evaluation criteria make the results more accurate, thus capturing semantic complexity, improving retrieval accuracy, and consequently improving the accuracy of intelligent question-answering conversation processing.
[0210] In one embodiment, the second determining module is specifically used to perform the following steps for each preset question when determining the matching score between the input text and each preset question based on the keyword list and the keyword set corresponding to each preset question in the preset knowledge base:
[0211] Determine the keyword matching result of each keyword in the keyword list in the keyword set corresponding to the current preset question in the preset knowledge base;
[0212] Determine the matching score for each keyword matching result;
[0213] Each matching score is multiplied by the weight of the corresponding keyword to obtain the weight score;
[0214] The weighted scores of each keyword are summed to obtain the matching score between the input text and the current preset question.
[0215] In one embodiment, the fourth determining module is specifically used for:
[0216] The matching score for each preset question is multiplied by the first weight to obtain the first score corresponding to each preset question.
[0217] The similarity score of each preset question is multiplied by the second weight to obtain the second score corresponding to each preset question.
[0218] Add the first score and the second score for each preset question to obtain the comprehensive score for each preset question.
[0219] Based on the comprehensive scores, the target question is determined from the preset questions.
[0220] In one embodiment, the fifth determining module is specifically used for:
[0221] If there are multiple target problems, the common ancestor node of each target problem is determined from the concept tree; wherein, the concept tree is a tree structure constructed based on multiple preset problems, the keywords corresponding to each preset problem, and the weights of the keywords corresponding to each preset problem.
[0222] Based on the common ancestor node, the concept tree is pruned to obtain a list of common ancestor child nodes;
[0223] Based on the list of common ancestor child nodes, determine and output the follow-up questions;
[0224] Upon receiving supplementary text returned based on the follow-up question, the target child node is determined from the list of common ancestor child nodes based on the supplementary text;
[0225] The response content of the preset question corresponding to the target sub-node is used as the response content of the input text.
[0226] In one embodiment, the fifth determining module includes a determining unit, the determining unit being used to:
[0227] Determine the set of keywords for the supplementary text;
[0228] The supplementary text is matched with each common ancestor child node in the list of common ancestor child nodes by keyword similarity.
[0229] If the common ancestor child node with the highest similarity is a leaf node in the concept tree, then the common ancestor child node with the highest similarity is determined as the target child node.
[0230] Figure 3 An example is a schematic diagram of the physical structure of an electronic device, such as... Figure 3As shown, the electronic device may include a processor 310, a communications interface 320, a memory 330, and a communication bus 340, wherein the processor 310, communications interface 320, and memory 330 communicate with each other via the communication bus 340. The processor 310 can call logical instructions in the memory 330 to execute the following method: determining a list of keywords based on input text;
[0231] Based on the keyword list and the keyword set corresponding to each preset question in the preset knowledge base, the matching score between the input text and each preset question is determined.
[0232] The input text and each of the preset questions are respectively input into the word embedding model to obtain the text vectors output by the word embedding model respectively;
[0233] Determine the similarity score between the text vector of the input text and the text vector corresponding to each preset question;
[0234] Based on the matching scores and similarity scores, the target question is determined from each preset question;
[0235] The response content of the input text is determined based on the target question.
[0236] Furthermore, the logical instructions in the aforementioned memory 330 can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to related technologies, or a portion of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.
[0237] In another aspect, embodiments of this application also provide a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, is implemented to perform the methods provided in the above embodiments, such as: determining a list of keywords based on input text;
[0238] Based on the keyword list and the keyword set corresponding to each preset question in the preset knowledge base, the matching score between the input text and each preset question is determined.
[0239] The input text and each of the preset questions are respectively input into the word embedding model to obtain the text vectors output by the word embedding model respectively;
[0240] Determine the similarity score between the text vector of the input text and the text vector corresponding to each preset question;
[0241] Based on the matching scores and similarity scores, the target question is determined from each preset question;
[0242] The response content of the input text is determined based on the target question.
[0243] In another aspect, embodiments of this application also provide a computer program product having a computer program stored thereon, which, when executed by a processor, is implemented to perform the methods provided in the above embodiments, such as: determining a list of keywords based on input text;
[0244] Based on the keyword list and the keyword set corresponding to each preset question in the preset knowledge base, the matching score between the input text and each preset question is determined.
[0245] The input text and each of the preset questions are respectively input into the word embedding model to obtain the text vectors output by the word embedding model respectively;
[0246] Determine the similarity score between the text vector of the input text and the text vector corresponding to each preset question;
[0247] Based on the matching scores and similarity scores, the target question is determined from each preset question;
[0248] The response content of the input text is determined based on the target question.
[0249] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without any creative effort.
[0250] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the parts that contribute to the related technology, can be embodied in the form of software products. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments or some parts of the embodiments.
[0251] Finally, it should be noted that the above embodiments are only used to illustrate this application and are not intended to limit this application. Although this application has been described in detail with reference to the embodiments, those skilled in the art should understand that various combinations, modifications, or equivalent substitutions of the technical solutions of this application do not depart from the spirit and scope of the technical solutions of this application.
Claims
1. A question-and-answer processing method, characterized in that, include: Determine the keyword list based on the input text; Based on the keyword list and the keyword set corresponding to each preset question in the preset knowledge base, the matching score between the input text and each preset question is determined. The input text and each of the preset questions are respectively input into the word embedding model to obtain the text vectors output by the word embedding model respectively; Determine the similarity score between the text vector of the input text and the text vector corresponding to each preset question; Based on the matching scores and similarity scores, the target question is determined from each preset question; The response content of the input text is determined based on the target question; Determining the response content of the input text based on the target question includes: If there are multiple target questions, the common ancestor node of each target question is determined from the concept tree. The concept tree is a tree structure constructed based on multiple preset questions, keywords corresponding to each preset question, and the weights of the keywords corresponding to each preset question. The nodes of the concept tree include leaf nodes and non-leaf nodes. Each leaf node corresponds to a single real preset question in a preset knowledge base. Each non-leaf node is a cluster summary of the preset questions corresponding to its multiple subordinate leaf nodes. The common ancestor node is the non-leaf node in the concept tree that has the closest common ancestor relationship among all target questions. Based on the common ancestor node, the concept tree is pruned to obtain a list of common ancestor child nodes; Based on the list of common ancestor child nodes, determine and output the follow-up questions; Upon receiving supplementary text returned based on the follow-up question, the target child node is determined from the list of common ancestor child nodes based on the supplementary text; The response content of the preset question corresponding to the target sub-node is used as the response content of the input text.
2. The question-and-answer processing method according to claim 1, characterized in that, The step of determining the target child node from the list of common ancestor child nodes based on the supplementary text includes: Determine the set of keywords for the supplementary text; The supplementary text is matched with each common ancestor child node in the list of common ancestor child nodes by keyword similarity. If the common ancestor child node with the highest similarity is a leaf node in the concept tree, then the common ancestor child node with the highest similarity is determined as the target child node.
3. The question-and-answer processing method according to claim 1, characterized in that, The concept tree is constructed based on the following steps: Hierarchical clustering is performed based on multiple pre-defined questions to obtain a clustering tree; The clustering tree is optimized based on a large language model and preset prompt words to obtain an optimized clustering tree; The keywords of the preset question corresponding to each node in the optimized clustering tree and the weights of the keywords of the preset question corresponding to each node are added to the corresponding nodes to obtain the concept tree.
4. The question-and-answer processing method according to claim 1, characterized in that, When determining the matching score between the input text and each preset question based on the keyword list and the keyword set corresponding to each preset question in the preset knowledge base, the following steps are performed for each preset question: Determine the keyword matching result of each keyword in the keyword list in the keyword set corresponding to the current preset question in the preset knowledge base; Determine the matching score for each keyword matching result; Each matching score is multiplied by the weight of the corresponding keyword to obtain the weight score; The weighted scores of each keyword are summed to obtain the matching score between the input text and the current preset question.
5. The question-and-answer processing method according to claim 1, characterized in that, The step of determining the target question from each preset question based on each matching score and each similarity score includes: The matching score for each preset question is multiplied by the first weight to obtain the first score corresponding to each preset question. The similarity score of each preset question is multiplied by the second weight to obtain the second score corresponding to each preset question. Add the first score and the second score for each preset question to obtain the comprehensive score for each preset question. Based on the comprehensive scores, the target question is determined from the preset questions.
6. A question-and-answer processing device, characterized in that, include: The first determining module is used to determine a list of keywords based on the input text; The second determining module is used to determine the matching score between the input text and each preset question based on the keyword list and the keyword set corresponding to each preset question in the preset knowledge base. The input module is used to input the input text and each of the preset questions into the word embedding model respectively, and obtain the text vectors output by the word embedding model respectively; The third determining module is used to determine the similarity score between the text vector of the input text and the text vector corresponding to each preset question; The fourth determining module is used to determine the target question from each preset question based on each matching score and each similarity score; The fifth determining module is used to determine the response content of the input text based on the target question; The fifth determining module is further configured to, if there are multiple target questions, determine the common ancestor node of each target question from the concept tree; wherein, the concept tree is a tree structure constructed based on multiple preset questions, keywords corresponding to each preset question, and weights of the keywords corresponding to each preset question; the nodes of the concept tree include leaf nodes and non-leaf nodes; the leaf nodes correspond to a single real preset question in the preset knowledge base; the non-leaf nodes are cluster summaries of the preset questions corresponding to their subordinate leaf nodes; the common ancestor node is a non-leaf node in the concept tree that has the closest common ancestor relationship among the target questions; the concept tree is pruned based on the common ancestor node to obtain a list of common ancestor child nodes; follow-up questions are determined based on the list of common ancestor child nodes and output; supplementary text returned based on the follow-up questions is received, and target child nodes are determined from the list of common ancestor child nodes based on the supplementary text; the response content of the preset question corresponding to the target child node is used as the response content of the input text.
7. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the question-and-answer processing method as described in any one of claims 1-5.
8. A storage medium, said storage medium being a non-transitory computer-readable storage medium, wherein a computer program is stored thereon, characterized in that, When the computer program is executed by a processor, it implements the question-and-answer processing method as described in any one of claims 1-5.
9. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the question-and-answer processing method according to any one of claims 1-5.