Query statement management method and device, readable storage medium, and electronic device
By classifying and aggregating the query statements and combining with standard question matching, the problem of clustering inaccuracy is solved, the processing efficiency and accuracy of the query statements is improved, and the effective matching of the problems raised by different methods is achieved.
Patent Information
- Application Number
- CN202010824310.0
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2020-08-17
- Publication Date
- 2024-11-22
- Estimated Expiration
- 2040-08-17
AI Technical Summary
The existing technology has clustering inaccuracy problems in query statement management, which leads to insufficient accuracy of the same type of problems and affects processing efficiency.
By obtaining the query statements in the log, performing classification processing and aggregation processing, forming a cluster set, and matching standard questions for each cluster set, determining the matching standard questions to improve processing efficiency and accuracy.
The matching of essentially the same problems raised by different methods is achieved, the processing efficiency of query statements is improved, the accuracy of clustering is enhanced, and the efficient matching of different query statements is ensured.
Smart Images

Figure CN111949785B_ABST
Abstract
Description
Technical Field
[0001] The present disclosure relates to the technical field of statement management, and in particular to a query statement management method and device, a readable storage medium, and an electronic device. Background Art
[0002] For business scenarios involving question-and-answer, many business problems will be encountered in the process of answering questions. For example, real estate agents will encounter problems such as the loan ratio of the second house and how to calculate taxes. Real estate agents themselves also have questions that need to be answered, such as the interpretation of a certain company rule or regulation and how to withdraw housing provident funds. Since various problems facing different groups of people need to be solved, knowledge management needs to accumulate and precipitate the problems and generate corresponding knowledge, providing a convenient search portal so that people can directly search for answers when they have the same questions in the future.
[0003] However, different people have different ways of asking questions. The same question may be asked in multiple ways. For example, for the question "How to modify and reset the email password", possible questions include "What should I do if I forget my email password", "How to modify my email password", "What are the rules for modifying email passwords", "How to retrieve my email password if it is lost", etc.; therefore, it is necessary to manage query statements. Summary of the invention
[0004] In order to solve the above technical problems, the present disclosure is proposed. The embodiments of the present disclosure provide a query statement management method and device, a readable storage medium, and an electronic device.
[0005] According to one aspect of an embodiment of the present disclosure, a query statement management method is provided, including:
[0006] Get multiple query statements in the log that meet the preset conditions;
[0007] Classify the multiple query statements, and perform aggregation processing based on the classification results to obtain at least one cluster set; wherein each cluster set includes at least one query statement;
[0008] Standard question matching is performed on each query statement in the cluster set respectively, and a set number of matching standard question sentences is determined for each cluster set.
[0009] Optionally, the preset conditions include at least one of the following: having an answer but no click, or having no answer;
[0010] The obtaining of multiple query statements in the log that meet the preset conditions includes:
[0011] The query statements obtained from the log are screened based on the preset conditions to obtain multiple query statements that meet the preset conditions.
[0012] Optionally, the classifying process is performed on the multiple query statements, and aggregation process is performed based on the classification process result to obtain at least one clustering set, including:
[0013] Classifying the multiple query statements into at least one category; wherein each category includes at least two query statements;
[0014] Clustering is performed on at least two query statements included in each category to obtain at least one cluster set.
[0015] Optionally, the classifying the multiple query statements into at least one category includes:
[0016] For each of the multiple query statements, segment the query statement to obtain at least one word;
[0017] At least one word vector corresponding to the at least one term is processed through a classification model to determine a category label corresponding to the query statement; wherein the classification model is trained with a training statement set with known category labels, and the statement training set corresponds to the business scenario of the query statement.
[0018] Optionally, performing standard question matching on each query statement in the cluster set respectively and determining a set number of matching standard questions for each cluster set includes:
[0019] For each of the query statements in the cluster set, the query statement and each of the plurality of pre-stored standard questions are scored by inputting a network model to obtain a score of matching between the query statement and each of the plurality of standard questions;
[0020] Determining a set number of standard questions matching the query sentence based on the score;
[0021] Based on the standard questions matching each of the query sentences included in the cluster set, a set number of matching standard questions corresponding to the cluster set is determined.
[0022] Optionally, determining a set number of standard questions matching the query statement based on the score includes:
[0023] sorting the scores corresponding to the plurality of standard questions according to the sizes of the scores;
[0024] According to the ranking, a set number of the standard questions with the highest scores are determined as the standard questions of the query statement.
[0025] Optionally, determining a set number of matching standard questions corresponding to the cluster set based on the standard questions matching each query statement included in the cluster set includes:
[0026] Based on the standard question sentence matching each of the query sentences included in the cluster set, obtaining a plurality of standard question sentences corresponding to the cluster set;
[0027] Based on the score corresponding to each of the plurality of standard questions, a set number of the standard questions are determined as matching standard questions corresponding to the cluster set.
[0028] Optionally, the step of scoring the query statement with each of the plurality of pre-stored standard questions by inputting a network model to obtain a score of the query statement matching each of the plurality of standard questions includes:
[0029] Performing word segmentation on the query statement and each of the standard questions respectively, and determining a plurality of word vectors corresponding to the query statement and a plurality of word vectors corresponding to the standard questions;
[0030] Processing multiple word vectors corresponding to the query sentence through a bidirectional long short-term memory network to obtain sentence features; processing multiple word vectors corresponding to the standard question sentence through a bidirectional long short-term memory network to obtain standard question features;
[0031] Connecting the sentence feature and the standard question feature to obtain a connection feature;
[0032] A one-dimensional matrix is determined based on the connection features, and a score of matching between the query statement and the standard question and a position of the standard question among the multiple standard questions are determined through the one-dimensional matrix; wherein the one-dimensional matrix represents a matching relationship between the query statement and the multiple standard questions.
[0033] Optionally, it also includes:
[0034] Based on the matching standard question, the answer content corresponding to the query statement is determined in the cluster set, and the query statement and the answer content are stored in a knowledge base.
[0035] Optionally, the determining, based on the matching standard question, that the cluster set includes answer content corresponding to the query statement includes:
[0036] Based on the set number of matching standard questions, determining a set number of answer contents corresponding to the standard questions;
[0037] The set number of answer contents are used as answer contents corresponding to the cluster set.
[0038] According to another aspect of an embodiment of the present disclosure, a query statement management device is provided, including:
[0039] A statement acquisition module is used to obtain multiple query statements in the log that meet preset conditions;
[0040] A statement processing module, used for classifying the multiple query statements, and performing aggregation processing based on the classification processing results to obtain at least one cluster set; wherein each of the cluster sets includes at least one query statement;
[0041] The standard matching module is used to perform standard question matching on each query statement in the cluster set, and determine a set number of matching standard questions for each cluster set.
[0042] Optionally, the preset conditions include at least one of the following: having an answer but no click, or having no answer;
[0043] The statement acquisition module is specifically used to filter the query statements obtained from the log based on the preset conditions to obtain multiple query statements that meet the preset conditions.
[0044] Optionally, the statement processing module includes:
[0045] A classification unit, configured to classify the multiple query statements into at least one category; wherein each category includes at least two query statements;
[0046] The clustering unit is used to cluster the at least two query statements included in each category to obtain at least one cluster set.
[0047] Optionally, the classification unit is specifically used to segment the query statement for each of the multiple query statements to obtain at least one word; process at least one word vector corresponding to the at least one word through a classification model to determine the category label corresponding to the query statement; wherein the classification model is trained with a training statement set with known category labels, and the statement training set corresponds to the business scenario of the query statement.
[0048] Optionally, the standard matching module includes:
[0049] a scoring unit, configured to score each query statement in the cluster set with each of a plurality of pre-stored standard questions by inputting a network model to obtain a score of matching between the query statement and each of the plurality of standard questions;
[0050] A sentence matching unit, configured to determine a set number of standard question sentences matching the query sentence based on the score;
[0051] The set matching unit is used to determine a set number of matching standard questions corresponding to the cluster set based on the standard questions matching each of the query sentences included in the cluster set.
[0052] Optionally, the sentence matching unit is specifically configured to sort the multiple scores corresponding to the multiple standard questions according to the sizes of the scores; and according to the sorting, determine a set number of the standard questions with the highest scores as the standard questions of the query sentence.
[0053] Optionally, the set matching unit is specifically used to obtain multiple standard questions corresponding to the cluster set based on the standard questions matching each of the query statements included in the cluster set; and determine a set number of the standard questions as matching standard questions corresponding to the cluster set based on the scores corresponding to each of the multiple standard questions.
[0054] Optionally, the scoring unit is specifically used to perform word segmentation on the query statement and each of the standard questions respectively, determine multiple word vectors corresponding to the query statement and multiple word vectors corresponding to the standard questions; process the multiple word vectors corresponding to the query statement through a bidirectional long short-term memory network to obtain sentence features; process the multiple word vectors corresponding to the standard questions through a bidirectional long short-term memory network to obtain standard question features; connect the sentence features and the standard question features to obtain connection features; determine a one-dimensional matrix based on the connection features, and determine the score of the query statement matching the standard question and the position of the standard question in the multiple standard questions through the one-dimensional matrix; wherein the one-dimensional matrix represents the matching relationship between the query statement and the multiple standard questions.
[0055] Optionally, the device further comprises:
[0056] The answer matching module is used to determine the answer content corresponding to the query statement in the cluster set based on the matching standard question, and store the query statement and the answer content in a knowledge base.
[0057] Optionally, the answer matching module is specifically used to determine a set number of answer contents corresponding to the standard questions based on the set number of matching standard questions; and use the set number of answer contents as answer contents corresponding to the cluster set.
[0058] According to another aspect of the embodiments of the present disclosure, a computer-readable storage medium is provided, wherein the storage medium stores a computer program, and the computer program is used to execute the query statement management method described in any of the above embodiments.
[0059] According to another aspect of the embodiments of the present disclosure, an electronic device is provided, wherein the electronic device includes:
[0060] processor;
[0061] a memory for storing instructions executable by the processor;
[0062] The processor is used to read the executable instructions from the memory and execute the instructions to implement the query statement management method described in any one of the above embodiments.
[0063] Based on a query statement management method and device, a readable storage medium, and an electronic device provided in the above-mentioned embodiments of the present disclosure, multiple query statements that meet preset conditions are obtained in a log; the multiple query statements are classified and processed, and aggregation processing is performed based on the results of the classification processing to obtain at least one cluster set; wherein each cluster set includes at least one query statement; standard question matching is performed on each of the query statements in the cluster set, and a set number of matching standard questions is determined for each cluster set; this embodiment determines a set number of matching standard questions that match the cluster set through classification and aggregation processing, thereby matching corresponding standard questions to substantially the same questions raised in different ways, so as to improve the processing efficiency of different query statements.
[0064] The technical solution of the present disclosure is further described in detail below through the accompanying drawings and embodiments. BRIEF DESCRIPTION OF THE DRAWINGS
[0065] The above and other purposes, features and advantages of the present disclosure will become more apparent by describing the embodiments of the present disclosure in more detail in conjunction with the accompanying drawings. The accompanying drawings are used to provide a further understanding of the embodiments of the present disclosure and constitute a part of the specification. Together with the embodiments of the present disclosure, they are used to explain the present disclosure and do not constitute a limitation of the present disclosure. In the drawings, the same reference numerals generally represent the same components or steps.
[0066] Figure 1 It is a flowchart of a query statement management method provided by an exemplary embodiment of the present disclosure.
[0067] Figure 2 This disclosure Figure 1 A schematic flow chart of step 104 in the illustrated embodiment.
[0068] Figure 3 This disclosure Figure 2 A flowchart diagram of step 1041 in the embodiment shown.
[0069] Figure 4 It is a schematic diagram of the process of obtaining training query statements in the query statement management method provided in an embodiment of the present disclosure.
[0070] Figure 5 This disclosure Figure 1 A schematic flow chart of step 106 in the illustrated embodiment.
[0071] Figure 6 It is a flowchart of a label-embeding model processing query statements provided by an optional embodiment of the present disclosure.
[0072] Figure 7 It is a flowchart of a query statement management method provided by another exemplary embodiment of the present disclosure.
[0073] Figure 8 It is a structural diagram of a query statement management device provided by an exemplary embodiment of the present disclosure.
[0074] Fig. 9 is a structural diagram of an electronic device provided by an exemplary embodiment of the present disclosure. DETAILED DESCRIPTION
[0075] Below, the exemplary embodiments according to the present disclosure will be described in detail with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of the present disclosure, rather than all the embodiments of the present disclosure, and it should be understood that the present disclosure is not limited to the exemplary embodiments described here.
[0076] It should be noted that the relative arrangement of components and steps, the numerical expressions and numerical values set forth in these embodiments do not limit the scope of the present disclosure unless specifically stated otherwise.
[0077] Those skilled in the art can understand that the terms "first" and "second" in the embodiments of the present disclosure are only used to distinguish different steps, devices or modules, etc., and neither represent any specific technical meaning nor indicate the necessary logical order between them.
[0078] It should also be understood that in the embodiments of the present disclosure, “plurality” may refer to two or more than two, and “at least one” may refer to one, two, or more than two.
[0079] It should also be understood that any component, data or structure mentioned in the embodiments of the present disclosure can generally be understood as one or more, unless explicitly limited or otherwise indicated in the context.
[0080] In addition, the term "and / or" in the present disclosure is only a description of the association relationship of associated objects, indicating that three relationships may exist. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone. In addition, the character " / " in the present disclosure generally indicates that the associated objects before and after are in an "or" relationship.
[0081] It should also be understood that the description of the various embodiments in the present disclosure focuses on the differences between the various embodiments, and the same or similar aspects thereof can be referenced to each other, and for the sake of brevity, they will not be described one by one.
[0082] At the same time, it should be understood that for the convenience of description, the sizes of the various parts shown in the drawings are not drawn according to the actual proportional relationship.
[0083] The following description of at least one exemplary embodiment is merely illustrative in nature and is in no way intended to limit the present disclosure, its application, or uses.
[0084] Technologies, methods, and equipment known to ordinary technicians in the relevant art may not be discussed in detail, but where appropriate, the technologies, methods, and equipment should be considered part of the specification.
[0085] It should be noted that like reference numerals and letters refer to similar items in the following figures, and therefore, once an item is defined in one figure, it need not be further discussed in subsequent figures.
[0086] The disclosed embodiments can be applied to electronic devices such as terminal devices, computer systems, servers, etc., which can operate with many other general or special computing system environments or configurations. Examples of well-known terminal devices, computing systems, environments and / or configurations suitable for use with electronic devices such as terminal devices, computer systems, servers, etc. include, but are not limited to: personal computer systems, server computer systems, thin clients, thick clients, handheld or laptop devices, microprocessor-based systems, set-top boxes, programmable consumer electronics, network personal computers, small computer systems, large computer systems, and distributed cloud computing technology environments including any of the above systems, etc.
[0087] Electronic devices such as terminal devices, computer systems, servers, etc. can be described in the general context of computer system executable instructions (such as program modules) executed by computer systems. Generally, program modules can include routines, programs, object programs, components, logic, data structures, etc., which perform specific tasks or implement specific abstract data types. Computer systems / servers can be implemented in a distributed cloud computing environment, where tasks are performed by remote processing devices linked through a communication network. In a distributed cloud computing environment, program modules can be located on local or remote computing system storage media including storage devices.
[0088] Application Overview
[0089] In the process of implementing the present disclosure, the inventors found that the management of query statements in the prior art usually adopts kmeans algorithm clustering to cluster the same type of questions as much as possible. This technical solution has at least the following problems: the amount of data after hourly data deduplication is small, the accuracy of hourly clustering is insufficient, and the clustering is inaccurate.
[0090] Exemplary Methods
[0091] Figure 1 FIG. 1 is a flowchart of a query statement management method provided by an exemplary embodiment of the present disclosure. This embodiment can be applied to electronic devices, such as Figure 1 As shown, the following steps are included:
[0092] Step 102: Obtain multiple query statements in the log that meet preset conditions.
[0093] Optionally, the query statements in the logs can be obtained by logging, where logging is a term in the field of data collection (especially user behavior data collection), which refers to the related technologies and implementation processes for capturing, processing and sending specific user behaviors or events.
[0094] Optionally, for the preset conditions in this embodiment, multiple query statements meeting the preset conditions may be obtained by screening the acquired query statements.
[0095] Step 104 , classify the multiple query statements, and perform aggregation processing based on the classification processing results to obtain at least one cluster set.
[0096] Each clustering set includes at least one query statement. In order to overcome the disadvantage of inaccurate clustering in the prior art, the present embodiment adds classification of query statements, and improves the accuracy of clustering through classification.
[0097] Step 106 , perform standard question matching on each query statement in the cluster set, and determine a set number of matching standard questions for the cluster set.
[0098] In this embodiment, in the same cluster set, by matching each query statement with a standard question respectively, a set number of matching standard questions corresponding to each cluster set are obtained. Compared with matching each query statement separately, this improves processing efficiency and makes the matching standard questions more matched with the cluster set.
[0099] The above-mentioned embodiment of the present disclosure provides a query statement management method, which obtains multiple query statements that meet preset conditions in a log; classifies the multiple query statements, and performs aggregation processing based on the results of the classification processing to obtain at least one cluster set; wherein each of the cluster sets includes at least one query statement; performs standard question matching on each of the query statements in the cluster set, and determines a set number of matching standard questions for each cluster set; this embodiment determines a set number of matching standard questions that match the cluster set through classification and aggregation processing, thereby matching corresponding standard questions to substantially the same questions raised in different ways, so as to improve the processing efficiency of different query statements.
[0100] In some optional embodiments, the preset conditions include but are not limited to at least one of the following: having an answer but no click, having no answer, etc.;
[0101] In this embodiment, a click refers to a click by the person being asked the question, which can be considered as the questioner's approval of the answer; no answer means that there is no corresponding answer content in the log for the query statement; there is an answer but no click means that there is an answer content in the log for the query statement, but the questioner has not clicked on these answers.
[0102] Step 102 includes:
[0103] The query statements obtained from the log are filtered based on the preset conditions to obtain multiple query statements that meet the preset conditions.
[0104] This embodiment uses conditional screening to consider query statements that have answers and clicks as standard questions, and have corresponding answer content that has been clicked by the questioner. At this time, such query statements are not processed; this embodiment mainly matches standard questions with query statements that have no answers or have answers but no clicks to obtain corresponding answer content.
[0105] like Figure 2 As shown in the above Figure 1 Based on the illustrated embodiment, step 104 may include the following steps:
[0106] Step 1041: classify the multiple query statements into at least one category.
[0107] Each category includes at least two query statements.
[0108] The category in this embodiment may be a category corresponding to a business scenario corresponding to a query statement. For example, multiple category labels are obtained through manual summarization. During classification, the category of the query statement is determined by determining the category label corresponding to the query statement.
[0109] Step 1042: cluster at least two query statements included in each category to obtain at least one cluster set.
[0110] In this embodiment, clustering is performed on multiple query statements included in each category in the classification results. Optionally, kmeans clustering is used. By clustering the query statements in each category, query statements of the same category with similar contents are clustered together as much as possible, so as to unify operations when summarizing standard questions in the later stage.
[0111] like Figure 3 As shown in the above Figure 2 Based on the illustrated embodiment, step 1041 may include the following steps:
[0112] Step 301: for each query statement in a plurality of query statements, segment the query statement to obtain at least one word.
[0113] This embodiment decomposes the query statement into multiple words through word segmentation, and processes each word separately to obtain a corresponding word vector.
[0114] Step 302: Process at least one word vector corresponding to at least one word through a classification model to determine a category label corresponding to the query statement.
[0115] Among them, the classification model is trained with a set of training sentences with known category labels, and the sentence training set corresponds to the business scenario of the query sentence.
[0116] Optionally, this embodiment uses a classification model to process at least one word vector corresponding to the query statement. For example, the fasttext model is used for classification. Fasttext is an open source word vector and text classification tool, which is very similar to word2vec in model architecture. Before using the fasttext model for classification, the fasttext model needs to be trained. When processing the training corpus, the training query is first segmented. Optionally, the segmentation is based on a keyword segmenter in the business scenario field plus a dedicated dictionary in the knowledge base; to avoid the impact of segmentation errors, the 2-gram feature of the word is added during training (n-gram is an algorithm based on a statistical language model. Its basic idea is to perform a sliding window operation of size n on the content of the text according to bytes, forming a sequence of byte segments of length n; each byte segment is called a gram), such as Figure 4 The training query statement processing process shown. In addition, a neural network model is used to obtain word vectors during training. Before applying the neural network model, pre-training is performed to improve the accuracy of the word vectors of the words. The pre-trained corpus is a plurality of query statements and their corresponding titles and contents in the business scenarios corresponding to the query statements (for example, 500,000+ queries in the real estate field, 30,000+ titles and contents of the knowledge base). Pre-training is performed in advance through large data to obtain the underlying parameters (word vectors) with strong versatility, which are then directly applied to the classification task here to enhance the model effect. The process of classification using the trained fasttext model in this embodiment is to process the input word vector and directly output the category label. For example, the content label is manually summarized and sorted into multiple (e.g., 213) categories of labels. During prediction, the query with good word classification is input into the classification model to obtain the predicted content label of the query. For example, query statement 1: How to make an appointment with a VR photographer, after classification, its category label is determined to be: field survey; query statement 2: Can the resignation form be printed only after the resignation review is passed, after classification, its category label is determined to be: resignation; query statement 3: What to do when the contract expires, after classification, its category label is determined to be: contract; query statement 4: How to modify the designated hospital, after classification, its category label is determined to be: social security provident fund.
[0117] like Figure 5 As shown in the above Figure 1 Based on the illustrated embodiment, step 106 may include the following steps:
[0118] Step 1061 , for each query statement in the cluster set, the query statement and each of the pre-stored standard questions are scored by inputting a network model to obtain a score of matching the query statement with each of the pre-stored standard questions.
[0119] Step 1062: Determine a set number of standard questions that match the query statement based on the score.
[0120] Step 1063: Determine a set number of matching standard questions corresponding to the cluster set based on the standard questions matching each query statement included in the cluster set.
[0121] To avoid inaccurate clustering sets obtained by clustering, in this embodiment, a set number of standard questions are matched for each query in each clustering set through a network model (such as a label-embedding model). Optionally, a standard question match is made through a label-embedding model. In this model, standard questions are used as labels, which are also a summary of the query and contain a lot of useful semantic information. This overcomes the shortcomings of other common classification models such as SVM, LSTM, etc., which convert labels into one-hot encodings that have no actual meaning and exist independently of text information, resulting in information loss. The label-embedding model used in this embodiment converts labels into semantic vectors and converts classification tasks into vector matching tasks.
[0122] Optionally, step 1062 includes:
[0123] Sort multiple scores corresponding to multiple standard questions according to the size of the scores;
[0124] According to the ranking, a set number of standard questions with the highest scores are determined as standard questions of the query statement.
[0125] In this embodiment, the scores corresponding to all standard questions are sorted to obtain a set number of standard questions with the highest scores. The efficiency of obtaining standard questions is improved by sorting. For example, the scores are sorted in descending order. When obtaining matching standard questions, only a set number (such as 3 or 4) of standard questions before sorting need to be obtained in order to obtain matching standard questions.
[0126] Optionally, step 1063 includes:
[0127] Based on the standard question sentence matching each query sentence included in the cluster set, a plurality of standard question sentences corresponding to the cluster set are obtained;
[0128] Based on the score corresponding to each standard question in the plurality of standard questions, a set number of standard questions are determined as matching standard questions corresponding to the cluster set.
[0129] This embodiment obtains a corresponding matching standard question for each query statement in the cluster set to obtain multiple standard questions corresponding to the cluster set. In order to obtain a set number of matching standard questions corresponding to the cluster set, multiple standard questions can be screened to obtain a set number of standard questions with the highest scores as matching standard questions.
[0130] In some optional embodiments, step 1061 includes:
[0131] The query statement and each standard question are segmented respectively to determine multiple word vectors corresponding to the query statement and multiple word vectors corresponding to the standard question;
[0132] The multiple word vectors corresponding to the query sentence are processed through a bidirectional long short-term memory network to obtain sentence features; the multiple word vectors corresponding to the standard question sentence are processed through a bidirectional long short-term memory network to obtain standard question features;
[0133] Connect the sentence features and the standard question features to obtain the connection features;
[0134] A one-dimensional matrix is determined based on the connection features, and the score of the query sentence matching the standard question sentence and the position of the standard question sentence in the plurality of standard question sentences are determined through the one-dimensional matrix.
[0135] The one-dimensional matrix represents the matching relationship between the query sentence and multiple standard questions.
[0136] This embodiment adopts a supervised label-embedding model based on Bilstm. The model structure has 4 layers, such as Figure 6 As shown in the figure, the first layer is the look-up layer, which obtains the embedding vector of each word; the second layer is the Bilstm layer. This implementation uses two bidirectional LSTMs to make the sentence expression more accurate and extract the sentence features more accurately; the third layer concatenates the Bilstm expressions of the query and label and adds them to the weight matrix M 2m*1 The weight matrix is transformed into one dimension by operation (the value of the weight matrix is determined through training), and finally normalized by Sigmoid activation function to score, and the matching score represented by a one-dimensional vector and the position of the corresponding standard question are obtained.
[0137] Before applying the label-embeding model for standard question matching, the weight matrix M is learned through multiple rounds of training. 2m*1, so that the score output by the model is more accurate. As a supervised learning task, its true label is the one-hot vector of the label. The pre-trained word vector is used here. The query and label are passed through the same tokenizer, and their respective embedding vectors are obtained after tokenization. Get the position of the standard question. Optionally, the label-embedding training data set is generated by: using the query as input and the standard question as output to train a fasttext classification model, and obtaining the top few with the highest scores (for example, top5); manually screening which of the top few is the correct labeled question of the query; combining the correct standard question and the query into a positive sample pair, and the remaining standard questions and the query into a negative sample pair; the training data set generated by this method can better distinguish between standard questions that can answer the query and similar but cannot answer the query.
[0138] Figure 7 FIG. 1 is a flow chart of a query statement management method provided by another exemplary embodiment of the present disclosure. Figure 7 As shown, the following steps are included:
[0139] Step 102: Obtain multiple query statements in the log that meet preset conditions.
[0140] Step 104: classify and aggregate multiple query statements to obtain at least one cluster set, wherein the cluster set includes at least one query statement.
[0141] Step 106 , perform standard question matching on each query statement in the cluster set, and determine a set number of matching standard questions for the cluster set.
[0142] Step 708: Determine the answer content corresponding to the query statement in the cluster set based on the matching standard question, and store the query statement and the answer content in the knowledge base.
[0143] In this embodiment, query statements that have been classified, clustered, and matched constitute a set number of standard questions corresponding to each cluster set. Optionally, in order to establish a knowledge base that can promptly feedback answer content after receiving a question, after establishing the cluster set and its corresponding matching standard question, the answer content corresponding to the cluster set is obtained based on the matching standard question. After receiving the query statement, the corresponding set number of answer contents can be output according to the answer content corresponding to the corresponding cluster set. This embodiment increases the possibility of these output answer contents being clicked.
[0144] Optionally, step 708 includes:
[0145] Based on a set number of matching standard questions, determining a set number of answer contents corresponding to the standard questions;
[0146] The set number of answer contents are used as the answer contents corresponding to the cluster set.
[0147] In this embodiment, after determining the standard question corresponding to the query statement, the query is pushed to the knowledge base management background for management, and the answer content corresponding to the standard question is used as the answer content corresponding to the cluster set, wherein the standard question and its corresponding answer content can be saved in advance. Optionally, the category of the pushed query and the recommended top3 standard questions are manually judged. If the query category is correct, the standard question can be produced. If the category is incorrect, it flows back to the sample library, and the query is classified into the correct category after the model is optimized; if the top3 recommended standard question, if the recommendation is accurate, the correct standard question is selected. If it is incorrect, the correct standard question is found in the existing standard questions. Otherwise, a new standard question can be generated through manual summary. Regardless of whether the category and standard question are correct or not, the result is sent back to the sample library to provide training samples for iterative optimization of the model. For the standard question generated by the query, go to the knowledge attachment step. If the standard question can be attached to the existing knowledge, the corresponding knowledge is attached and pushed into the online knowledge base; if there is no knowledge attachment, a new knowledge is generated and pushed into the online knowledge base. The knowledge production process is completed at this point. This embodiment can not only accumulate correct samples for iterative updating of the model, but also realize an efficient and accurate knowledge production process.
[0148] Any query statement management method provided in the embodiments of the present disclosure may be executed by any appropriate device with data processing capabilities, including but not limited to: a terminal device and a server, etc. Alternatively, any query statement management method provided in the embodiments of the present disclosure may be executed by a processor, such as the processor executing any query statement management method mentioned in the embodiments of the present disclosure by calling corresponding instructions stored in a memory. This will not be described in detail below.
[0149] Exemplary Devices
[0150] Figure 8 FIG. 1 is a schematic diagram of the structure of a query statement management device provided by an exemplary embodiment of the present disclosure. Figure 8 As shown, the device provided in this embodiment includes:
[0151] The statement acquisition module 81 is used to acquire multiple query statements that meet preset conditions in the log.
[0152] The statement processing module 82 is used to classify the multiple query statements and perform aggregation processing based on the classification processing results to obtain at least one clustering set.
[0153] Each cluster set includes at least one query statement.
[0154] The standard matching module 83 is used to perform standard question matching on each query statement in the cluster set, and determine a set number of matching standard questions for each cluster set.
[0155] The above-mentioned embodiment of the present disclosure provides a query statement management device, which obtains multiple query statements that meet preset conditions in a log; classifies and aggregates the multiple query statements to obtain at least one cluster set, and the cluster set includes at least one query statement; performs standard question matching on each of the query statements in the cluster set, and determines a set number of matching standard questions for each cluster set; this embodiment determines a set number of matching standard questions that match the cluster set through classification and aggregation processing, so as to match corresponding standard questions to substantially the same questions raised in different ways, so as to improve the processing efficiency of different query statements.
[0156] In some optional embodiments, the preset conditions include but are not limited to at least one of the following: having an answer but no click, having no answer, etc.;
[0157] The statement acquisition module 81 is specifically used to filter the query statements obtained from the log based on preset conditions to obtain multiple query statements that meet the preset conditions.
[0158] In some optional embodiments, the statement processing module 82 includes:
[0159] A classification unit, used to classify the multiple query statements into at least one category; wherein each category includes at least two of the query statements;
[0160] The clustering unit is used to cluster at least two query statements included in each category to obtain at least one cluster set.
[0161] Optionally, the classification unit is specifically used to segment the query statements for each of the multiple query statements to obtain at least one word; process at least one word vector corresponding to the at least one word through a classification model to determine the category label corresponding to the query statement; wherein the classification model is trained with a training statement set with known category labels, and the statement training set corresponds to the business scenario of the query statement.
[0162] In some optional embodiments, the standard matching module 83 includes:
[0163] A scoring unit is used to score each query statement in the cluster set with each of the pre-stored standard questions by inputting the network model to obtain a score of the query statement matching each of the pre-stored standard questions;
[0164] A sentence matching unit, used for determining a set number of standard question sentences matching the query sentence based on the score;
[0165] The set matching unit is used to determine a set number of matching standard questions corresponding to the cluster set based on the standard questions matching each query statement included in the cluster set.
[0166] Optionally, the sentence matching unit is specifically configured to sort the multiple scores corresponding to the multiple standard questions according to the size of the scores; and according to the sorting, determine a set number of standard questions with the highest scores as the standard questions of the query sentence.
[0167] Optionally, the set matching unit is specifically used to obtain multiple standard questions corresponding to the clustering set based on the standard questions matching each query statement included in the clustering set; and determine a set number of standard questions as matching standard questions corresponding to the clustering set based on the score corresponding to each standard question in the multiple standard questions.
[0168] Optionally, the scoring unit is specifically used to segment the query statement and each standard question respectively, determine multiple word vectors corresponding to the query statement and multiple word vectors corresponding to the standard question; process the multiple word vectors corresponding to the query statement through a bidirectional long short-term memory network to obtain sentence features; process the multiple word vectors corresponding to the standard question through a bidirectional long short-term memory network to obtain standard question features; connect the sentence features and the standard question features to obtain connection features; determine a one-dimensional matrix based on the connection features, and determine the score of the query statement matching the standard question and the position of the standard question in the multiple standard questions through the one-dimensional matrix.
[0169] The one-dimensional matrix represents the matching relationship between the query sentence and multiple standard questions.
[0170] In some optional embodiments, the device provided in this embodiment further includes:
[0171] The answer matching module is used to determine the answer content corresponding to the query statement in the cluster set based on the matching standard question, and store the query statement and the answer content in the knowledge base.
[0172] Optionally, the answer matching module is specifically used to determine a set number of answer contents corresponding to the standard questions based on a set number of matching standard questions; and use the set number of answer contents as answer contents corresponding to the cluster set.
[0173] Exemplary Electronic Devices
[0174] Below, reference Fig. 9The electronic device according to the embodiment of the present disclosure is described. The electronic device may be any one or both of the first device 100 and the second device 200, or a stand-alone device independent of them, and the stand-alone device may communicate with the first device and the second device to receive the collected input signals from them.
[0175] Fig. 9 A block diagram of an electronic device according to an embodiment of the present disclosure is illustrated.
[0176] like Fig. 9 As shown, the electronic device 90 includes one or more processors 91 and a memory 92 .
[0177] The processor 91 may be a central processing unit (CPU) or other forms of processing units having data processing capabilities and / or instruction execution capabilities, and may control other components in the electronic device 90 to perform desired functions.
[0178] The memory 92 may include one or more computer program products, which may include various forms of computer-readable storage media, such as volatile memory and / or non-volatile memory. The volatile memory may include, for example, random access memory (RAM) and / or cache memory (cache), etc. The non-volatile memory may include, for example, read-only memory (ROM), hard disk, flash memory, etc. One or more computer program instructions may be stored on the computer-readable storage medium, and the processor 91 may run the program instructions to implement the query statement management method of each embodiment of the present disclosure described above and / or other desired functions. Various contents such as input signals, signal components, noise components, etc. may also be stored in the computer-readable storage medium.
[0179] In one example, the electronic device 90 may further include: an input device 93 and an output device 94 , and these components are interconnected via a bus system and / or other forms of connection mechanisms (not shown).
[0180] For example, when the electronic device is the first device 100 or the second device 200, the input device 93 may be the microphone or microphone array described above, for capturing input signals of a sound source. When the electronic device is a stand-alone device, the input device 93 may be a communication network connector, for receiving collected input signals from the first device 100 and the second device 200.
[0181] In addition, the input device 93 may also include, for example, a keyboard, a mouse, and the like.
[0182] The output device 94 can output various information to the outside, including determined distance information, direction information, etc. The output device 94 can include, for example, a display, a speaker, a printer, a communication network and a remote output device connected thereto, and the like.
[0183] Of course, to simplify, Fig. 9 Only some of the components related to the present disclosure in the electronic device 90 are shown, and components such as a bus, an input / output interface, etc. are omitted. In addition, according to specific application situations, the electronic device 90 may also include any other appropriate components.
[0184] Exemplary computer program products and computer-readable storage media
[0185] In addition to the above-mentioned methods and devices, an embodiment of the present disclosure may also be a computer program product, which includes computer program instructions, which, when executed by a processor, enable the processor to execute the steps of the query statement management method according to various embodiments of the present disclosure described in the above-mentioned "Exemplary Method" section of this specification.
[0186] The computer program product may be written in any combination of one or more programming languages to write program code for performing the operations of the disclosed embodiments, including object-oriented programming languages such as Java, C++, etc., and conventional procedural programming languages such as "C" or similar programming languages. The program code may be executed entirely on the user computing device, partially on the user device, as a separate software package, partially on the user computing device and partially on a remote computing device, or entirely on a remote computing device or server.
[0187] In addition, an embodiment of the present disclosure may also be a computer-readable storage medium on which computer program instructions are stored. When the computer program instructions are executed by a processor, the processor executes the steps of the query statement management method according to various embodiments of the present disclosure described in the above "Exemplary Method" section of this specification.
[0188] The computer readable storage medium can adopt any combination of one or more readable media. The readable medium can be a readable signal medium or a readable storage medium. The readable storage medium can include, for example, but is not limited to, a system, device or device of electricity, magnetism, light, electromagnetic, infrared, or semiconductor, or any combination of the above. More specific examples (non-exhaustive list) of readable storage media include: an electrical connection with one or more wires, a portable disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above.
[0189] The basic principles of the present disclosure are described above in conjunction with specific embodiments. However, it should be noted that the advantages, strengths, effects, etc. mentioned in the present disclosure are only examples and not limitations, and it cannot be considered that these advantages, strengths, effects, etc. are required by each embodiment of the present disclosure. In addition, the specific details disclosed above are only for the purpose of illustration and ease of understanding, and are not limitations. The above details do not limit the present disclosure to the necessity of adopting the above specific details to be implemented.
[0190] Each embodiment in this specification is described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the embodiments can be referred to each other. For the system embodiment, since it basically corresponds to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment.
[0191] The block diagrams of the devices, apparatuses, equipment, and systems involved in this disclosure are only illustrative examples and are not intended to require or imply that they must be connected, arranged, and configured in the manner shown in the block diagrams. As will be appreciated by those skilled in the art, these devices, apparatuses, equipment, and systems can be connected, arranged, and configured in any manner. Words such as "including," "comprising," "having," and the like are open words, referring to "including but not limited to," and can be used interchangeably therewith. The words "or" and "and" used herein refer to the words "and / or," and can be used interchangeably therewith, unless the context clearly indicates otherwise. The word "such as" used herein refers to the phrase "such as but not limited to," and can be used interchangeably therewith.
[0192] The method and apparatus of the present disclosure may be implemented in many ways. For example, the method and apparatus of the present disclosure may be implemented by software, hardware, firmware, or any combination of software, hardware, and firmware. The above order of steps for the method is for illustration only, and the steps of the method of the present disclosure are not limited to the order specifically described above, unless otherwise specifically stated. In addition, in some embodiments, the present disclosure may also be implemented as a program recorded in a recording medium, which includes machine-readable instructions for implementing the method according to the present disclosure. Therefore, the present disclosure also covers a recording medium storing a program for executing the method according to the present disclosure.
[0193] It should also be noted that in the apparatus, device and method of the present disclosure, each component or each step can be decomposed and / or recombined. Such decomposition and / or recombination should be regarded as equivalent solutions of the present disclosure.
[0194] The above description of the disclosed aspects is provided to enable any person skilled in the art to make or use the present disclosure. Various modifications to these aspects will be readily apparent to those skilled in the art, and the general principles defined herein may be applied to other aspects without departing from the scope of the present disclosure. Therefore, the present disclosure is not intended to be limited to the aspects shown herein, but rather to the widest scope consistent with the principles and novel features disclosed herein.
[0195] The above description has been given for the purpose of illustration and description. In addition, this description is not intended to limit the embodiments of the present disclosure to the forms disclosed herein. Although multiple example aspects and embodiments have been discussed above, those skilled in the art will recognize certain variations, modifications, changes, additions and sub-combinations thereof.
Claims
1. A query statement management method, characterized in that: include: Get multiple query statements in the log that meet the preset conditions; The preset conditions include: having answers but no clicks; Classify the multiple query statements, and perform aggregation processing based on the classification results to obtain at least one cluster set; wherein each cluster set includes at least one query statement; Performing standard question matching on each query statement in the cluster set respectively, and determining a set number of matching standard question sentences for each cluster set; The performing standard question matching on each query statement in the cluster set respectively, and determining a set number of matching standard questions for each cluster set, comprises: For each of the query statements in the cluster set, the query statement and each of the pre-stored standard questions are scored by inputting a network model to obtain a score of the query statement matching each of the multiple standard questions; the query statement is converted into a semantic vector by using the network model, and the classification task is converted into a vector matching task; Determining a set number of standard questions matching the query sentence based on the score; Based on the standard questions matching each of the query sentences included in the cluster set, a set number of matching standard questions corresponding to the cluster set is determined.
2. The method according to claim 1, characterized in that: The preset conditions also include: no answer; The obtaining of multiple query statements in the log that meet the preset conditions includes: The query statements obtained from the log are screened based on the preset conditions to obtain multiple query statements that meet the preset conditions.
3. The method according to claim 1, characterized in that The classifying process is performed on the multiple query statements, and the aggregation process is performed based on the classification process result to obtain at least one clustering set, including: Classifying the multiple query statements into at least one category; wherein each category includes at least two query statements; Clustering is performed on at least two query statements included in each category to obtain at least one cluster set.
4. The method according to claim 3, characterized in that The classifying the multiple query statements into at least one category includes: For each of the multiple query statements, segment the query statement to obtain at least one word; At least one word vector corresponding to the at least one term is processed through a classification model to determine a category label corresponding to the query statement; wherein the classification model is trained with a training statement set with known category labels, and the statement training set corresponds to the business scenario of the query statement.
5. The method according to any one of claims 1 to 4, characterized in that: The step of determining a set number of standard questions matching the query statement based on the score includes: sorting the scores corresponding to the plurality of standard questions according to the sizes of the scores; According to the ranking, a set number of the standard questions with the highest scores are determined as the standard questions of the query statement.
6. The method according to claim 5, characterized in that The determining, based on the standard questions matching each of the query statements included in the cluster set, a set number of matching standard questions corresponding to the cluster set includes: Based on the standard question sentence matching each of the query sentences included in the cluster set, obtaining a plurality of standard question sentences corresponding to the cluster set; Based on the score corresponding to each of the plurality of standard questions, a set number of the standard questions are determined as matching standard questions corresponding to the cluster set.
7. The method according to any one of claims 1 to 4, characterized in that: The step of scoring the query statement and each of the plurality of pre-stored standard questions by inputting the network model to obtain a score of the query statement matching each of the plurality of standard questions includes: Performing word segmentation on the query statement and each of the standard questions respectively, and determining a plurality of word vectors corresponding to the query statement and a plurality of word vectors corresponding to the standard questions; Processing multiple word vectors corresponding to the query sentence through a bidirectional long short-term memory network to obtain sentence features; processing multiple word vectors corresponding to the standard question sentence through a bidirectional long short-term memory network to obtain standard question features; Connecting the sentence feature and the standard question feature to obtain a connection feature; A one-dimensional matrix is determined based on the connection features, and a score of matching between the query statement and the standard question and a position of the standard question among the multiple standard questions are determined through the one-dimensional matrix; wherein the one-dimensional matrix represents a matching relationship between the query statement and the multiple standard questions.
8. The method according to any one of claims 1 to 4, characterized in that: Also includes: Based on the matching standard question, the answer content corresponding to the query statement is determined in the cluster set, and the query statement and the answer content are stored in a knowledge base.
9. The method according to claim 8, characterized in that The determining, based on the matching standard question, that the cluster set includes answer content corresponding to the query statement includes: Based on the set number of matching standard questions, determining a set number of answer contents corresponding to the standard questions; The set number of answer contents are used as answer contents corresponding to the cluster set.
10. A query statement management device, characterized in that: include: A statement acquisition module is used to acquire multiple query statements in the log that meet preset conditions; The preset conditions include: having answers but no clicks; A statement processing module, used for classifying the multiple query statements, and performing aggregation processing based on the classification processing results to obtain at least one cluster set; wherein each of the cluster sets includes at least one query statement; A standard matching module, used to perform standard question matching on each query statement in the cluster set, and determine a set number of matching standard questions for each cluster set; The standard matching module comprises: A scoring unit is used to score each query statement in the cluster set with each of the pre-stored standard questions by inputting a network model to obtain a score of the query statement matching each of the pre-stored standard questions; convert the query statement into a semantic vector by using the network model, and convert the classification task into a vector matching task; A sentence matching unit, configured to determine a set number of standard question sentences matching the query sentence based on the score; The set matching unit is used to determine a set number of matching standard questions corresponding to the cluster set based on the standard questions matching each of the query sentences included in the cluster set.
11. The device according to claim 10, characterized in that The preset conditions also include: no answer; The statement acquisition module is specifically used to filter the query statements obtained from the log based on the preset conditions to obtain multiple query statements that meet the preset conditions.
12. The device according to claim 10, characterized in that The statement processing module comprises: A classification unit, configured to classify the multiple query statements into at least one category; wherein each category includes at least two query statements; The clustering unit is used to cluster the at least two query statements included in each category to obtain at least one cluster set.
13. The device according to claim 12, characterized in that The classification unit is specifically used to segment the query statement for each of the multiple query statements to obtain at least one word; process at least one word vector corresponding to the at least one word through a classification model to determine the category label corresponding to the query statement; wherein the classification model is trained with a training statement set with known category labels, and the statement training set corresponds to the business scenario of the query statement.
14. The device according to any one of claims 10 to 13, characterized in that: The sentence matching unit is specifically configured to sort the multiple scores corresponding to the multiple standard questions according to the sizes of the scores; and according to the sorting, determine a set number of the standard questions with the highest scores as the standard questions of the query sentence.
15. The device according to claim 14, characterized in that The set matching unit is specifically configured to obtain a plurality of standard questions corresponding to the cluster set based on the standard questions matching each of the query sentences included in the cluster set; Based on the score corresponding to each of the plurality of standard questions, a set number of the standard questions are determined as matching standard questions corresponding to the cluster set.
16. The device according to any one of claims 10 to 13, characterized in that: The scoring unit is specifically used to perform word segmentation on the query sentence and each of the standard questions respectively, determine multiple word vectors corresponding to the query sentence and multiple word vectors corresponding to the standard question; process the multiple word vectors corresponding to the query sentence through a bidirectional long short-term memory network to obtain sentence features; process the multiple word vectors corresponding to the standard question through a bidirectional long short-term memory network to obtain standard question features; connect the sentence features and the standard question features to obtain connection features; A one-dimensional matrix is determined based on the connection features, and a score of matching between the query statement and the standard question and a position of the standard question among the multiple standard questions are determined through the one-dimensional matrix; wherein the one-dimensional matrix represents a matching relationship between the query statement and the multiple standard questions.
17. The device according to any one of claims 10 to 13, characterized in that: The device also includes: The answer matching module is used to determine the answer content corresponding to the query statement in the cluster set based on the matching standard question, and store the query statement and the answer content in a knowledge base.
18. The device according to claim 17, characterized in that The answer matching module is specifically used to determine a set number of answer contents corresponding to the standard questions based on the set number of matching standard questions; and use the set number of answer contents as the answer contents corresponding to the cluster set.
19. A computer-readable storage medium, characterized in that: The storage medium stores a computer program, and the computer program is used to execute the query statement management method described in any one of claims 1 to 9.
20. An electronic device, characterized in that: The electronic device comprises: processor; a memory for storing instructions executable by the processor; The processor is used to read the executable instructions from the memory and execute the instructions to implement the query statement management method described in any one of claims 1-9 above.
