User question rewriting method and system for multi-turn question answering of large-scale dialogue system

Abstract

The application discloses a user question rewriting method and system for multi-round question-answering of a large-scale dialogue system, and belongs to the technical field of information processing. The method comprises the following steps: obtaining user input; inputting the current question into a judgment model to obtain a question with a subject or a question without a subject; mounting the question with the subject on a child node of a multi-way tree, mounting the question without the subject on a grandchild node of the multi-way tree, and mounting historical questions on the child node or the grandchild node of the multi-way tree to obtain a mounted multi-way tree; wherein the mounted multi-way tree comprises a root node, a plurality of child nodes and a plurality of grandchild nodes; the plurality of child nodes belong to the root node, and the plurality of child nodes are different theme questions; the plurality of grandchild nodes belong to one or more child nodes, and the plurality of grandchild nodes are different themeless questions; the grandchild nodes in each child node are sorted to obtain relevant themeless question clusters to form a plurality of question lists with subjects and backgrounds.

Description

Technical Field

[0001] This invention relates to the field of information processing technology, and in particular to a method and system for rewriting user questions for multi-turn question answering in large-scale dialogue systems. Background Technology

[0002] In practical use, each user's question may be related to the previous or k previous questions. Therefore, in a certain question, the user's question may lack necessary background or have referential relationships between the subjects in the question. This will cause the retrieval system to fail to retrieve highly relevant knowledge blocks, and the model will not be able to obtain knowledge that is highly relevant to the question, resulting in poor answering ability of the model. Therefore, how to resolve referential issues and enhance background induction has become a problem.

[0003] Traditional referential resolution techniques tend to be rule-based or referential recognition and resolution-based models. They can achieve certain performance on some traditional non-dialogue datasets. However, in multi-turn dialogues, the repetition of referentials becomes more and more serious as the number of user dialogue turns increases. Therefore, this technique is difficult to use in modern LLM-based dialogue systems. Summary of the Invention

[0004] This invention provides a method and system for rewriting user questions in multi-turn question-answering systems for large-scale dialogue systems, in order to solve the problems of being unable to resolve and discover referential information throughout the entire dialogue cycle and being unable to alleviate the problem of weak background information in questioning.

[0005] The first aspect of this invention provides a method for rewriting user questions for multi-turn question answering in large-scale dialogue systems, comprising the following steps:

[0006] Obtain user input, which includes historical questions and the current question;

[0007] The current question is input into the judgment model to obtain a question with a subject or a question without a subject;

[0008] The questions with subjects are attached to the child nodes of the multi-branch tree, the questions without subjects are attached to the grandchild nodes of the multi-branch tree, and the historical questions are attached to the child or grandchild nodes of the multi-branch tree to obtain the attached multi-branch tree.

[0009] The mounted multi-branch tree includes a root node, multiple child nodes, and multiple grandchild nodes; the multiple child nodes belong to the root node and each child node is a different topic for a question; the multiple grandchild nodes belong to one or more child nodes and each grandchild node is a different topic for a question.

[0010] The grandchild nodes in each of the child nodes are sorted to obtain a related subject-free question cluster, to form a plurality of question lists with subjects and backgrounds.

[0011] In some embodiments of the first aspect, in the step of inputting the current question into a judgment model to obtain a subject-containing question or a subject-free question, the judgment model specifically comprises:

[0012] The judgment model comprises an HMM module and an RCM module;

[0013] The HMM module is used for performing part-of-speech detection on the current question to obtain an output part-of-speech annotation sequence;

[0014] The RCM module is used for performing value assignment calculation on the output part-of-speech annotation sequence to obtain a sequence score, a mean value and a standard deviation corresponding to the sentence;

[0015] Based on the sequence score, the mean value and the standard deviation, it is determined whether the user input is the subject-containing question or the subject-free question.

[0016] In some embodiments of the first aspect, the model expression of the HMM module is specifically:

[0017] λ=(A,B,Π)

[0018] A=[a ij ] N＊N ,a ij =P(i t+1 =q j |i t =q i )

[0019] B=[b j (k)] N＊M ,b j (k)=P(o t =v k |i t =q j )

[0020] Π=[π(i)] N ,π(i)=P(i1=q i )

[0021] In the above formula, λ is a three-tuple, Π is a hidden state probability distribution, for a hidden state set Q, Q={q1,q2,...,q N}, there is an observation set V={v1,v2,...,v M}, N is the number of possible states, M is the possible observation, I is a set of state sequences, I={i1,i2,...,i r}, O is a set of observation sequences, O = {o1, o2,..., o r}, A is a state transition matrix constituting a Markov chain, a ij identifies the probability of state q i transitioning to state q j at time t+1, B is a probability matrix of generating an observation state, b j (k) is the generation probability of observation state v k at time t under hidden state q j , and π(i) is a set of probabilities of sequence I being in a certain state at a certain time t.

[0022] In some embodiments of the first aspect, the RCM module comprises a score function, a mean function and a standard deviation function;

[0023] The expression of the score function is specifically:

[0024]

[0025] In the above formula, RCM(T) is the sequence score corresponding to the sentence, Word(t i ) is the output part-of-speech annotation value, t i is the part-of-speech annotation of the i-th word of the sentence t, long is the length of the sequence, r is a pronoun, and T is the output part-of-speech annotation sequence;

[0026]

[0027] In the above formula, n is a common noun, nz is another proper name, PER is a person name, LOC is a place name, ORG is an organization name, u is an auxiliary word, q is a quantifier, ad is an adverbial word, v is a common verb, f is a direction noun, vd is a verb-adverb, an is a noun-adjective, xc is another virtual word, w is a punctuation mark, p is a preposition, d is an adverb, vn is a noun-verb, nw is a work name, TIME is a time;

[0028] The expression of the mean function is specifically:

[0029]

[0030] In the above formula, μ is the mean value, Num is the i-th item in the question sequence, and RCM(T i ) is the score under the i-th question;

[0031] The expression of the standard deviation function is specifically:

[0032]

[0033] In the above formula, μ is the mean value, σ is the standard value, Num is the i-th item in the question sequence, and RCM(Ti ) is the score under the ith question.

[0034] In some embodiments of the first aspect, in determining whether the user input is the subject-bearing question or the subjectless question based on the sequence score, the mean and the standard deviation, the step specifically comprises:

[0035] when the sequence score is less than the difference between the mean and the standard deviation, determining that the current question is the subjectless question;

[0036] when the sequence score is greater than the difference between the mean and the standard deviation, determining that the current question is the subject-bearing question.

[0037] In some embodiments of the first aspect, the subjectless question is mounted at the grandchild node of the multi-way tree, and the step specifically comprises:

[0038] calculating the similarity of the subjectless question and different subject questions based on a ranking function;

[0039] mounting the subjectless question at the grandchild node under the child node where the subject question with the highest similarity is located.

[0040] In some embodiments of the first aspect, the expression of the ranking function is specifically:

[0041] DGT(L, u) = BM25(L, u) * ln(d i )

[0042]

[0043] In the above formula, DGT(L, u) is, L is a set of subject questions, u is a subjectless question, d i is the relative distance of each subject question l i in the set of subject questions L, F num is the total number of documents, Seq Len is the number of terms in the query, f(q i , D) is the number of occurrences of q i in the document D, q i is the ith term in the query, and len(D) is the length of the document D, avg len is the average length of all documents, k is the first hyperparameter, and b is the second hyperparameter.

[0044] In some embodiments of the first aspect, in sorting the grandchild nodes in each of the child nodes to obtain a relevant subjectless question cluster to form a plurality of subject and background question lists, the step specifically comprises:

[0045] determining the minimum information entropy change of each grandchild node under a child node by using an information entropy function;

[0046] determining the order of the grandchild nodes based on the minimum information entropy change, obtaining a relevant no-topic question cluster, and forming a plurality of question lists with subjects and backgrounds.

[0047] In some embodiments of the first aspect, the expression of the information entropy function is specifically:

[0048] min(△H(u))

[0049]

[0050] In the above formula, H(u) is the information entropy in a cluster, p(x i ) is the probability that u is xi, and x i is the i-th input

[0051] The second aspect of the application provides a user question rewriting system for multi-round question answering of a large-scale dialogue system, which comprises:

[0052] An acquisition module is configured to acquire user inputs, wherein the user inputs comprise historical questions and a current question;

[0053] A judgment module is configured to input the current question into a judgment model to obtain a question with a subject or a question without a subject;

[0054] A mounting module is configured to mount the question with a subject on a child node of a multi-way tree, mount the question without a subject on a grandchild node of the multi-way tree, and mount the historical questions on the child node or the grandchild node of the multi-way tree to obtain a mounted multi-way tree;

[0055] A multi-way tree module comprises a root node, a plurality of child nodes and a plurality of grandchild nodes; the plurality of child nodes belong to the root node, and the plurality of child nodes are different topic questions; the plurality of grandchild nodes belong to one or more child nodes, and the plurality of grandchild nodes are different no-topic questions;

[0056] An ordering module is configured to order the grandchild nodes in each child node to obtain a relevant no-topic question cluster and form a plurality of question lists with subjects and backgrounds.

[0057] As can be seen from the above technical solutions, the application has at least the following advantages:

[0058] The embodiment of the present application provides a user question rewriting method and system for multi-round question-answering of a large-scale dialogue system, wherein the multi-branch tree is formed by mounting the question with a subject and the question without a subject on the multi-branch tree, and the mounted multi-branch tree comprises a root node, a plurality of child nodes and a plurality of grandchild nodes; the plurality of child nodes belong to the root node, and the plurality of child nodes are different theme questions; the plurality of grandchild nodes belong to one or more child nodes, and the plurality of grandchild nodes are different themeless questions, so that the multi-branch tree maintains a global session record in the DGT session, and the reference resolution and discovery in the whole dialogue period can be realized compared with the traditional context technology.

[0059] In addition, the historical question is mounted on the child node or the grandchild node of the multi-branch tree, and the grandchild nodes in each child node are sorted to obtain a relevant themeless question cluster, thereby forming a plurality of questions with subjects and backgrounds, realizing the global session regulation of continuous questions or multi-hop questions, realizing the theme management in the whole session life cycle, and relieving the weak background problem of the question.

[0060] In summary, the present scheme solves the problems that the reference resolution and discovery cannot be realized in the whole dialogue period and the weak background problem of the question cannot be relieved. BRIEF DESCRIPTION OF DRAWINGS

[0061] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the following will briefly introduce the drawings needed to be used in the embodiments or the prior art description. Obviously, the drawings in the following description only some embodiments of the present application, and for those skilled in the art, other drawings can be obtained without creative labor under the premise of these drawings.

[0062] Figure 1 The whole step flowchart of the user question rewriting method for multi-round question-answering of a large-scale dialogue system provided by the embodiment of the present application;

[0063] Figure 2 The whole flowchart of the user question rewriting method for multi-round question-answering of a large-scale dialogue system provided by the embodiment of the present application;

[0064] Figure 3 The HMM model for multi-round question-answering of a large-scale dialogue system provided by the embodiment of the present application detects the part of speech of the user input;

[0065] Figure 4 The mounted multi-branch tree for multi-round question-answering of a large-scale dialogue system provided by the embodiment of the present application; DETAILED DESCRIPTION

[0066] The embodiment of the present application provides a user question rewriting method and system for multi-round question answering of a large-scale dialogue system, which can effectively cope with open-world question answering compared with the prior art.

[0067] On the one hand, the DGT algorithm can break through the context attention boundary, expand the potential analysis of anaphora resolution to the entire session, and avoid the problem that two questions with large spans cannot be retrieved after too many rounds of question answering by the user; on the other hand, the DGT and FQC algorithms can organize the user's session theme while chaining the user's questions, avoid the problem of error inheritance caused by inaccurate answers multiple times without context, and simplify multi-round question answering into single-question retrieval and discovery, thereby solving the problems of anaphora resolution and discovery and weak background questions in the multi-round question answering.

[0068] In order to make the invention purpose, features and advantages of the present application more obvious and easy to understand, the technical solutions in the embodiments of the present application will be described clearly and completely below in conjunction with the drawings in the embodiments of the present application. Obviously, the following described embodiments are only a part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor are within the scope of protection of the present application.

[0069] Please refer to Figure 1 and Figure 2 The present application provides a user question rewriting method for multi-round question answering of a large-scale dialogue system, comprising the following steps:

[0070] S1, obtaining user input, the user input comprising historical questions and current questions.

[0071] S2, inputting the current question into a judgment model to obtain a question with a subject or a question without a subject, that is, the question of the current user is first detected for anaphora, the user's question is divided into subject questions and subordinate questions, and the internal cache record is obtained.

[0072] Preferably, the judgment model comprises an HMM module and an RCM module, and the specific step S2 is:

[0073] S21, performing part-of-speech detection on the current question by using the trained HMM module to obtain an output part-of-speech annotation sequence.

[0074] In the embodiment, when the user statement lacks a subject and there is a pronoun, the sequence output has a word with a part of speech of r, the statement is marked as a theme sub-question query (Qc If a user asks a question with a subject but no word of the second 'r', mark the statement as a new topic question (query(Q)). t ).

[0075] In this embodiment, the model expression of the HMM module is specifically as follows:

[0076] λ = (A, B, Π)

[0077] A = [a ij ]N ＊N a ij =P(i t+1 =q j |i t =q i )

[0078] B = [b] j (k)] N＊M b j (k)=P(o t =v k |i t =q j )

[0079] Π=[π(i)] N , π(i)=P(i1=q i )

[0080] In the above formula, λ is a triple, Π is the probability distribution of the hidden state, and for the set of hidden states Q, we have Q = {q1, q2, ..., q}. N}, where the observation set V = {v1, v2, ..., v} M Let N be the number of possible states, M be the number of possible observations, and I be a sequence of states I = {i1, i2, ..., i...} r}, where O is a set of observation sequences O={o1, o2, ..., o r Let A be the state transition matrix that makes up the Markov chain, and a ij Identify state q at time t i Transition to state q at time t+1 j The probability of B is the probability matrix generated by the observed state. j (k) represents the observed state v at time t. k In the hidden state q j The generation probability is given by π(i), which is the set of probabilities that sequence I is in a certain state at a certain time t.

[0081] Specifically, such as Figure 3 As shown, the creation of an HMM module includes the following steps:

[0082] First, assume Q is the set of all possible hidden states (i.e., the set of part-of-speech tags, with M possibilities, which is the number of possible hidden states), and V is the set of all possible observation states (i.e., the set of user input, with N data points, which is the number of possible observation states), and Q = {q1, q2, ..., q...} M}, V = {v1, v2, ..., v N For a sequence of length T, where I is the state sequence and O is the observation sequence, then I = {i1, i2, ..., i}. T}, O = {o1, o2, ..., o T}, any hidden state i t ∈Q, any observation state o t ∈V.

[0083] Next, we calculate the hidden state of sequence index t as i. t =q t The hidden state at time t+1 is i t+1 =q j Define the state transition probability a of the HMM from t to t+1. ij For: a ij =P(i t+1 =q j |i t =q i ), by a ij The state transition matrix A of the Markov chain is A = [a ij ] N＊N

[0084] For the hidden state i at time t t =q j The corresponding observation state o t =v k Then the observed state v at that moment k In the hidden state q j The probability of generation is b j (k), satisfying b j (k)=P(o t =v k |i t =q j For b j (k) can form the probability matrix B = [b] for the generation of the observed state. j (k)] N＊M

[0085] The hidden state probability distribution Π = [π(i)] when t = 1 N , where π(i) = P(i1 = q i )

[0086] Thus the HMM module can be determined by the hidden state distribution Π, the state transition probability matrix A, and the observation state probability matrix B, and the HMM is described by the triplet λ = (A, B, Π).

[0087] S22, the RCM module is used to perform assignment calculation on the output part-of-speech tagging sequence to obtain the sequence score, mean value and standard deviation corresponding to the sentence;

[0088] The RCM module includes a score function, a mean value function and a standard deviation function;

[0089] In this embodiment, the expression of the score function is specifically as follows:

[0090]

[0091] In the above formula, RCM(T) is the sequence score corresponding to the sentence, Word(t i ) is the output part-of-speech tagging assignment, t i is the part-of-speech tagging of the i-th word for the sentence t, long is the length of the sequence, r is a pronoun, and T is the output part-of-speech tagging sequence;

[0092]

[0093] In the above formula, n is a common noun, nz is another proper name, PER is a personal name, LOC is a place name, ORG is an organization name, u is an auxiliary word, q is a quantifier, ad is an adverbial word, v is a common verb, f is a direction noun, vd is a dynamic adverb, an is a noun-adjective word, xc is another virtual word, w is a punctuation mark, p is a preposition, d is an adverb, vn is a noun-verb, nw is a work name, TIME is a time, and LAC is a label meaning in a table;

[0094] In this embodiment, the expression of the mean value function is specifically as follows:

[0095]

[0096] In the above formula, μ is the mean value, Num is the i-th item in the question sequence, and RCM(T i ) is the score under the i-th question;

[0097] In this embodiment, the expression of the standard deviation function is specifically as follows:

[0098]

[0099] In the above formula, μ is the mean value, σ is the standard value, Num is the i-th item in the question sequence, and RCM(T i ) is the score under the i-th question.

[0100] S23, determining whether the user input is a subject-bearing question or a subjectless question based on the sequence score, the mean and the standard deviation, i.e. obtaining the score of the sequence corresponding to the sentence by RCM(T), obtaining μ and σ by the above calculation, and based on the principle of normal distribution, maintaining the d-conversation subject tree as shown in Figure 4

[0101] Preferably, step S23 is specifically:

[0102] When the sequence score is less than the difference between the mean and the standard deviation, it is determined that the current question is a subjectless question, i.e. when RCM(T) < μ-σ, the input will be determined as a subjectless question, and thus will be mounted on the grandchild node (the leaf node of the tree).

[0103] When the sequence score is greater than the difference between the mean and the standard deviation, it is determined that the current question is a subject-bearing question, i.e. when RCM(T) ≥ μ-σ, the input is determined as a subject-bearing question, and thus will be mounted on the child node (the second layer node of the tree).

[0104] S3, mounting the subject-bearing question, the subjectless question and the historical question in the multi-way tree.

[0105] In the embodiment, step S3 is specifically:

[0106] S31, mounting the subject-bearing question in the child node of the multi-way tree;

[0107] S32, mounting the subjectless question in the grandchild node of the multi-way tree;

[0108] Preferably, step S32 is specifically:

[0109] S321, calculating the similarity between the subjectless question and the different subject questions based on the ranking function;

[0110] The expression of the ranking function is specifically:

[0111] DGT(L, u) = BM25(L, u) * ln(d i )

[0112]

[0113] In the above formula, DGT(L, u) is, L is the subject question set, u is the subjectless question, d i is the relative distance from each subject question l i in the subject question set L, F num is the total number of documents, Seq Len is the number of terms in the query, and f(q i , D) is q i ​The number of times TF,q appears in document D i It is the i-th term in the query, len(D) is the length of document D, and avg len Let k be the average length of all documents, k be the first hyperparameter, and b be the second hyperparameter.

[0114] S322, attaches the question without a subject to the grandchild node of the child node of the question with the highest similarity.

[0115] S33, attach the historical questions to the child or grandchild nodes of the multi-way tree to obtain the attached multi-way tree. For the user's historical question list Q = {q1,q2,…,q...} M These will be categorized into topic questions combined with topic questions under L. i When asking consecutive questions, it is necessary to link the questions to ensure that the current question has sufficient background knowledge and retrieval recall conditions.

[0116] Among them, such as Figure 4 As shown, the mounted multi-branch tree includes a root node, multiple child nodes, and multiple grandchild nodes; the multiple child nodes belong to the root node, and the multiple child nodes each ask a different question on a different topic; the multiple grandchild nodes belong to one or more child nodes, and the multiple grandchild nodes each ask a different question without a topic.

[0117] In summary, steps S2 and S3 maintain a global session record tree through the DGT session subject tree. Compared with traditional context techniques, this enables the resolution and discovery of references throughout the entire dialogue cycle. Furthermore, by using the DGT algorithm, we can break through the attention boundaries of the context and extend the potential resolution of references to the entire session. This avoids the inability to retrieve two questions with a large span after the user has conducted too many rounds of question and answer.

[0118] S4, sort the grandchild nodes within each child node to obtain related unrelated question clusters, forming multiple question lists with subjects and backgrounds. Therefore, when a question is asked... i When the number of unrelated questions (u) is greater than 1, they need to be sorted. Generally, user questions have a high degree of contextual relationship, so we will actually sort them. i The untitled question u is used to determine the minimum range through information entropy; for u, there is information entropy. For a continuous question cluster u, there needs to be a minimum information entropy change min(ΔH(u)). By finding the minimum information entropy change, we can obtain the relevant topicless question cluster u.

[0119] Preferably, step S4 specifically includes:

[0120] S41, determine the minimum information entropy change of each grandchild node under the child node by using the information entropy function;

[0121] The expression of the information entropy function is specifically:

[0122] min (AH (u))

[0123]

[0124] In the above formula, H (u) is the information entropy in a cluster, p (x i ) is the probability that u is xi, and x i is the ith input.

[0125] S42, based on the minimum information entropy change, determine the ranking of the grandchild node, obtain the related subject-free question cluster, and form a plurality of question lists with subjects and backgrounds.

[0126] Compared with the method of using traditional bag-of-words or large model summary generation, only the DGT+FQC algorithm is used to realize the global session regulation of continuous questioning or multi-hop questioning, realize the theme management of the whole session session life cycle, alleviate the problem of weak background of questioning, through the DGT and FQC algorithm, the user's questioning can be chained and organized while the user's session theme is organized, and the problem of error inheritance caused by inaccurate multiple answers without context is avoided, and the multi-round question and answer is simplified and unified as single problem retrieval and discovery.

[0127] S5, based on all related question lists with subjects and backgrounds, using a retrieval system to obtain an LLM summary reply to the user's question.

[0128] Based on the above embodiment, the application also provides a user question rewriting system for multi-round question and answer of a large-scale dialogue system, comprising:

[0129] An acquisition module is configured to acquire user input, wherein the user input includes historical questions and current questions;

[0130] A judgment module is configured to input the current question into a judgment model to obtain a question with a subject or a question without a subject;

[0131] A mounting module is configured to mount the question with the subject to a child node of a multi-way tree, mount the question without the subject to a grandchild node of the multi-way tree, and mount the historical question to the child node or the grandchild node of the multi-way tree to obtain a mounted multi-way tree;

[0132] The multi-way tree module comprises a root node, a plurality of child nodes, and a plurality of grandchild nodes; the plurality of child nodes belong to the root node, and the plurality of child nodes are different subject questions; the plurality of grandchild nodes belong to one or more child nodes, and the plurality of grandchild nodes are different subject-free questions.

[0133] The sorting module is configured to sort the grandchildren nodes in each sub-node to obtain a related no-topic question cluster, so as to form a plurality of question lists with subjects and backgrounds.

[0134] The above examples are only used to illustrate the technical solutions of the present application, but not to limit it; although the present application has been described in detail with reference to the foregoing examples, those skilled in the art should understand that the technical solutions recorded in the foregoing examples can be modified, or some technical features can be replaced by equivalents; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present application.

[0135] The terms "first", "second", "third", "fourth" and the like used in the description of the application and the above drawings, if any, are used to distinguish similar objects, and do not necessarily indicate a specific order or sequence. It should be understood that the data thus used can be interchanged under appropriate circumstances, so that the embodiments of the application described herein can be implemented in an order other than that illustrated or described herein. In addition, the terms "comprise" and "have" and any variations thereof are intended to cover non-exclusive inclusion, for example, a process, method, system, product or device comprising a series of steps or units does not necessarily have to be limited to those steps or units clearly listed, but can include other steps or units not clearly listed or inherent to these processes, methods, products or devices.

[0136] Finally, it should also be noted that in this paper, relationship terms such as first and second are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or sequence between the entities or operations.

Claims

1. A method for user question rewriting for multi-turn question answering of a large-scale dialogue system, characterized in that, The method comprises the following steps: obtaining user input, the user input comprising historical questions and a current question; inputting the current question into a judgment model to obtain a subject-containing question or a subject-free question; wherein the judgment model comprises an HMM module and an RCM module; performing part-of-speech detection on the current question by using the trained HMM module to obtain an output part-of-speech annotation sequence; performing assignment calculation on the output part-of-speech annotation sequence by using the RCM module to obtain a sequence score, a mean value and a standard deviation corresponding to the sentence; determining, based on the sequence score, the mean value and the standard deviation, whether the user input is the subject-containing question or the subject-free question; mounting the subject-containing question on a child node of a multi-way tree, mounting the subject-free question on a grandchild node of the multi-way tree, and mounting the historical questions on the child node or the grandchild node of the multi-way tree to obtain a mounted multi-way tree; wherein the mounted multi-way tree comprises a root node, a plurality of child nodes and a plurality of grandchild nodes; the plurality of child nodes belong to the root node, and the plurality of child nodes are different theme questions; the plurality of grandchild nodes belong to one or more of the child nodes, and the plurality of grandchild nodes are different subject-free questions; sorting the grandchild nodes in each child node to obtain a relevant subject-free question cluster to form a plurality of subject-and-context-containing question lists; wherein the plurality of subject-and-context-containing question lists are formed in the following manner: determining the minimum information entropy change of each grandchild node under a child node by using an information entropy function; based on the minimum information entropy change, determining the sorting of the grandchild nodes to obtain a relevant subject-free question cluster to form a plurality of subject-and-context-containing question lists. 2.The user question rewriting method of claim 1, wherein, The model expression of the HMM module is specifically: λ=(A,B,Π) A = [a ij ] N*N , a ij = P(i t+1 = q j | i t = q i ) B = [b j (k)] N*M , b j (k) = P(o t = v k | i t = q j ) Pi = [pi(i)] N pi(i) = P(i1 = q i ) In the above formula, λ is a triple, Π is the probability distribution of the hidden state, and Q is the set of hidden states, Q={q1,q2,…,q N Let V be the set of observations, V = {v1, v2, ..., v} M Let N be the number of possible states, M be the number of possible observations, and I be a sequence of states, I = {i1, i2, ..., i...} r }, where O is a set of observation sequences, O={o1,o2,…,o r Let A be the state transition matrix that makes up the Markov chain, and a ij Identify state q at time t i Transition to state q at time t+1 j The probability of B is the probability matrix generated by the observed state. j (k) represents the observed state v at time t. k In the hidden state q j The generation probability is given by π(i), which is the set of probabilities that sequence I is in a certain state at a certain time t. 3.The user question rewriting method of claim 1, wherein, The RCM module comprises a score function, a mean value function and a standard deviation function; The expression of the score function is specifically: In the above equation, RCM(T) is the sequence score corresponding to the sentence, Word(t i ) is the output part-of-speech assignment, t i is the part-of-speech assignment for the i-th word of the sentence t, long is the length of the sequence, r is a pronoun, T is the output part-of-speech sequence, n is a common noun, nz is another proper name, PER is a person name, LOC is a place name, ORG is an organization name, u is a auxiliary word, q is a quantifier, ad is an adverbial, v is a common verb, f is a direction noun, vd is a verb-adverbial, an is a noun-adjective, xc is another function word, w is a punctuation, p is a preposition, d is an adverb, vn is a noun-verb, nw is a work name, TIME is a time; The expression of the mean value function is specifically: In the above formula, μ is the mean, Num is the nth term in the question sequence, and RCM(T) i () represents the score for the i-th question; The expression of the standard deviation function is specifically: In the above equation, μ is the mean, σ is the standard value, Num is the number of the item in the question sequence, RCM(Ti) is the score under the ith question. i ) is the score under the ith question. 4.The user question rewriting method of claim 1, wherein, In the step of determining, based on the sequence score, the mean value and the standard deviation, whether the user input is the subject-containing question or the subject-free question, specifically: when the sequence score is less than the difference between the mean value and the standard deviation, determining that the current question is the subject-free question; when the sequence score is greater than the difference between the mean value and the standard deviation, determining that the current question is the subject-containing question. 5.The user question rewriting method of claim 1, wherein, In the step of mounting the subject-free question on the grandchild node of the multi-way tree, specifically: based on a sorting function, performing similarity calculation on the subject-free question and different theme questions; mounting the subject-free question on the grandchild node under the child node of the theme question with the highest similarity. 6.The user question rewriting method of claim 5, wherein, The expression of the sorting function is specifically: DGT(L, u) = BM25(L, u) * ln(d i ) In the above formula, DGT(L, u) is the relevance distance of the non-topic question u and the topic question set L, L is the topic question set, u is the non-topic question, d i is the relative distance of each topic question l i in the topic question set L to the non-topic question u, F num is the total number of documents, Seq Len is the number of terms in the query, f(q i , D) is the number of times q i appears in the document D, q i is the i-th term in the query, len(D) is the length of the document D, avg len is the average length of all documents, k is the first hyperparameter, and b is the second hyperparameter. 7.The user question rewriting method of claim 1, wherein, The expression of the information entropy function is specifically: min(ΔH(u)) In the above equation, H(u) is the information entropy in a cluster, p(x i ) is the probability of u being x i , and x i is the i-th input.

8. A user question reformulation system for multi-turn question answering in large-scale dialogue systems, characterized in that, The application of the user question rewriting method according to any one of claims 1 to 7 comprises: an obtaining module configured to obtain user input, the user input comprising historical questions and a current question; A judging module is configured to input the current question into a judging model to obtain a question with a subject or a question without a subject; A mounting module is configured to mount the question with the subject to a child node of a multi-way tree, mount the question without the subject to a grandchild node of the multi-way tree, and mount the historical question to the child node or the grandchild node of the multi-way tree to obtain a mounted multi-way tree; The multi-way tree module includes a root node, a plurality of child nodes, and a plurality of grandchild nodes; the plurality of child nodes belong to the root node, and the plurality of child nodes are different subject questions respectively; the plurality of grandchild nodes belong to one or more child nodes, and the plurality of grandchild nodes are different subjectless questions respectively; An ordering module is configured to order the grandchild nodes in each child node to obtain a relevant subjectless question cluster, thereby forming a plurality of question lists with subjects and backgrounds.

Images