A question chain generation method, device and equipment based on question graph data

Abstract

The application provides a question chain generation method, device and equipment based on question graph data, which comprises the following steps: S1, in a question set, slot label prediction is performed on a question, a connection relationship between questions is created by using a label vector distance, and question graph data is constructed with a single question as a node and a connection as an edge; S2, the question graph data is sent to a graph Transformer for graph representation learning, the nodes of the question graph data are embedded and represented, and a representation vector of the question graph data is obtained; S3, a question to be predicted and the representation vector of the question graph data are linked through a question chain generation model to realize link prediction, a question with the highest confidence is obtained, and a link is constructed according to the question with the highest confidence and the input question to be predicted, so that a one-hop question chain between the two questions is constructed. The application constructs a question graph by using the slot correlation between questions, surpasses the correlation of the question chain which is simply dependent on the semantic correlation of the question, forms a question chain with stronger logic, and realizes efficient generation of the question chain.

Description

Technical Field

[0001] This invention relates to the field of relation prediction technology, and in particular to a method, apparatus and device for generating question chains based on question graph data. Background Technology

[0002] Question chain generation, as a subtask of question generation, belongs to the sequential question generation (SQG) task, which involves generating a series of interconnected questions. A question chain requires semantic consistency and content relevance between questions. This is typically achieved using relation prediction models from the field of natural language processing to determine whether a link exists between two questions, identifying linked questions as a single-hop question chain. The main shortcomings of most current question chain generation models are:

[0003] (1) Question chain generation based on question set does not rely on a large amount of relational corpus information, but the correlation between questions in the generated question chain is weak;

[0004] (2) Question chain generation based on graph data enhances the semantic connection between questions, but graph relation prediction relies on a large amount of existing entity relation node information, and the model is difficult to achieve good performance in domains lacking large amounts of relation data. Summary of the Invention

[0005] In view of this, the purpose of the present invention is to provide a method, apparatus and device for generating question chains based on question graph data, so as to improve the above-mentioned problems.

[0006] This invention provides a method for generating question chains based on question graph data, comprising:

[0007] S1. In the question set, slot label prediction is performed for the questions, and the connection relationship between questions is created using the label vector distance. Question graph data is constructed with individual questions as nodes and connections as edges.

[0008] S2, the question graph data is fed into the graph Transformer for graph representation learning, the nodes of the question graph data are embedded to obtain the representation vector of the question graph data;

[0009] S3, the representation vectors of the question to be predicted and the question graph data are used to perform link prediction through the question chain generation model to obtain the question with the highest confidence. Then, a link is constructed between the highest confidence question and the input question to be predicted, thereby constructing a one-hop question chain between the two questions.

[0010] Preferably, step S1 specifically includes:

[0011] For the question set Q = {q1, ..., q...} NThe question q is obtained using a slot prediction model. i The slot label sequence; where q i Let X be the i-th question in the question set Q, and N be the size of the question set. Let X = {x1, ..., x2}. cn} represents the slot type, x j Let be the i-th slot, and cn be the total number of slots. Then the set of label types for the slot label sequence of question q is c = {[PAD], [UNK], [O], [B_x]}. i ],[I_x i ]}; where, [B_x i [Starting with x] i The starting identifier of the slot, [I_x i As x i The remaining identifiers for the slots are: [PAD] for the padding token tag in the question, [UNK] for the unrecognized sequence tag, and [O] for the token tag without a slot.

[0012] Rhetorical question q i ={w i1 , ..., w iqn}, where iqn is the question q i The total number of characters, where any single character is w; if the question is q i The a-th character w ia Marked [B_x] i The characters from the (a+1)th to the (a+wcn)th are marked [I_x] i ], then x i The slot value is c′ in ={w a w a+1 , ..., w a+wcn}, its length is a+wcn+1;

[0013] Regarding the question q i ={wi1, ..., w iqn} and q j ={w j1 , ..., w jqm}, where any one of its slot values ​​is c′ i ={c′ i1 , ..., c′ in} and slot value c′ j ={c′ j1 , ..., c′ jm The word embedding model GloVe is used to embed the slot values ​​into vectors, and their vector representations are obtained as follows:

[0014] w′ ia =GloVe(c′)i )

[0015] w′ jb =GloVe(c′) j )

[0016] For vector w′ ia sum vector w′ jb Euclidean distance dE(w′) ia , w′ jb Perform the calculation:

[0017]

[0018] Iterating through the question q i and question q j The Euclidean distance between all paired slot values, if w′ ia With w′ jb If the Euclidean distance of the node q is less than the connection threshold θ, then the question node q is considered to be... i And question node q j The adjacent node is the question node q. i And question node q j If the relation "connection" exists, then the question node q is not considered. i And question node q j There is no relationship "connection";

[0019] After traversing the question set Q, the corresponding question graph data G = {C′, Q} is constructed, where the relation edges of the question graph are "connections", and the set of nodes is the question set Q.

[0020] Preferably, S2 specifically includes:

[0021] For the question graph data G, the graph transformer outputs the embedding representation vector for each question node. The graph transformer contains an L-layer network structure, and the output sequence becomes the input sequence for the next time step. The specific formula for representing the graph attention module is as follows:

[0022]

[0023] Wherein, the operator ∨ ​​concatenates N attention contents, a(q i q j ) represents the attention weights of two different questions within the graph attention mechanism, a n (q i q j A represents the attention weight value in the nth attention layer. L A K The feature matrix represents the attention weight values. Let A be the feature matrix of the nth attention part; for each attention weight a, the feature matrices A of node i and node j are obtained by operation. K A L That is, the degree of association between two nodes, i and j;

[0024] Under each graph attention module, the operations of the feedforward neural network layer and the regularization layer are as follows:

[0025]

[0026]

[0027] FFN(x) = W1 × (f(W2 × x + b1) + b2)

[0028] Where FFN is a feedforward neural network, q' i represents the output of FFN, LayerNorm represents the normalization layer, W1 and W2 are the feature matrices in the feedforward neural network, and b1 and b2 are the bias values.

[0029] After iterative optimization, the output question node embedding representation vector is used. This is the representation vector for the question graph data.

[0030] Preferably, in step S3, the problem chain generation model adopts the ConvKB model, and its prediction method is as follows:

[0031]

[0032] in, For a given question chain S in the question graph data and the question q to be predicted j A function that determines whether a chain relationship exists outputs a relationship prediction score. Represents the chain of questions S and the question to be predicted q. j The chain of prediction problems, h” S Let g be the vector representation of the problem chain S; r For the question chain S and the question to be predicted q j Relational vectors, || is the concatenation symbol for vectors, ω m Let Ω be the m-th convolutional filter, Ω be the hyperparameter representing the number of filters in the convKB model, and * be the convolution operator; W∈R Ωk×1 The linear transformation matrix for the final score of the convKB model; level n The last question q in the question chain S n The graph complexity coefficient in question graph data.

[0033] Preferably, the loss function L used for training the convKB model is expressed as:

[0034]

[0035] Among them, l Sj λ is the dimension of the problem chain vector, and λ is a regularization setting constant parameter.

[0036] This invention also provides a question chain generation device based on question graph data, which includes:

[0037] The question graph construction unit is used to predict slot labels for questions in a question set, create connections between questions using label vector distance, and construct question graph data with individual questions as nodes and connections as edges.

[0038] The embedding representation unit is used to feed the question graph data into the graph Transformer for graph representation learning, embedding the nodes of the question graph data to obtain the representation vector of the question graph data;

[0039] The question chain construction unit is used to perform link prediction by passing the representation vector of the question to be predicted and the question graph data through the question chain generation model, to obtain the question with the highest confidence, and to construct a link between the input question to be predicted based on the question with the highest confidence and thus construct a one-hop question chain between the two questions.

[0040] Preferably, the question graph construction unit is specifically used for:

[0041] For the question set Q = {q1, ..., q...} N The question q is obtained using a slot prediction model. i The slot label sequence; where q i Let X be the i-th question in the question set Q, and N be the size of the question set. Let X = {x1, ..., x2}. cn} represents the slot type, x j Let be the i-th slot, and cn be the total number of slots. Then the set of label types for the slot label sequence of question q is c = {[PAD], [UNK], [O], [B_x]}. i ],[I_x i ]}; where, [B_x i [Starting with x] i The starting identifier of the slot, [I_x i As x i The remaining identifiers for the slots are: [PAD] for the padding token tag in the question, [UNK] for the unrecognized sequence tag, and [O] for the token tag without a slot.

[0042] Rhetorical question q i ={w i1 , ..., w iqn}, where iqn is the question q i The total number of characters, where any single character is w; if the question is q i The a-th character w ia Marked [B_x] i The characters from the (a+1)th to the (a+wcn)th are marked [I_x] i ], then x i The slot value is c′ in ={w a w a+1 , ..., w a+wcn}, its length is a+wcn+1;

[0043] Regarding the question q i ={wi1, ..., w iqn} and q j ={w j1 , ..., w jqm}, where any one of its slot values ​​is c′ i ={c′ i1 , ..., c′ in} and slot value c′ j ={c′ j1 , ..., c′ jm The word embedding model GloVe is used to embed the slot values ​​into vectors, and their vector representations are obtained as follows:

[0044] w′ ia =GloVe(c′) i )

[0045] w′ jb =GloVe(c′) j )

[0046] For vector w′ ia sum vector w′ jb Euclidean distance dE(w′) ia , w′ jb Perform the calculation:

[0047]

[0048] Iterating through the question q i and question q j The Euclidean distance between all paired slot values, if w′ ia With w′ jb If the Euclidean distance of the node q is less than the connection threshold θ, then the question node q is considered to be... i And question node q j The adjacent node is the question node q. i And question node q jIf the relation "connection" exists, then the question node q is not considered. i And question node q j There is no relationship "connection";

[0049] After traversing the question set Q, the corresponding question graph data G = {C′, Q} is constructed, where the relation edges of the question graph are "connections", and the set of nodes is the question set Q.

[0050] Preferably, the embedded representation unit is specifically used for:

[0051] For the question graph data G, the graph transformer outputs the embedding representation vector for each question node. The graph transformer contains an L-layer network structure, and the output sequence becomes the input sequence for the next time step. The specific formula for representing the graph attention module is as follows:

[0052]

[0053] Wherein, the operator ∨ ​​concatenates N attention contents, a(q i q j ) represents the attention weights of two different questions within the graph attention mechanism, a n (q i q j A represents the attention weight value in the nth attention layer. L A K The feature matrix represents the attention weight values. Let A be the feature matrix of the nth attention part; for each attention weight a, the feature matrices A of node i and node j are obtained by operation. K A L That is, the degree of association between two nodes, i and j;

[0054] Under each graph attention module, the operations of the feedforward neural network layer and the regularization layer are as follows:

[0055]

[0056]

[0057] FFN(x) = W1 × (f(W2 × x + b1) + b2)

[0058] Where FFN is a feedforward neural network, q' i represents the output of FFN, LayerNorm represents the normalization layer, W1 and W2 are the feature matrices in the feedforward neural network, and b1 and b2 are the bias values.

[0059] After iterative optimization, the output question node embedding representation vector is used. This is the representation vector for the question graph data.

[0060] Preferably, the problem chain generation model adopts the ConvKB model, and its prediction method is as follows:

[0061]

[0062] in, For a given question chain S in the question graph data and the question q to be predicted j A function that determines whether a chain relationship exists outputs a relationship prediction score. Represents the chain of questions S and the question to be predicted q. j The chain of prediction problems, h” S Let g be the vector representation of the problem chain S; r For the question chain S and the question to be predicted q j Relational vectors, || is the concatenation symbol for vectors, ω m Let Ω be the m-th convolutional filter, Ω be the hyperparameter representing the number of filters in the convKB model, and * be the convolution operator; W∈R Ωk×1 The linear transformation matrix for the final score of the convKB model; level n The last question q in the question chain S n The graph complexity coefficient in question graph data.

[0063] This invention also provides a question chain generation device based on question graph data, which includes a memory and a processor. The memory stores a computer program that can be executed by the processor to achieve the question chain generation method based on question graph data as described above.

[0064] Compared with the prior art, this embodiment has the following advantages:

[0065] 1. By combining two question chain generation methods—one based on question sets and the other based on graph data—we can leverage their respective strengths and compensate for their weaknesses, effectively improving the relevance of the generated question chains and reducing the reliance on large amounts of entity relationship data.

[0066] 2. By utilizing the slot relationships between questions to construct a question graph, the association of question chains that rely solely on the semantic relevance of questions is surpassed, forming a more logical question chain. This allows for capturing the entity relationships maintained by multi-hop questions by focusing only on whether the questions before and after each hop of the question chain have logical relationships, thus achieving highly efficient generation of question chains. Attached Figure Description

[0067] To more clearly illustrate the technical solution of the present invention, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0068] Figure 1 A flowchart illustrating the question chain generation method based on question graph data provided in the first embodiment of the present invention;

[0069] Figure 2 A schematic diagram of the overall working principle provided for embodiments of the present invention;

[0070] Figure 3 This is a schematic diagram of the structure of the question chain generation device based on question graph data according to the second embodiment of the present invention. Detailed Implementation

[0071] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0072] Please see Figure 1 and Figure 2 The first embodiment of the present invention provides a question chain generation method based on question graph data, which can be executed by a question chain generation device based on question graph data (hereinafter referred to as the generation device), specifically, by one or more processors within the generation device, to achieve the following steps:

[0073] S1. In the question set, slot label prediction is performed for the questions, and the connection relationship between questions is created using the label vector distance. Question graph data is constructed with individual questions as nodes and connections as edges.

[0074] In this embodiment, a question set Q = {q1, ..., q} can be constructed from the continuous question data in the CrossWOZ dialogue data. N For example, a question set of size N = 102000 can be constructed.

[0075] Then, an attention-based bidirectional RNN slot prediction model is used to obtain the slot label sequence for question q.

[0076] Specifically, let X = {x1, ..., x} cn} represents the slot type, x iLet be the i-th slot, and cn be the total number of slots. Then the set of label types for the slot label sequence of question q is c = {[PAD], [UNK], [O], [B_x]}. i ],[I_x i ]}; where, [B_x i [Starting with x] i The starting identifier of the slot, [I_x i As x i The remaining identifiers for the slots are: [PAD] for the paddingtoken tag in the question, [UNK] for the unrecognized sequence tag, and [O] for the token tag without a slot.

[0077] If we are dealing with question q i If the query is "["Do you know which cinemas are showing Avatar?"]", then the slot label set is [[O], [UNK], [B-location], [I-location], [B-movie name], [I-movie name]]. This model can extract slot labels containing 120 tags. Example labels are as follows:

[0078] Slot Labels [O] [UNK] [B-Location] [I-Location] [B-Movie Title] [I-Movie Title]

[0079] Then, the rhetorical question q i ={w i1 , ..., w iqn}, where iqn is the question q i The total number of characters, where any single character is w; if the question is q i The a-th character w ia Marked [B_x] i The characters from the (a+1)th to the (a+wcn)th are marked [I_x] i ], then x i The slot value is c′ in ={w a w a+1 , ..., w a+wcn}, whose length is a+wcn+1; then for the question q i ={wi1, ..., w iqn} and q j ={w j1 , ..., w jqm}, where any one of its slot values ​​is c′ i ={c′ i1 , ..., c′ in} and slot value c′ j ={c′ j1 , ..., c′ jm The word embedding model GloVe is used to embed the slot values ​​into vectors, and their vector representations are obtained as follows:

[0080] w′ ia =GloVe(c′) i )

[0081] w′ jb =GloVe(c′) j )

[0082] For vector w′ ia sum vector w′ jb Euclidean distance dE(w′) ia , w′ jb Perform the calculation:

[0083]

[0084] Iterating through the question q i and question q j The Euclidean distance between all paired slot values, if w′ ia With w′ jb If the Euclidean distance is less than the connection threshold θ, then the question node q is considered to be... i And question node q j The adjacent node is the question node q. i And question node q j If the relation "connection" exists, then the question node q is not considered. i And question node q j There is no relationship "connection".

[0085] For example, regarding the question q i The slot label sequence for ["Do you know which cinemas are showing Avatar?"] is c' i ={[O][O][O][O][O][O][B-location][I-location][I-location][O][O][O][UNK][B-movie name][I-movie name][I-movie name][UNK][O][UNK]}, for any optional question q j The slot label sequence c′ for the question "Thank you, how do I get to Universal Studios Beijing?" j ={[O][O][UNK][O][O][B-place name][I-place name][B-place][I-place][I-place][I-place][O][O][O][O][O][O][O][O][UNK]}. Use question q i and question q j The vector representation of the slot label sequence is used to calculate the Euclidean distance between the two nodes. If the Euclidean distance between the two nodes is less than the connection threshold θ = 0.7, then the question node q is considered to be... i And question node q j There is a relationship called "connection".

[0086] Finally, after traversing the question set Q, the corresponding question graph data G = {C′, Q} is constructed, where the relation edges of the question graph are "connections", and the set of nodes is the question set Q.

[0087] In this embodiment, the above steps are repeated until Euclidean distance is determined between every pair of questions. Finally, the corresponding question graph data G = {C′, Q} is obtained based on the question set Q, where the graph's relational edges are "connections," and the set of nodes is the question set Q.

[0088] S2, the question graph data is fed into the graph Transformer for graph representation learning, the nodes of the question graph data are embedded to obtain the representation vector of the question graph data.

[0089] In this embodiment, for the question graph data G, the embedding representation vector of each question node is output according to the graph transformer. The graph transformer contains an L-layer network structure, and the output sequence becomes the input sequence for the next time step. The specific formula for representing the graph attention module is as follows:

[0090]

[0091] Wherein, the operator ∨ ​​concatenates N attention contents, a(q i q j ) represents the attention weights of two different questions within the graph attention mechanism, a n (q i q j A represents the attention weight value in the nth attention layer. L A K The feature matrix represents the attention weight values. Let A be the feature matrix of the nth attention part; for each attention weight a, the feature matrices A of node i and node j are obtained by operation. K A L That is, the degree of association between two nodes, i and j;

[0092] Under each graph attention module, the operations of the feedforward neural network layer and the regularization layer are as follows:

[0093]

[0094]

[0095] FFN(x) = W1 × (f(W2 × x + b1) + b2)

[0096] Where FFN is a feedforward neural network, q'i represents the output of FFN, LayerNorm represents the normalization layer, W1 and W2 are the feature matrices in the feedforward neural network, and b1 and b2 are the bias values.

[0097] After iterative optimization, the output question node embedding representation vector is used. This is the representation vector for the question graph data.

[0098] S3, the representation vectors of the question to be predicted and the question graph data are used to perform link prediction through the question chain generation model to obtain the question with the highest confidence. Then, a link is constructed between the highest confidence question and the input question to be predicted, thereby constructing a one-hop question chain between the two questions.

[0099] The problem chain generation model can adopt the ConvKB model, and the calculation method used for ConvKB model prediction is as follows:

[0100]

[0101] in, For a given question chain S in the question graph data and the question q to be predicted j A function that determines whether a chain relationship exists outputs a relationship prediction score. h" represents the predicted question chain consisting of the question chain S and the question to be predicted. S Let g be the vector representation of the problem chain S; r Let S be the relation vector between the question chain S and the question to be predicted, || be the concatenation symbol for the vectors, and ω be the concatenation symbol. m Let Ω be the m-th convolutional filter, Ω be the hyperparameter representing the number of filters in the convKB model, and * be the convolution operator; W∈R Ωk×1 The linear transformation matrix for predicting the final score of the reasoning problem chain relationship prediction model; level n The last question q in the question chain S n The graph complexity coefficient in question graph data.

[0102] For example, regarding the question q i [“Do you know which cinemas are showing Avatar?”] This leads to the question that forms a chain of questions: [“Thank you, how do I get to Universal Studios Beijing?”]

[0103] In this embodiment, the loss function L used for training the ConvKB model is:

[0104]

[0105] Among them, l Sjλ represents the dimension of the question chain vector, and λ is a constant parameter for regularization. By optimizing the training with respect to the loss function, a question chain generation model based on question graph data can be obtained.

[0106] Compared with the prior art, this embodiment has the following advantages:

[0107] 1. By combining two question chain generation methods—one based on question sets and the other based on graph data—we can leverage their respective strengths and compensate for their weaknesses, effectively improving the relevance of the generated question chains and reducing the reliance on large amounts of entity relationship data.

[0108] 2. By utilizing the slot relationships between questions to construct a question graph, the association of question chains that rely solely on the semantic relevance of questions is surpassed, forming a more logical question chain. This allows for capturing the entity relationships maintained by multi-hop questions by focusing only on whether the questions before and after each hop of the question chain have logical relationships, thus achieving highly efficient generation of question chains.

[0109] Please see Figure 3 The second embodiment of the present invention also provides a question chain generation device based on question graph data, which includes:

[0110] Question graph construction unit 210 is used to predict slot labels for questions in a question set, create connections between questions using label vector distances, and construct question graph data with individual questions as nodes and connections as edges.

[0111] Embedding representation unit 220 is used to feed question graph data into graph Transformer for graph representation learning, embedding representation of nodes in question graph data, and obtaining representation vector of question graph data;

[0112] The question chain construction unit 230 is used to perform link prediction by passing the representation vector of the question to be predicted and the question graph data through the question chain generation model to obtain the question with the highest confidence, and to construct a link between the question with the highest confidence and the input question to be predicted, thereby constructing a one-hop question chain between the two questions.

[0113] Preferably, the question graph construction unit 210 is specifically used for:

[0114] For the question set Q = {q1, ..., q...} N The question q is obtained using a slot prediction model. i The slot label sequence; where q i Let X be the i-th question in the question set Q, and N be the size of the question set. Let X = {x1, ..., x2}. cn} represents the slot type, x i Let be the i-th slot, and cn be the total number of slots. Then the set of label types for the slot label sequence of question q is c = {[PAD], [UNK], [O], [B_x]}.i ],[I_x i ]}; where, [B_x i [Starting with x] i The starting identifier of the slot, [I_x i As x i The remaining identifiers for the slots are: [PAD] for the padding token tag in the question, [UNK] for the unrecognized sequence tag, and [O] for the token tag without a slot.

[0115] Rhetorical question q i ={w i1 , ..., w iqn}, where iqn is the question q i The total number of characters, where any single character is w; if the question is q i The a-th character w ia Marked [B_x] i The characters from the (a+1)th to the (a+wcn)th are marked [I_x] i ], then x i The slot value is c′ in ={w a w a+1 , ..., w a+wcn}, its length is a+wcn+1;

[0116] Regarding the question q i ={wi1, ..., w iqn} and q j ={w j1 , ..., w jqm}, where any one of its slot values ​​is c′ i ={c′ i1 , ..., c′ in} and slot value c′ j ={c′ j1 , ..., c′ jm The word embedding model GloVe is used to embed the slot values ​​into vectors, and their vector representations are obtained as follows:

[0117] w′ ia =GloVe(c′) i )

[0118] w′ jb =GloVe(c′) j )

[0119] For vector w′ ia sum vector w′ jb Euclidean distance d E (w′ ia , w′jb Perform the calculation:

[0120]

[0121] Iterating through the question q i and question q j The Euclidean distance between all paired slot values, if w′ ia With w′ jb If the Euclidean distance of the node q is less than the connection threshold θ, then the question node q is considered to be... i And question node q j The adjacent node is the question node q. i And question node q j If the relation "connection" exists, then the question node q is not considered. i And question node q j There is no relationship "connection";

[0122] After traversing the question set Q, the corresponding question graph data G = {C′, Q} is constructed, where the relation edges of the question graph are "connections", and the set of nodes is the question set Q.

[0123] Preferably, the embedded representation unit 220 is specifically used for:

[0124] For the question graph data G, the graph transformer outputs the embedding representation vector for each question node. The graph transformer contains an L-layer network structure, and the output sequence becomes the input sequence for the next time step. The specific formula for representing the graph attention module is as follows:

[0125]

[0126] Wherein, the operator ∨ ​​concatenates N attention contents, a(q i q j ) represents the attention weights of two different questions within the graph attention mechanism, a n (q i q j A represents the attention weight value in the nth attention layer. L A K The feature matrix represents the attention weight values. Let A be the feature matrix of the nth attention part; for each attention weight a, the feature matrices A of node i and node j are obtained by operation. K A L That is, the degree of association between two nodes, i and j;

[0127] Under each graph attention module, the operations of the feedforward neural network layer and the regularization layer are as follows:

[0128]

[0129]

[0130] FFN(x) = W1 × (f(W2 × x + b1) + b2)

[0131] Where FFN is a feedforward neural network, W1 and W2 are feature matrices in the feedforward neural network, and b1 and b2 are bias values.

[0132] After iterative optimization, the output question node embedding representation vector is used. This is the representation vector for the question graph data.

[0133] Preferably, the problem chain generation model adopts the ConvKB model, and its prediction method is as follows:

[0134]

[0135] in, For a given question chain S in the question graph data and the question q to be predicted j A function that determines whether a chain relationship exists outputs a relationship prediction score. Represents the chain of questions S and the question to be predicted q. j The chain of prediction problems, h” S Let g be the vector representation of the problem chain S; r For the question chain S and the question to be predicted q j Relational vectors, || is the concatenation symbol for vectors, ω m Let Ω be the m-th convolutional filter, Ω be the hyperparameter representing the number of filters in the convKB model, and * be the convolution operator; W∈R Ωk×1 The linear transformation matrix for the final score of the convKB model; level n The last question q in the question chain S n The graph complexity coefficient in question graph data.

[0136] The third embodiment of the present invention also provides a question chain generation device based on question graph data, which includes a memory and a processor. The memory stores a computer program, which can be executed by the processor to realize the question chain generation method based on question graph data as described above.

[0137] In the several embodiments provided in this invention, it should be understood that the disclosed systems and methods can also be implemented in other ways. The system and method embodiments described above are merely illustrative. For example, the flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of the present invention. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions marked in the blocks may occur in a different order than those marked in the drawings. For example, two consecutive blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in a block diagram and / or flowchart, and combinations of blocks in block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or action, or using a combination of dedicated hardware and computer instructions.

[0138] In addition, the functional modules in the various embodiments of the present invention can be integrated together to form an independent part, or each module can exist independently, or two or more modules can be integrated to form an independent part.

[0139] If the aforementioned functions are implemented as software functional modules and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, electronic device, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks. It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. In the absence of further restrictions, an element defined by the phrase "comprising a..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0140] The above description is merely a preferred embodiment of the present invention and is not intended to limit the invention. Various modifications and variations can be made to the present invention by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A method for generating question chains based on question graph data, characterized in that, include: S1. In the question set, slot label prediction is performed for the questions, and the connection relationship between questions is created using the label vector distance. Question graph data is constructed with individual questions as nodes and connections as edges. S2, the question graph data is fed into the graph Transformer for graph representation learning, the nodes of the question graph data are embedded to obtain the representation vector of the question graph data; S3, the representation vectors of the question to be predicted and the question graph data are used to predict the link through a question chain generation model to obtain the question with the highest confidence, and a link is constructed between the question to be predicted and the question with the highest confidence, thereby constructing a one-hop question chain between the two questions; Step S1 specifically includes: For the question set The question is obtained using a slot prediction model. The slot label sequence; where, For Question Collection Let N be the i-th question and N be the size of the question set. ={ } is a slot type. For the first One slot, If the total number of slots is , then the set of label types for the slot label sequence of question q is . Among them, [B_ [Starting with] The starting identifier of the slot, [I_ As The remaining identifiers for the slots are: [PAD] for the padding token tag in the question, [UNK] for the unrecognized sequence tag, and [O] for the token tag without a slot. Rhetorical question q i ={w i1 , ..., w iqn }, where iqn is the question q i The total number of characters, where any single character is w; if the question is q i No. The character w ia Labeled The (a+1)th to (a+wcn)th characters are marked [I_ ], then x i The slot value Its length is a+wcn+1; Regarding the question q i ={w i1 , ..., w iqn } and q j ={w j1 , ..., w jqm }, where any one of its slot values ​​is respectively and slot value The word embedding model GloVe is used to embed the slot values ​​into vectors, and their vector representations are obtained as follows: For vectors sum vector European distance Perform the calculation: Enumeration Questions and questions The Euclidean distance between all paired slot values, if and The Euclidean distance is less than the connection threshold. Then the question node is considered to be... and question node Adjacent nodes, i.e., question nodes and question node The relation "connection" exists; otherwise, the question node... and question node There is no relationship "connection"; Traversing the set of questions Then, the corresponding question graph data was constructed. In the question graph, the relation edges are "connections", and the set of nodes is the set of questions. .

2. The question chain generation method based on question graph data according to claim 1, characterized in that: S2 specifically includes: Question graph data The graph transformer outputs the embedding representation vector for each question node. ; where the graph transformer contains In a layered network structure, the output sequence becomes the input sequence for the next time step. The specific formula for representing the graph attention module is as follows: Among them, operators For serial computation Each attention content, To determine the attention weights for two different questions within the graph attention mechanism, The attention weights are the values ​​for the nth attention layer. , The feature matrix represents the attention weight values. For the first The feature matrix of the head attention part; for each attention weight Calculate the nodes and nodes Feature matrix , i.e., computing nodes and nodes The degree of correlation between two nodes; Under each graph attention module, the operations of the feedforward neural network layer and the regularization layer are as follows: in, It is a feedforward neural network. This represents the output of FFN. Indicates the normalization layer. and This is the feature matrix in a feedforward neural network. and This is the bias value; After iterative optimization, the output question node embedding representation vector is used. This is the representation vector for the question graph data.

3. The question chain generation method based on question graph data according to claim 2, characterized in that: In step S3, the problem chain generation model adopts the ConvKB model, and its prediction method is as follows: in, For a question graph data, a certain question chain Questions to be predicted A function that determines whether a chain relationship exists outputs a relationship prediction score. Representing a problem chain Questions to be predicted The chain of prediction problems For the problem chain Vector representation of; For the problem chain Questions to be predicted relation vectors, The symbol for connecting vectors. Ω represents the m-th convolutional filter, Ω is the hyperparameter for the number of filters in the convKB model, and * represents the convolution operator. This is the linear transformation matrix for the final score of the convKB model; For the problem chain The last question The graph complexity coefficient in question graph data.

4. The question chain generation method based on question graph data according to claim 3, characterized in that: The loss function used in training the convKB model Represented as: in, It is the dimension of the problem chain vector. It is a regularization setting constant parameter.

5. A question chain generation device based on question graph data, characterized in that, include: The question graph construction unit is used to predict slot labels for questions in a question set, create connections between questions using label vector distance, and construct question graph data with individual questions as nodes and connections as edges. The embedding representation unit is used to feed the question graph data into the graph Transformer for graph representation learning, embedding the nodes of the question graph data to obtain the representation vector of the question graph data; The question chain construction unit is used to predict the link between the question to be predicted and the representation vector of the question graph data through a question chain generation model, obtain the question with the highest confidence, and construct a link between the question to be predicted based on the question with the highest confidence and the question to be predicted, thereby constructing a one-hop question chain between the two questions; wherein, the question graph construction unit is specifically used for: For the question set The question is obtained using a slot prediction model. The slot label sequence; where, For Question Collection Let N be the i-th question and N be the size of the question set. ={ } is a slot type. For the first One slot, Given the total number of slots, the question is... The set of label types for the slot label sequence is Among them, [B_ [Starting with] The starting identifier of the slot, [I_ As The remaining identifiers for the slots are: [PAD] for the padding token tag in the question, [UNK] for the unrecognized sequence tag, and [O] for the token tag without a slot. Rhetorical question ,in For question The total number of characters, any one of which is If a question No. each character Labeled , No. To the The character was marked [I_] ],but The slot value Its length is ; For questions and The values ​​of any one of its slots are respectively and slot value The word embedding model GloVe is used to embed the slot values ​​into vectors, and their vector representations are obtained as follows: For vectors sum vector European distance Perform the calculation: Enumeration Questions and questions The Euclidean distance between all paired slot values, if and The Euclidean distance is less than the connection threshold. Then the question node is considered to be... and question node Adjacent nodes, i.e., question nodes and question node The relation "connection" exists; otherwise, the question node... and question node There is no relationship "connection"; Traversing the set of questions Then, the corresponding question graph data was constructed. In the question graph, the relation edges are "connections", and the set of nodes is the set of questions. .

6. The question chain generation device based on question graph data according to claim 5, characterized in that: The embedded representation unit is specifically used for: Question graph data The graph transformer outputs the embedding representation vector for each question node. ; where the graph transformer contains In a layered network structure, the output sequence becomes the input sequence for the next time step. The specific formula for representing the graph attention module is as follows: Among them, operators For serial computation Each attention content, To determine the attention weights for two different questions within the graph attention mechanism, The attention weights are the values ​​for the nth attention layer. , The feature matrix represents the attention weight values. For the first The feature matrix of the head attention part; for each attention weight The operation yields the node. and nodes Feature matrix , i.e., computing nodes and nodes The degree of correlation between two nodes; Under each graph attention module, the operations of the feedforward neural network layer and the regularization layer are as follows: in, It is a feedforward neural network. This represents the output of FFN. Indicates the normalization layer. and This is the feature matrix in a feedforward neural network. and This is the bias value; After iterative optimization, the output question node embedding representation vector is used. This is the representation vector for the question graph data.

7. The question chain generation device based on question graph data according to claim 6, characterized in that: The problem chain generation model uses the ConvKB model, and its prediction method is as follows: in, For a question graph data, a certain question chain Questions to be predicted A function that determines whether a chain relationship exists outputs a relationship prediction score. Representing a problem chain Questions to be predicted The chain of prediction problems For the problem chain Vector representation of; For the problem chain Questions to be predicted relation vectors, The symbol for connecting vectors. Ω represents the m-th convolutional filter, Ω is the hyperparameter for the number of filters in the convKB model, and * represents the convolution operator. This is the linear transformation matrix for the final score of the convKB model; For the problem chain The last question The graph complexity coefficient in question graph data.

8. A question chain generation device based on question graph data, characterized in that, The system includes a memory and a processor, wherein the memory stores a computer program that can be executed by the processor to implement the question chain generation method based on question graph data as described in any one of claims 1 to 4.

Images