Common sense question answering method and system based on dynamic global semantic fusion

Abstract

This invention discloses a commonsense question answering method and system based on dynamic global semantic fusion. The method involves obtaining an initial representation of the question context for the commonsense question to be answered; predicting a soft mask based on the correlation between the initial representation of the question context and the entity representation of the initial knowledge subgraph; generating a dynamic knowledge subgraph based on the soft mask; extracting features from the dynamic knowledge subgraph to obtain its initial representation; fusing word-level local information between the question context and the initial representation of the dynamic knowledge subgraph to obtain a word-level local information fused representation of the question context and the dynamic knowledge subgraph; performing multi-head attention learning on the word-level local information fused representation of the question context and the dynamic knowledge subgraph to obtain an updated representation of the question context and the dynamic knowledge subgraph; pooling the updated representation of the dynamic knowledge subgraph to obtain a pooled representation of the dynamic knowledge subgraph; and obtaining the answer to the commonsense question based on the pooled representation of the dynamic knowledge subgraph.

Description

Technical Field

[0001] This invention relates to the field of common sense question answering technology, and in particular to a common sense question answering method and system based on dynamic global semantic fusion. Background Technology

[0002] The statements in this section merely refer to the background art related to this invention and do not necessarily constitute prior art.

[0003] Common sense question answering involves providing correct answers to common sense questions through common sense reasoning. Large pre-trained language models like Roberta and T5 implicitly encode factual knowledge and have achieved significant success in factual question answering; however, they lack structured knowledge and cannot reason about common sense to complete the task. Knowledge graphs store structured knowledge with multiple relationships, and reasoning using knowledge graphs can greatly improve the accuracy of common sense question answering. Answering common sense questions requires structured knowledge to reason about the relationships between the entities mentioned in the question, that is, to infer the potential relationships between these entities. The background knowledge needed to answer the question is not included in the question itself, but this is common sense for humans; for computers, knowledge graphs provide structured knowledge for common sense reasoning.

[0004] Combining Language Models (LM) and Knowledge Graphs (KG) for commonsense question answering allows for the learning of factual knowledge from LM and structured knowledge from KG, compensating for the limitations of single knowledge and achieving some progress in commonsense question answering. However, these methods lack mutual fusion between LM and GNN, resulting in a lack of mutual updates between the two modalities, which limits the model's structured reasoning ability. To address this issue, researchers have proposed commonsense question answering models that fuse LM and KG information. Yasunaga's QA-GNN model uses a joint graph to unidirectionally update information from both modalities, but this limits the interaction between the question context and KG, failing to detect subtle differences in the question. Therefore, some have proposed using modal interaction layers or bidirectional attention modules to perform bidirectional information fusion at the word level, enhancing the model's structured reasoning ability and its ability to handle negative commonsense questions. For example, Sun's JointLK model uses a bidirectional attention module to perform word-level bidirectional fine-grained fusion updates of the question context and KG, solving the problem of ineffective information fusion. However, accurately answering commonsense questions requires a deeper understanding of both modal information.

[0005] Existing commonsense question answering methods that fuse logical reasoning (LM) and knowledge graph (KG) perform word-level local information interaction fusion, neglecting the fusion of question context and overall semantic information from the knowledge graph. This results in a lack of understanding of complex commonsense questions. Secondly, retrieving question-related subgraphs from the knowledge graph through string matching or semantic similarity introduces many semantically irrelevant entity nodes. Finally, existing research demonstrates that edge information in the knowledge graph plays a crucial role in knowledge reasoning and question answering, and existing methods, such as Generative Neural Networks (GNNs), fail to effectively aggregate this edge information. Summary of the Invention

[0006] To address the shortcomings of existing technologies, this invention provides a commonsense question-answering method and system based on dynamic global semantic fusion; this invention can obtain information and solve problems by answering commonsense questions, thereby improving information acquisition efficiency.

[0007] On the one hand, it provides a commonsense question-answering method based on dynamic global semantic fusion;

[0008] Commonsense question answering methods based on dynamic global semantic fusion include:

[0009] Obtain the common sense questions to be answered and the initial knowledge subgraph;

[0010] The common sense questions to be answered and the initial knowledge subgraph are input into the trained common sense question answering model, and the answer corresponding to the common sense question is output.

[0011] The trained commonsense question-answering model is used for:

[0012] Feature extraction is performed on the common sense questions to be answered to obtain an initial representation of the question context;

[0013] Based on the correlation between the initial representation of the question context and the entity representation of the initial knowledge subgraph, a soft mask is predicted; based on the soft mask, entity nodes related to common sense questions are identified from the initial knowledge subgraph, and then a dynamic knowledge subgraph is generated.

[0014] Feature extraction is performed on the dynamic knowledge subgraph to obtain its initial representation;

[0015] The initial representation of the problem context and the initial representation of the dynamic knowledge subgraph are fused with word-level local information to obtain the word-level local information fused representation of the problem context and the word-level local information fused representation of the dynamic knowledge subgraph.

[0016] Multi-head attention learning is performed on the word-level local information fusion representation of the problem context and the word-level local information fusion representation of the dynamic knowledge subgraph to obtain the updated representation of the problem context and the updated representation of the dynamic knowledge subgraph.

[0017] The updated representation of the dynamic knowledge subgraph is pooled to obtain the pooled representation of the dynamic knowledge subgraph, and the answer to the common sense question is obtained based on the pooled representation of the dynamic knowledge subgraph.

[0018] On the other hand, it provides a commonsense question-answering system based on dynamic global semantic fusion;

[0019] A commonsense question-answering system based on dynamic global semantic fusion includes:

[0020] The acquisition module is configured to acquire common sense questions to be answered and an initial knowledge subgraph.

[0021] The question-answering module is configured to: input the common sense question to be answered and the initial knowledge subgraph into the trained common sense question-answering model, and output the answer corresponding to the common sense question;

[0022] The trained commonsense question-answering model is used for:

[0023] Feature extraction is performed on the common sense questions to be answered to obtain an initial representation of the question context;

[0024] Based on the correlation between the initial representation of the question context and the entity representation of the initial knowledge subgraph, a soft mask is predicted; based on the soft mask, entity nodes related to common sense questions are identified from the initial knowledge subgraph, and then a dynamic knowledge subgraph is generated.

[0025] Feature extraction is performed on the dynamic knowledge subgraph to obtain its initial representation;

[0026] The initial representation of the problem context and the initial representation of the dynamic knowledge subgraph are fused with word-level local information to obtain the word-level local information fused representation of the problem context and the word-level local information fused representation of the dynamic knowledge subgraph.

[0027] Multi-head attention learning is performed on the word-level local information fusion representation of the problem context and the word-level local information fusion representation of the dynamic knowledge subgraph to obtain the updated representation of the problem context and the updated representation of the dynamic knowledge subgraph.

[0028] The updated representation of the dynamic knowledge subgraph is pooled to obtain the pooled representation of the dynamic knowledge subgraph, and the answer to the common sense question is obtained based on the pooled representation of the dynamic knowledge subgraph.

[0029] Furthermore, an electronic device is also provided, including:

[0030] Memory, used for non-transitory storage of computer-readable instructions; and

[0031] Processor, for executing the computer-readable instructions,

[0032] When the computer-readable instructions are executed by the processor, they perform the method described in the first aspect above.

[0033] In another aspect, a storage medium is also provided for non-transitory storage of computer-readable instructions, wherein when the non-transitory computer-readable instructions are executed by a computer, the instructions of the method described in the first aspect are executed.

[0034] In another aspect, a computer program product is also provided, including a computer program that, when run on one or more processors, is used to implement the method described in the first aspect above.

[0035] The above technical solution has the following advantages or beneficial effects:

[0036] This invention designs a multi-layer attention fusion module that utilizes cross-attention to fuse word-level local information and multi-head attention to fuse global semantic information, thus more effectively integrating word-level local and global semantic information. In the commonsense reasoning part, this invention designs a DGNN (Dynamic Graph Neural Network) module based on the Graph Attention Network (GAT) to form a dynamic knowledge subgraph, limiting the propagation of noisy entity information and maintaining the integrity of the knowledge subgraph. Furthermore, this invention utilizes the attention weights between the question and edge information to learn the edge information of the knowledge subgraph to improve the model's structured commonsense reasoning ability. Attached Figure Description

[0037] The accompanying drawings, which form part of this invention, are used to provide a further understanding of the invention. The illustrative embodiments of the invention and their descriptions are used to explain the invention and do not constitute an improper limitation of the invention.

[0038] Figure 1 This is a flowchart of the commonsense question answering method based on dynamic global semantic fusion graph neural network as described in Embodiment 1 of the present invention;

[0039] Figure 2 This is the dynamic graph neural network structure diagram described in Embodiment 1 of the present invention;

[0040] Figure 3 This is a structural diagram of the cross-attention fusion described in Embodiment 1 of the present invention;

[0041] Figure 4 This is a structural diagram of context-aware interactive fusion as described in Embodiment 1 of the present invention. Detailed Implementation

[0042] It should be noted that the following detailed descriptions are exemplary and intended to provide further illustration of the invention. Unless otherwise specified, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains.

[0043] Example 1

[0044] This embodiment provides a commonsense question-answering method based on dynamic global semantic fusion;

[0045] Commonsense question answering methods based on dynamic global semantic fusion include:

[0046] S101: Obtain the common sense questions to be answered and the initial knowledge subgraph;

[0047] S102: Input the common sense question to be answered and the initial knowledge subgraph into the trained common sense question answering model, and output the answer to the common sense question;

[0048] The trained commonsense question-answering model is used for:

[0049] S102-1: Extract features from the common sense question to be answered to obtain an initial representation of the question context;

[0050] S102-2: Based on the correlation between the initial representation of the question context and the entity representation of the initial knowledge subgraph, predict the soft mask; based on the soft mask, identify entity nodes related to the common sense question from the initial knowledge subgraph, and then generate a dynamic knowledge subgraph;

[0051] S102-3: Extract features from the dynamic knowledge subgraph to obtain the updated initial representation of the dynamic knowledge subgraph;

[0052] S102-4: The initial representation of the problem context and the updated initial representation of the dynamic knowledge subgraph are fused with word-level local information to obtain the word-level local information fused representation of the problem context and the word-level local information fused representation of the dynamic knowledge subgraph.

[0053] S102-5: Perform multi-head attention learning on the word-level local information fusion representation of the problem context and the word-level local information fusion representation of the dynamic knowledge subgraph to obtain the updated representation of the problem context and the updated representation of the dynamic knowledge subgraph.

[0054] S102-6: Perform pooling processing on the update representation of the dynamic knowledge subgraph to obtain the pooled representation of the dynamic knowledge subgraph, and obtain the answer to the common sense question based on the pooled representation of the dynamic knowledge subgraph.

[0055] Furthermore, the trained commonsense question-answering model, the training process of which includes:

[0056] Construct a commonsense question-answering model and build a training set, which consists of commonsense questions with known answers and an initial knowledge subgraph;

[0057] The training set is input into the commonsense question answering model to train the model. Training is stopped when the loss function value of the model stops decreasing, or when the number of iterations exceeds the set number, and the trained commonsense question answering model is obtained.

[0058] Furthermore, the initial knowledge subgraph includes several entities, with connecting edges between them. A portion of a knowledge graph is called a knowledge subgraph, and the knowledge subgraph of the initial input commonsense question-answering model is called the initial knowledge graph. The knowledge graph consists of entities and relations. Entities are concrete things, and edges represent the relations and attributes of things. For example: Entity 1 is a classroom, Entity 2 is a table, and the relation is: the table is placed in the classroom.

[0059] Furthermore, the network structure of the trained commonsense question-answering model includes:

[0060] A pre-trained language model, wherein the input of the pre-trained language model is used to input common sense questions to be answered;

[0061] The output of the pre-trained language model is connected to the first input of the first global semantic commonsense reasoning layer; the second input of the first global semantic commonsense reasoning layer is used to input the initial knowledge subgraph.

[0062] The first output of the first global semantic common sense reasoning layer is connected to the first input of the second global semantic common sense reasoning layer; the second output of the first global semantic common sense reasoning layer is connected to the second input of the second global semantic common sense reasoning layer.

[0063] The first output of the second global semantic common sense reasoning layer is connected to the first input of the third global semantic common sense reasoning layer; the second output of the second global semantic common sense reasoning layer is connected to the second input of the third global semantic common sense reasoning layer; and so on.

[0064] The first output of the (N-2)th global semantic common sense reasoning layer is connected to the first input of the (N-1)th global semantic common sense reasoning layer; the second output of the (N-2)th global semantic common sense reasoning layer is connected to the second input of the (N-1)th global semantic common sense reasoning layer.

[0065] The first output of the (N-1)th global semantic common sense reasoning layer is connected to the first input of the Nth global semantic common sense reasoning layer; the second output of the (N-1)th global semantic common sense reasoning layer is connected to the second input of the Nth global semantic common sense reasoning layer.

[0066] The first output of the Nth global semantic commonsense reasoning layer is connected to the first input of the prediction layer; the second output of the Nth global semantic commonsense reasoning layer is connected to the second input of the prediction layer; the output of the prediction layer is used to output the predicted answer to the commonsense question.

[0067] Furthermore, the internal structures of the first global semantic common sense reasoning layer, the second global semantic common sense reasoning layer, the third global semantic common sense reasoning layer, the (N-2)th global semantic common sense reasoning layer, the (N-1)th global semantic common sense reasoning layer, and the Nth global semantic common sense reasoning layer are the same. The first global semantic common sense reasoning layer includes: a dynamic graph neural network and a multi-layer attention fusion module connected in sequence.

[0068] The multi-layer attention fusion module includes: a cross-attention fusion module and a context-aware interaction fusion module connected in sequence.

[0069] Furthermore, the dynamic graph neural network includes:

[0070] The first input terminal of the dynamic graph neural network, the second input terminal of the dynamic graph neural network, the first output terminal of the dynamic graph neural network, and the second output terminal of the dynamic graph neural network;

[0071] The first input terminal of the dynamic graph neural network is used to input the initial representation of the problem context;

[0072] The second input terminal of the dynamic graph neural network is used to input the initial knowledge subgraph;

[0073] The first output of the dynamic graph neural network is used to output an initial representation of the problem context;

[0074] The second output of the dynamic graph neural network is used to output the initial representation of the dynamic knowledge subgraph;

[0075] The first input of the dynamic graph neural network is connected to the input of the multiplier through an average pooling layer.

[0076] The second input terminal of the dynamic graph neural network is connected to the input terminal of the multiplier;

[0077] The output of the multiplier is used to output a soft mask;

[0078] Based on the soft mask and the initial knowledge subgraph, a dynamic knowledge subgraph is obtained;

[0079] The dynamic knowledge subgraph is input into a Graph Attention Network (GAT) and outputs an initial representation of the dynamic knowledge subgraph.

[0080] Furthermore, this invention improves Graph Attention Networks (GAT) to obtain an improved Graph Attention Network. The improved Graph Attention Network calculates edge weight scores using edge information and problem context representation, and then adds edge weight scores to the message aggregation function of the Graph Attention Network (GAT) to aggregate edge information.

[0081] Furthermore, the cross-attention fusion module includes:

[0082] The system comprises a first gated fusion unit (Gate Fusion), a second gated fusion unit (Gate Fusion), a first alignment unit (Aligned representation computation), and a second alignment unit (Aligned representation computation).

[0083] The initial representation of the problem context is input into the first gate fusion unit (Gate Fusion) and the first alignment unit, respectively.

[0084] The initial representation of the dynamic knowledge subgraph is input into the second gate fusion unit (Gate Fusion) and the second alignment unit, respectively.

[0085] The input of the first gate fusion unit, Gate Fusion, is also connected to the output of the second alignment unit, Alignedrepresentation computation.

[0086] The input of the second gate fusion unit Gate Fusion is also connected to the output of the first alignment unit Alignedrepresentation computation;

[0087] The output of the first gate fusion unit, Gate Fusion, is used to output a word-level local information fusion representation of the problem context;

[0088] The output of the second gated fusion unit, Gate Fusion, is used to output a word-level local information fusion representation of the dynamic knowledge subgraph.

[0089] Furthermore, the first gate fusion unit (Gate Fusion) and the second gate fusion unit (Gate Fusion) operate on the same principle. The working principle of the first gate fusion unit (Gate Fusion) is as follows:

[0090] The initial representation and the aligned representation of the problem context are used to learn a weight vector by vector concatenation and Hadamard, and then normalized by the Sigmoid function to control the fusion ratio of the initial representation and the aligned representation of the problem context.

[0091] Vector concatenation and Hadamard product are used to fuse the problem context and the problem context aligned representation to obtain a preliminary fused representation of the problem context;

[0092] Using the learned weight vectors, the initial representation of the problem context and the preliminary fusion representation of the problem context are fused to obtain word-level local information of the problem context.

[0093] Furthermore, the first alignment unit (Aligned representation computation) and the second alignment unit (Aligned representation computation) operate on the same principle. The working principle of the first alignment unit is as follows:

[0094] First, word-level attention matrices are computed using the initial representation of the question context and the initial representation of the dynamic knowledge subgraph.

[0095] Secondly, using word attention matrices and dynamic subgraph representations, an aligned representation of the question context is computed.

[0096] Furthermore, the context-aware interaction fusion module includes: a first branch and a second branch in parallel;

[0097] The input of the first branch is used to input a word-level local information fusion representation of the problem context; the output of the first branch is used to output an updated representation of the problem context.

[0098] The input of the second branch is used to input the word-level local information fusion representation of the dynamic knowledge subgraph; the output of the second branch is used to output the updated representation of the dynamic knowledge subgraph.

[0099] The first branch includes: a first multi-head attention mechanism layer, a first adder, a first normalization layer, a second multi-head attention mechanism layer, a second adder, a second normalization layer, a first feedforward neural network, a third adder, and a third normalization layer, which are connected in sequence.

[0100] The input of the first adder is also connected to the input of the first multi-head attention mechanism layer; the input of the second adder is also connected to the input of the second multi-head attention mechanism layer; the input of the third adder is also connected to the input of the first feedforward neural network.

[0101] The second branch includes: a third multi-head attention mechanism layer, a fourth adder, a fourth normalization layer, a fourth multi-head attention mechanism layer, a fifth adder, a fifth normalization layer, a second feedforward neural network, a sixth adder, and a sixth normalization layer, which are connected in sequence.

[0102] The input of the fourth adder is also connected to the input of the third multi-head attention mechanism layer; the input of the fifth adder is also connected to the input of the fourth multi-head attention mechanism layer; the input of the sixth adder is also connected to the input of the second feedforward neural network.

[0103] The input of the second multi-head attention mechanism layer is also connected to the output of the fourth normalization layer;

[0104] The input of the fourth multi-head attention mechanism layer is also connected to the output of the first normalization layer.

[0105] Further, step S102-1: extracting features from the common sense question to be answered to obtain an initial representation of the question context, specifically including:

[0106] A pre-trained language model is used to extract features from the common sense questions to be answered, resulting in an initial representation of the question context.

[0107] Further, S102-1: The step of extracting features from the common sense question to be answered to obtain an initial representation of the question context specifically includes:

[0108] The question context c[q; a] is encoded using a pre-trained language model RoBERTa-Large to obtain an initial representation of the question, Q. 0 as follows:

[0109]

[0110] Where M is the length of the initial representation of the problem, and d1 is the size of the hidden state dimension.

[0111] The problem representation, i.e., the initial problem representation vector sequence, is obtained through the LM Encoder Layer of a large-scale pre-trained language model.

[0112] Further, S102-2: The step of predicting a soft mask based on the correlation between the initial representation of the problem context and the entity representation of the initial knowledge subgraph; identifying entity nodes related to the commonsense question from the initial knowledge subgraph based on the soft mask, and then generating a dynamic knowledge subgraph, specifically includes:

[0113] A dynamic graph neural network is used to predict a soft mask based on the correlation between the initial representation of the question context and the entity representation of the initial knowledge subgraph. Based on the soft mask, entity nodes related to common sense questions are identified from the initial knowledge subgraph, thereby generating a dynamic knowledge subgraph.

[0114] Further, S102-2: The step of predicting a soft mask based on the correlation between the initial representation of the problem context and the entity representation of the initial knowledge subgraph; identifying entity nodes related to the commonsense question from the initial knowledge subgraph based on the soft mask, and then generating a dynamic knowledge subgraph, specifically includes:

[0115] (21) A soft mask is predicted by utilizing the correlation between the initial representation of the problem context and the knowledge subgraph representation. The soft mask acts as an information control valve, allowing only entity nodes related to the problem to propagate information.

[0116] (22) Multiply the soft mask and the initial knowledge subgraph representation to obtain the dynamic knowledge subgraph.

[0117] Furthermore, (21) a soft mask is predicted by utilizing the correlation between the initial representation of the problem context and the knowledge subgraph representation. The soft mask acts as an information control valve, allowing only entity nodes related to the problem to propagate information, specifically including:

[0118] Using the problem context to represent Q (l-1) and knowledge subgraph entity representation X (l-1) To predict a soft mask m based on the correlation between them. l The goal is to form a dynamic knowledge subgraph on the knowledge subgraph, with a soft mask m. l As an information control valve, it only allows information to be propagated by entity nodes that are relevant to the problem, and restricts the propagation of information by noisy nodes.

[0119] The soft mask is calculated as follows:

[0120]

[0121]

[0122]

[0123] Among them, Q (l-1) It is a representation of the problem context. The problem context represents Q. (l-1) Mean pooling representation, It is a knowledge subgraph entity node The correlation score between the problem context representation and mean pooling, m l Indicates a soft mask. This represents the i-th entity node in the knowledge subgraph, where d1 represents the scaling factor, which is a constant. It is Q (l-1) Mean pooling representation, This indicates that parameters can also be trained. Let represent the entity representation of the i-th knowledge subgraph in the (l-1)-th layer, and σ be the sigmoid function.

[0124] Furthermore, (22) the multiplication of the soft mask and the initial knowledge subgraph representation to obtain the dynamic knowledge subgraph specifically includes:

[0125] The dynamic knowledge subgraph is obtained by multiplying a soft mask with the initial entity representation, which encourages the entities that need to be initiated and penalizes other entities, thus limiting the information propagation process to the dynamic knowledge subgraph.

[0126] The dynamic knowledge subgraph is obtained as shown below:

[0127]

[0128] in, It is the initial representation of the dynamic knowledge subgraph. It is the first soft mask value in the soft mask. It is an entity node in the knowledge subgraph.

[0129] It should be understood that dynamic graph neural network modules are designed to better perform commonsense reasoning and reduce interference from noisy nodes in the knowledge subgraph. The structure of the dynamic graph neural network module is as follows: Figure 2 As shown.

[0130] Further, S102-3: The step of extracting features from the dynamic knowledge subgraph to obtain the updated initial representation of the dynamic knowledge subgraph specifically includes:

[0131] (31) Obtain the attention score between two entity nodes by using the representations of the two entity nodes;

[0132] (32) Based on the initial representation of the problem context and the edge information of the dynamic knowledge subgraph, the relevance is calculated to obtain the attention weight between the problem and the edge information;

[0133] (33) Based on the attention score between the two entity nodes, the side information is learned by adding attention weights between the question and the side information in the message aggregation function of the graph attention network GAT to update the dynamic knowledge subgraph representation and output the updated initial representation of the dynamic knowledge subgraph.

[0134] Furthermore, step (31) obtains the attention score between two entity nodes through their representations, specifically including:

[0135] The Graph Attention Network (GAT) is used to pass information across a dynamic knowledge subgraph to update the node representations of the dynamic knowledge subgraph, and to compute the attention score 'a' between two entities. ij as follows:

[0136]

[0137]

[0138] in, Let a represent the attention score between entity node i and entity node j in the dynamic knowledge subgraph. ij The attention score between normalized entity nodes is represented by d1, which is a scaling factor and is a constant. q and W k These are trainable parameters. and Let i and j represent the nodes of the dynamic knowledge subgraph at layer l-1.

[0139] Furthermore, step (32) involves performing relevance calculations based on the initial representation of the problem context and the edge information of the dynamic knowledge subgraph to obtain the attention weights between the problem and the edge information, specifically including:

[0140] To better encode side information in message aggregation, we use side information r. ij The edge attention score b is calculated. ij Then, each edge needs to be globally normalized to generate the edge attention weight B. ij The edge attention weights are calculated as follows:

[0141]

[0142] Where, r ij b represents the edge information between entity i and entity j in the dynamic knowledge subgraph. ij The information and question mentioned above refer to Q. l-1 Attention scores between W T These are trainable weight parameters. Bias vector, ";" indicates the concatenation of two vectors.

[0143]

[0144] in, It is a multilayer perceptron, r ij It is the edge information between entity node i and entity node j in the dynamic knowledge subgraph, B ij Let E represent the globally normalized edge attention weights, and let E represent the set of edges in the dynamic knowledge subgraph.

[0145] Further, step (33) involves learning edge information by adding attention weights between the question and edge information in the message aggregation function of the graph attention network GAT based on the attention score between the two entity nodes, in order to update the initial representation of the dynamic knowledge subgraph and output the updated initial representation of the dynamic knowledge subgraph, specifically including:

[0146] To update the representation of the dynamic knowledge subgraph, by adding B ij Aggregate edge information and update the entity node representation of the dynamic knowledge subgraph. The acquisition process, represented by the message aggregation function, is as follows:

[0147]

[0148] in, The message aggregation representation of the i-th entity node in the dynamic knowledge subgraph, a ij B represents the attention score between the i-th entity node and the j-th entity node in the dynamic knowledge subgraph. ij Note the weights when representing globally normalized edges;

[0149]

[0150] in, This represents the i-th entity node at the l-th level of the dynamic knowledge subgraph. This represents the i-th entity node at level l-1 of the dynamic knowledge subgraph, where N is the value of N. i E represents the neighboring nodes of node i. i It is the edge connected to node i. These are trainable parameters.

[0151] Further, S102-4: The initial representation of the problem context and the initial representation of the dynamic knowledge subgraph are fused with word-level local information to obtain a word-level local information fused representation of the problem context and a word-level local information fused representation of the dynamic knowledge subgraph, specifically including:

[0152] A cross-attention fusion module is used to fuse word-level local information from the initial representation of the problem context and the initial representation of the dynamic knowledge subgraph, resulting in a word-level local information fusion representation of the problem context and a word-level local information fusion representation of the dynamic knowledge subgraph.

[0153] Further, S102-4: The initial representation of the problem context and the updated initial representation of the dynamic knowledge subgraph are fused with word-level local information to obtain a word-level local information fused representation of the problem context and a word-level local information fused representation of the dynamic knowledge subgraph, specifically including:

[0154] (41): Alignment representation of the initial representation of the problem context and the dynamic knowledge subgraph representation based on cross attention;

[0155] (42): By using a gating mechanism, the aligned representation is fused with the original representations of the initial representation of the problem context and the dynamic knowledge subgraph representation, respectively, to obtain a word-level local information fusion representation of the problem and the dynamic knowledge subgraph.

[0156] Furthermore, the (41) step of obtaining the aligned representation of the initial representation of the problem context and the dynamic knowledge subgraph representation based on cross-attention specifically includes:

[0157] Using the problem context and dynamic knowledge subgraph representation To compute the word-level attention matrix S between the question and the knowledge subgraph l The word-level attention matrix calculation process is as follows:

[0158]

[0159] in, These are trainable parameters. The context of the problem is represented by Q. l-1 The i-th vector, Representing the dynamic knowledge subgraph representation X l The j-th vector, The context of the problem is represented by Q. l-1 The i-th vector and dynamic knowledge subgraph representation X l The j-th vector, Attention scores between;

[0160] Then, use S l The aligned representation of the computation problem is shown in the following calculation process:

[0161]

[0162]

[0163] in, S represents the word-level attention matrix l The row vector, S represents the word-level attention matrix with values ​​between [0,1]. l The row vector, where N represents the number of entity nodes in the dynamic knowledge subgraph. This represents the vector representation of the k-th entity node in the dynamic subgraph. Indicates context alignment of the problem The i-th vector representation, This represents the attention weights on entities in the knowledge subgraph. Problem context-aligned representation. This represents the weighted sum of the most critical entity nodes related to the problem in the knowledge subgraph.

[0164] Using S l The transpose of the expression is used to calculate the aligned representation using the same formulas as in formulas (12) and (13). This represents the weighted sum of keywords related to the knowledge subgraph in the context of the question.

[0165] It should be understood that a two-way fine-grained fusion of problem context and dynamic knowledge subgraph is achieved at the word level. This locates the most relevant entity nodes in the problem, and the cross-attention fusion structure is as follows: Figure 3 As shown.

[0166] Further, step (42): by using a gating mechanism to fuse the aligned representation with the original representations of the initial representation of the problem context and the dynamic knowledge subgraph representation, respectively, to obtain a word-level local information fusion representation of the problem and the dynamic knowledge subgraph, specifically including:

[0167] Using the context of the problem of alignment And aligned dynamic knowledge subgraph A gate fusion unit was designed to represent the original representation and the corresponding aligned representation;

[0168] Specifically, X is represented by a dynamic knowledge subgraph. l Aligned dynamic knowledge subgraph representation Integration, expressed as follows:

[0169]

[0170] Among them, X l This represents a dynamic knowledge subgraph representation. A dynamic knowledge subgraph representation representing alignment. W represents element-matrix multiplication. f and b f This represents the trainable parameters.

[0171] Gated fusion transmits alignment information, which includes a weight matrix that needs to be learned;

[0172] Gated fusion function

[0173]

[0174] in, and b g These are trainable parameters, σ is the activation function, and X is... l Dynamic knowledge subgraph representation, Aligned dynamic knowledge subgraph representation;

[0175] Therefore, the word-level local information fusion representation of dynamic knowledge subgraphs The expression is as follows:

[0176]

[0177] in, X represents the fusion representation of word-level local information in a dynamic knowledge subgraph. l This represents a dynamic knowledge subgraph representation. A dynamic knowledge subgraph representation representing alignment;

[0178] Regarding the problem context Q l-1 Perform the same operation to produce the output problem context.

[0179] Further, S102-5: The word-level local information fusion representation of the problem context and the word-level local information fusion representation of the dynamic knowledge subgraph are subjected to multi-head attention learning to obtain the updated representation of the problem context and the updated representation of the dynamic knowledge subgraph, specifically including:

[0180] A context-aware interaction fusion module is used to perform multi-head attention learning on the word-level local information fusion representation of the question context and the word-level local information fusion representation of the dynamic knowledge subgraph to obtain the updated representation of the question context and the updated representation of the dynamic knowledge subgraph.

[0181] Furthermore, the word-level local information fusion representation of the problem context and the word-level local information fusion representation of the dynamic knowledge subgraph are subjected to multi-head attention learning to obtain updated representations of the problem context and the dynamic knowledge subgraph, specifically including:

[0182] (51): Based on two multi-head attention of Transformer, global semantic information of problem context and dynamic knowledge subgraph is learned respectively;

[0183] (52): By exchanging key-value pairs among multiple attention, global semantic information is fused to achieve global fusion of semantic information of the problem context and dynamic knowledge subgraph;

[0184] (53): After each multi-head attention, a feedforward neural network, residual connection and layer normalization are applied to obtain the problem context update representation and the dynamic knowledge subgraph update representation.

[0185] Furthermore, the (51) method, which learns global semantic information of the problem context and the dynamic knowledge subgraph based on two multi-head attention mechanisms of Transformer, includes:

[0186] Given a problem context representation and the dynamic knowledge subgraph representation obtained from cross-attention fusion As shown by Transformer, the context representation of the mapping problem and dynamic knowledge subgraph representation To query the global semantic information Q in the context of the question q Global semantic information Q of dynamic knowledge subgraph x Keyword K q K x Sum of values ​​V q V x Q q K q and V q Form a group, Q x K x and V x Form another group. Each group performs multi-head attention computation separately and outputs the global semantic information O of the problem context. q and global semantic information O of dynamic knowledge subgraph x O q and O x The calculation process is the same, using O q The calculation process is used as a demonstration.

[0187] The single-head attention calculation process is as follows:

[0188]

[0189] in, d represents the global semantic information of the problem context (the result of the i-th attention head). k It's a scaling factor, which is a numerical value. Q represents a single head q K q V q .

[0190] The outputs of each attention head in the multi-head attention process are concatenated and subjected to a linear transformation as follows:

[0191] O q =MultiHead(Q q ,K q V q (18-1)

[0192]

[0193] Among them, O q Represents global semantic information in the context of the problem. This represents the global semantic information of the problem context (the result of the first attention head). These are trainable parameters. h represents the number of attention heads. After each multi-head attention, a residual connection is used, followed by layer normalization.

[0194] Furthermore, the (52) method: utilizing the exchange of key-value pairs between multi-head attention to fuse global semantic information, thereby achieving the fusion of global semantic information of the problem context and global semantic information of the dynamic knowledge subgraph, specifically includes: in order to achieve the fusion of global semantic information of the problem context and the knowledge subgraph, using two multi-head attention methods and exchanging key-value pairs to fuse global semantic information, and using the output O of the previous multi-head attention method. q and O x As input, mapping O q and O x As a query Q q′ Q x′ Keyword K q′ K x′ Sum V q′ V x′ ;

[0195] Q q′ K x′ and V x′ Form a group, Q x′ K q′ and V q′ They were then grouped into another set, each performing interactive computation by exchanging key-value pairs. The result was then calculated by fusing semantic information from the problem context. Global fusion results with dynamic knowledge subgraphs

[0196] The result of fusion of semantic information of the problem context Global fusion results with dynamic knowledge subgraphs The calculation process is the same, with input... To introduce,

[0197] The calculation method for a single attention head is as follows:

[0198]

[0199] in, The fusion result representing the semantic information of the problem context (the result of the i-th attention head), d k It's a scaling factor, which is a numerical value. Representing a single head

[0200] The outputs of each attention head in the multi-head attention process are concatenated and subjected to a linear transformation as follows:

[0201]

[0202]

[0203] in, The fusion result representing the semantic information of the problem context (the result of multi-head attention), The result of fusion of semantic information representing the problem context (the result of the first attention head), where h represents the number of attention heads; These are trainable parameters. After multi-head attention, a residual connection is used, followed by layer normalization.

[0204] Further, in step (53): after each multi-head attention, a feedforward neural network, residual connections, and layer normalization are applied to obtain the problem context update representation and the knowledge subgraph update representation, specifically including:

[0205] After each multi-head attention layer, a feedforward neural network (MLP) and residual connections are applied for layer normalization to obtain the output Q of the l-th layer global semantic commonsense reasoning layer. l ;

[0206] Residual connections help train deep neural networks more effectively. Problem context update representation Q l The calculation process is as follows:

[0207]

[0208] Among them, Q l This represents the problem context representation of the l-th layer of global semantic commonsense reasoning. This indicates that the problem context is updated.

[0209] Update representation in the context of the problem Perform the same operation on top, thereby producing the dynamic knowledge subgraph update representation of the output layer l-th global semantic commonsense reasoning layer.

[0210] Finally, the output of the l-th layer global semantic commonsense reasoning layer... and It serves as the input for the next layer of global semantic commonsense reasoning.

[0211] Understandably, to improve the understanding of complex common-sense problems, this invention utilizes context-aware interactive fusion to fuse the global semantic information of the problem context and the global semantic information of the dynamic knowledge subgraph. The Transformer can acquire global semantic information through multi-head attention. However, the traditional Transformer is not suitable for bidirectional fusion of semantic information. Therefore, this invention improves the Transformer by using two multi-head attention pairs to learn global semantic information, and then exchanging key-value pairs of global semantic information from the problem context and the knowledge subgraph within the multi-head attention to fuse the global semantic information. Specifically, the global semantic information O from the problem context... q and global semantic information O of dynamic knowledge subgraph x As input, it is then mapped to query Q. q′ Q x′ , key K q′ K x′ Value and value V q′ V x′ Q q′ K x′ 、 and V x′ Divide into one group, and add Q x′ K q′ and V q′ This forms another group, each computed using key-value pairs that are interchanged in a multi-head attention framework. The multi-head attention fusion structure is as follows: Figure 4 As shown.

[0212] It should be understood that in each layer of the global semantic commonsense reasoning layer, the multi-layer attention fusion module follows the dynamic graph attention module. To fuse word-level local information and global semantic information from the question context and the dynamic knowledge graph, this invention designs a multi-layer attention fusion module. Specifically, it includes cross-attention fusion that fuses word-level local information from the question context and the dynamic knowledge graph, and context-aware fusion that fuses the question context and the global semantic information of the KG (Knowledge Graph).

[0213] The l-th layer of global semantic commonsense reasoning inputs the question context representation of the (l-1)-th layer of global semantic commonsense reasoning. Dynamic knowledge subgraph representation of global semantic commonsense reasoning at layer l Then, the output is a problem context and a dynamic knowledge subgraph representation with bidirectional information fusion and updates.

[0214] Further, S102-6: The updated representation of the dynamic knowledge subgraph is subjected to pooling processing to obtain a pooled representation of the dynamic knowledge subgraph, and the answer to the commonsense question is obtained based on the pooled representation of the dynamic knowledge subgraph, specifically including:

[0215] A prediction layer is used to perform pooling processing on the update representation of the dynamic knowledge subgraph to obtain the pooled representation of the dynamic knowledge subgraph. The answer to the commonsense question is obtained based on the pooled representation of the dynamic knowledge subgraph.

[0216] Further, S102-6: The updated representation of the dynamic knowledge subgraph is subjected to pooling processing to obtain a pooled representation of the dynamic knowledge subgraph, and the answer to the commonsense question is obtained based on the pooled representation of the dynamic knowledge subgraph, specifically including:

[0217] The dynamic knowledge subgraph update representation is pooled based on graph pooling operations to obtain the dynamic knowledge subgraph pooled representation; the question context update representation and the dynamic knowledge subgraph pooled representation are input into the MLP multilayer perceptron to obtain the prediction score; based on the corresponding scores, the answer with the highest score is selected, thereby realizing common sense question answering.

[0218] Graph pooling operation definition: Aggregate the global information of a graph (dynamic knowledge subgraph) (including all nodes and all edges), and obtain a vector that can represent the entire graph. Then, perform pooling operation on this vector.

[0219] Further, S102-6: The updated representation of the dynamic knowledge subgraph is subjected to pooling processing to obtain a pooled representation of the dynamic knowledge subgraph, and the answer to the commonsense question is obtained based on the pooled representation of the dynamic knowledge subgraph, specifically including:

[0220] After N layers of global semantic commonsense reasoning, the final updated problem representation Q is obtained. N and knowledge subgraph representation X with bidirectional information fusion N Calculate the score p(q,a) of a correct answer as follows:

[0221]

[0222] Where, q N It is Q N Mean pooling representation, g N It is X N The graph pooling representation uses Softmax to normalize all questions—selecting the right ones to predict the most reasonable answer.

[0223] This invention belongs to the field of common sense question answering technology, and provides a common sense question answering method and system based on a dynamic global semantic fusion graph neural network model. It includes acquiring common sense questions and related knowledge subgraphs; and answering common sense questions using a trained common sense question answering model based on a dynamic global semantic fusion graph neural network, based on the acquired knowledge subgraphs. The model designs a novel multi-layer attention fusion module, utilizing cross-attention to fuse word-level local information and multi-head attention to fuse global semantic information, thus more effectively fusing word-level local and global semantic information. In the common sense reasoning part, this invention designs a DGNN (Dynamic Graph Neural Network) module based on an improved GAT (Graph Attention Network) to form a dynamic knowledge subgraph, limiting the propagation of noisy entity information. Furthermore, this invention also utilizes the attention weights between questions and edge information to learn the edge information of the knowledge subgraph to improve the model's structured common sense reasoning ability.

[0224] Example 2

[0225] This embodiment provides a commonsense question-answering system based on dynamic global semantic fusion;

[0226] A commonsense question-answering system based on dynamic global semantic fusion includes:

[0227] The acquisition module is configured to acquire common sense questions to be answered and an initial knowledge subgraph.

[0228] The question-answering module is configured to: input the common sense question to be answered and the initial knowledge subgraph into the trained common sense question-answering model, and output the answer corresponding to the common sense question;

[0229] The trained commonsense question-answering model is used for:

[0230] Feature extraction is performed on the common sense questions to be answered to obtain an initial representation of the question context;

[0231] Based on the correlation between the initial representation of the question context and the entity representation of the initial knowledge subgraph, a soft mask is predicted; based on the soft mask, entity nodes related to common sense questions are identified from the initial knowledge subgraph, and then a dynamic knowledge subgraph is generated.

[0232] Feature extraction is performed on the dynamic knowledge subgraph to obtain its initial representation;

[0233] The initial representation of the problem context and the initial representation of the dynamic knowledge subgraph are fused with word-level local information to obtain the word-level local information fused representation of the problem context and the word-level local information fused representation of the dynamic knowledge subgraph.

[0234] Multi-head attention learning is performed on the word-level local information fusion representation of the problem context and the word-level local information fusion representation of the dynamic knowledge subgraph to obtain the updated representation of the problem context and the updated representation of the dynamic knowledge subgraph.

[0235] The updated representation of the dynamic knowledge subgraph is pooled to obtain the pooled representation of the dynamic knowledge subgraph, and the answer to the common sense question is obtained based on the pooled representation of the dynamic knowledge subgraph.

[0236] It should be noted that the acquisition module and question-answering module mentioned above correspond to steps S101 to S102 in Embodiment 1. The examples and application scenarios implemented by the above modules and corresponding steps are the same, but are not limited to the content disclosed in Embodiment 1. It should be noted that the above modules, as part of the system, can be executed in a computer system such as a set of computer-executable instructions.

[0237] The descriptions of each embodiment in the above embodiments have different focuses. For parts not described in detail in a certain embodiment, please refer to the relevant descriptions in other embodiments.

[0238] The proposed system can be implemented in other ways. For example, the system embodiments described above are merely illustrative, and the division of modules described above is only a logical functional division. In actual implementation, there may be other division methods. For example, multiple modules may be combined or integrated into another system, or some features may be ignored or not executed.

[0239] Example 3

[0240] This embodiment also provides an electronic device, including: one or more processors, one or more memories, and one or more computer programs; wherein, the processor is connected to the memory, and the one or more computer programs are stored in the memory. When the electronic device is running, the processor executes the one or more computer programs stored in the memory to cause the electronic device to perform the method described in Embodiment 1.

[0241] It should be understood that in this embodiment, the processor can be a central processing unit (CPU), or it can be other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor can be a microprocessor or any conventional processor, etc.

[0242] Memory may include read-only memory and random access memory, and provides instructions and data to the processor. A portion of memory may also include non-volatile random access memory. For example, memory may also store information about the device type.

[0243] In the implementation process, each step of the above method can be completed by the integrated logic circuits in the processor hardware or by software instructions.

[0244] The method in Embodiment 1 can be directly implemented by a hardware processor, or implemented by a combination of hardware and software modules within the processor. The software modules can reside in readily available storage media in the art, such as random access memory, flash memory, read-only memory, programmable read-only memory, electrically erasable programmable memory, or registers. This storage medium is located in memory; the processor reads information from the memory and, in conjunction with its hardware, completes the steps of the above method. To avoid repetition, a detailed description is not provided here.

[0245] Those skilled in the art will recognize that the units and algorithm steps described in connection with the various examples of this embodiment can be implemented in electronic hardware or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this invention.

[0246] Example 4

[0247] This embodiment also provides a computer-readable storage medium for storing computer instructions, which, when executed by a processor, complete the method described in Embodiment 1.

[0248] The above description is merely a preferred embodiment of the present invention and is not intended to limit the invention. Various modifications and variations can be made to the present invention by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A commonsense question-answering method based on dynamic global semantic fusion, characterized by: include: Obtain the common sense questions to be answered and the initial knowledge subgraph; The common sense questions to be answered and the initial knowledge subgraph are input into the trained common sense question answering model, and the answer corresponding to the common sense question is output. The trained commonsense question-answering model is used for: Feature extraction is performed on the common sense questions to be answered to obtain an initial representation of the question context; Based on the correlation between the initial representation of the question context and the entity representation of the initial knowledge subgraph, a soft mask is predicted; based on the soft mask, entity nodes related to common sense questions are identified from the initial knowledge subgraph, and then a dynamic knowledge subgraph is generated. Feature extraction is performed on the dynamic knowledge subgraph to obtain the updated initial representation of the dynamic knowledge subgraph; The initial representation of the problem context and the updated initial representation of the dynamic knowledge subgraph are fused with word-level local information to obtain the word-level local information fused representation of the problem context and the word-level local information fused representation of the dynamic knowledge subgraph. Multi-head attention learning is performed on the word-level local information fusion representation of the problem context and the word-level local information fusion representation of the dynamic knowledge subgraph to obtain the updated representation of the problem context and the updated representation of the dynamic knowledge subgraph. Specifically: (51): Based on two multi-head attention mechanisms of Transformer, global semantic information of the problem context and dynamic knowledge subgraph is learned respectively; (52): By exchanging key-value pairs among multiple attention, global semantic information is fused to achieve global fusion of semantic information of the problem context and dynamic knowledge subgraph; (53): After each multi-head attention, a feedforward neural network, residual connections and layer normalization are applied to obtain the problem context update representation and the dynamic knowledge subgraph update representation; The updated representation of the dynamic knowledge subgraph is pooled to obtain the pooled representation of the dynamic knowledge subgraph, and the answer to the common sense question is obtained based on the pooled representation of the dynamic knowledge subgraph.

2. The commonsense question-answering method based on dynamic global semantic fusion as described in claim 1, characterized in that, The process of extracting features from the common sense questions to be answered to obtain an initial representation of the question context specifically includes: Encoding the problem context using the pre-trained language model RoBERTa-Large To obtain an initial representation of the problem, the initial representation of the problem as follows: in, It is the length of the initial representation of the problem. It is the size of the hidden state dimension.

3. The commonsense question-answering method based on dynamic global semantic fusion as described in claim 1, characterized in that, The soft mask is predicted based on the correlation between the initial representation of the problem context and the entity representation of the initial knowledge subgraph. Based on soft masking, entity nodes related to common sense questions are identified from the initial knowledge subgraph, and then a dynamic knowledge subgraph is generated, specifically including: (21) A soft mask is predicted by using the correlation between the initial representation of the problem context and the knowledge subgraph representation. The soft mask acts as an information control valve, allowing only entity nodes related to the problem to propagate information. (22) Multiply the soft mask and the initial knowledge subgraph representation to obtain the dynamic knowledge subgraph; Among them, (21) a soft mask is predicted by utilizing the correlation between the initial representation of the problem context and the knowledge subgraph representation. The soft mask acts as an information control valve, allowing only entity nodes related to the problem to propagate information, specifically including: Using the problem context and knowledge subgraph entity representation To predict a soft mask based on the correlation between them. The goal is to form dynamic knowledge subgraphs on knowledge subgraphs, using soft masks. As an information control valve, it only allows entities related to the problem to spread information and restricts the spread of information from noisy nodes; The soft mask is calculated as follows: （1） （2） （3） in, It is a representation of the problem context. It is a problem context representation Mean pooling representation, It is a knowledge subgraph entity node The correlation score between the problem context representation and mean pooling. Indicates a soft mask. Represents the knowledge subgraph. Each entity node represents, This represents the scaling factor, which is a constant. yes Mean pooling representation, It can also train parameters. Indicates the first The first in the layer Each knowledge subgraph entity is represented. yes function; Among them, (22) multiplying the soft mask and the initial knowledge subgraph representation to obtain the dynamic knowledge subgraph specifically includes: The dynamic knowledge subgraph is obtained by multiplying a soft mask with the initial entity representation, which encourages the entities that need to be initiated and penalizes other entities, thus limiting the information propagation process to the dynamic knowledge subgraph. Obtain the initial representation of the dynamic knowledge subgraph: （4） in, It is the initial representation of the dynamic knowledge subgraph. It is the soft mask The first soft mask value in the middle, It is an entity node in the knowledge subgraph.

4. The commonsense question-answering method based on dynamic global semantic fusion as described in claim 1, characterized in that, The step of extracting features from the dynamic knowledge subgraph to obtain the updated initial representation of the dynamic knowledge subgraph specifically includes: (31) Obtain the attention score between two entity nodes by using their representations; (32) Based on the initial representation of the problem context and the edge information of the dynamic knowledge subgraph, the relevance is calculated to obtain the attention weight between the problem and the edge information; (33) Based on the attention score between the two entity nodes, the side information is learned by adding attention weights between the question and the side information in the message aggregation function of the graph attention network GAT to update the dynamic knowledge subgraph representation and output the initial representation of the dynamic knowledge subgraph. Specifically, step (31) obtains the attention score between two entity nodes through their representations, including: The Graph Attention Network (GAT) is used to pass information across a dynamic knowledge subgraph to update the node representations of the dynamic knowledge subgraph and to compute the attention score between two entities. as follows: （5） （6） in, Represents entity nodes in a dynamic knowledge subgraph and dynamic knowledge subgraph entity nodes Attention scores between This represents the attention score between normalized entity nodes. This represents a scaling factor, which is a constant. and These are trainable parameters. and Indicates the first Layered dynamic knowledge subgraph nodes and express; Among them, (32) calculates the relevance based on the initial representation of the problem context and the edge information of the dynamic knowledge subgraph to obtain the attention weight between the problem and the edge information, specifically including: To better encode side information in message aggregation, side information is used. Calculate the edge attention score Then, each edge needs to be globally normalized to generate edge attention weights. The edge attention weights are calculated as follows: (7) in, Represents entities in a dynamic knowledge subgraph and dynamic knowledge subgraph entities Edge information between them The above text represents edge information and the problem. Attention scores between; （8） in, It is a multilayer perceptron. It is a dynamic knowledge subgraph entity node and dynamic knowledge subgraph entity nodes Edge information between them Note the weights when indicating globally normalized edges. Represents the set of edges in a dynamic knowledge subgraph; Specifically, (33) involves learning edge information by adding attention weights between the question and edge information in the message aggregation function of the graph attention network GAT based on the attention score between the two entity nodes, in order to update the initial representation of the dynamic knowledge subgraph and output the updated initial representation of the dynamic knowledge subgraph, including: To update the representation of the dynamic knowledge subgraph, by adding Aggregate edge information and update the entity node representation of the dynamic knowledge subgraph. The acquisition process is as follows: （9） in, Represents the dynamic knowledge subgraph. The message aggregation representation of each entity node. Represents the dynamic knowledge subgraph. The entity node and the dynamic knowledge subgraph. Attention score among entity nodes Note the weights when representing globally normalized edges; （10） in, Represents the dynamic knowledge subgraph. Layer Each entity node represents, Represents the dynamic knowledge subgraph. Layer Each entity node represents, Represents a node The neighboring nodes, It is a node Connected edges, These are trainable parameters.

5. The commonsense question-answering method based on dynamic global semantic fusion as described in claim 1, characterized in that, The initial representation of the problem context and the updated initial representation of the dynamic knowledge subgraph are fused with word-level local information to obtain a word-level local information fused representation of the problem context and a word-level local information fused representation of the dynamic knowledge subgraph, specifically including: (41): Alignment representation of the initial representation of the problem context and the initial representation of the dynamic knowledge subgraph updated based on cross attention; (42): The aligned representation is fused with the original representations of the initial representation of the problem context and the initial representation of the dynamic knowledge subgraph update through a gating mechanism to obtain a word-level local information fusion representation of the problem and the dynamic knowledge subgraph; Wherein, (41): the alignment representation of obtaining the initial representation of the problem context and updating the initial representation of the dynamic knowledge subgraph based on cross attention specifically includes: Using the problem context Update the initial representation of the dynamic knowledge subgraph To compute the word-level attention matrix between the question and the knowledge subgraph The word-level attention matrix calculation process is as follows: （11） in, These are trainable parameters. Representing the context of the problem No. A vector, Representation of dynamic knowledge subgraphs No. A vector, Representing the context of the problem No. vectors and dynamic knowledge subgraph representation No. A vector, Attention scores between; Then, use The aligned representation of the computation problem is shown in the following calculation process: （12） （13） in, Word-level attention matrix The row vector, This represents a word-level attention matrix with values ​​between [0,1]. The row vector, This represents the number of entity nodes in the dynamic knowledge subgraph. A vector representation of an entity node in a dynamic subgraph. Indicates context alignment of the problem The A vector representation. Represents attention weights on entities in a knowledge subgraph; problem context-aligned representation. This represents the weighted sum of the most critical entity nodes related to the problem in the knowledge subgraph; use The transpose of the expression is used to calculate the aligned representation using the same formulas as in formulas (12) and (13). , This represents the weighted sum of keywords related to the knowledge subgraph within the context of the question; Wherein, (42): by using a gating mechanism, the aligned representation is fused with the original representations of the initial representation of the question context and the initial representation of the dynamic knowledge subgraph update, respectively, to obtain a word-level local information fusion representation of the question and the dynamic knowledge subgraph, specifically including: Using the context of the problem of alignment The dynamic knowledge subgraph of the alignment updates the initial representation. A gate fusion unit was designed to represent the original representation and the corresponding aligned representation; Specifically, dynamic knowledge subgraph representation Update the initial representation with the aligned dynamic knowledge subgraph Integration, expressed as follows: （14） in, This represents a dynamic knowledge subgraph representation. A dynamic knowledge subgraph representation representing alignment. Represents element-matrix multiplication. and Indicates trainable parameters; Gated fusion transmits alignment information, which includes a weight matrix that needs to be learned; Gated fusion function : （15） in, and These are trainable parameters. It is an activation function. This represents a dynamic knowledge subgraph representation. A dynamic knowledge subgraph representation representing alignment; Therefore, the word-level local information fusion representation of dynamic knowledge subgraphs The expression is as follows: （16） in, This represents the fusion of word-level local information in a dynamic knowledge subgraph. This represents a dynamic knowledge subgraph representation. A dynamic knowledge subgraph representation representing alignment; In the context of the problem Perform the same operation to produce the output problem context. .

6. The commonsense question-answering method based on dynamic global semantic fusion as described in claim 1, characterized in that, The word-level local information fusion representation of the problem context and the word-level local information fusion representation of the dynamic knowledge subgraph are subjected to multi-head attention learning to obtain the updated representation of the problem context and the updated representation of the dynamic knowledge subgraph, specifically including: The (51) method, based on two multi-head attention mechanisms of Transformer, learns global semantic information about the problem context and the dynamic knowledge subgraph, respectively, including: Given a problem context representation and the dynamic knowledge subgraph representation obtained from cross-attention fusion As shown by Transformer, the context representation of the mapping problem and dynamic knowledge subgraph representation To query global semantic information in the context of the question Global semantic information of dynamic knowledge subgraphs Keywords , Sum , ; , and Form a group, , and Form another group; each group performs multi-head attention computation and outputs global semantic information of the problem context. and global semantic information of dynamic knowledge subgraphs ; and The calculation process is the same, using The calculation process is used as a demonstration process; The single-head attention calculation process is as follows: （17） in, Represents global semantic information in the context of the problem. It's a scaling factor, which is a numerical value. Representing a single head , , ; The outputs of each attention head in the multi-head attention process are concatenated and subjected to a linear transformation as follows: （18-1） （18-2） in, Represents global semantic information in the context of the problem. Represents global semantic information in the context of the problem. These are trainable parameters. , It is the number of attention heads; after each multi-head attention, a residual connection is used, and then layer normalization is performed; The above (52): Utilizing the exchange of key-value pairs between multi-head attention to fuse global semantic information, achieving the fusion of global semantic information of the problem context and global semantic information of the dynamic knowledge subgraph, specifically includes: In order to achieve the fusion of global semantic information of the problem context and the knowledge subgraph, two multi-head attentions and the exchange of key-value pairs are used to fuse global semantic information, using the output of the previous multi-head attention. and As input, mapping and As a query , Keywords , Sum , ; , and Form a group, , and They were then grouped into another set, each performing interactive computation by exchanging key-value pairs. The result was then calculated by fusing semantic information from the problem context. Global fusion results with dynamic knowledge subgraphs ; The result of fusion of semantic information of the problem context Global fusion results with dynamic knowledge subgraphs The calculation process is the same, with input... To introduce, The calculation method for a single attention head is as follows: （19） in, The result of fusion representing semantic information from the context of the problem. It's a scaling factor, which is a numerical value. , , Representing a single head , , ; The outputs of each attention head in the multi-head attention process are concatenated and subjected to a linear transformation as follows: （20-1） （20-2） in, The result of fusion representing semantic information from the context of the problem. The result of fusion representing semantic information from the context of the problem. Indicates the number of heads of attention; These are trainable parameters. After multi-head attention, a residual connection is used, followed by layer normalization. The (53) states that after each multi-head attention, a feedforward neural network, residual connections, and layer normalization are applied to obtain the problem context update representation and the knowledge subgraph update representation, specifically including: After each multi-head attention layer, a feedforward neural network (MLP) and residual connections are applied for layer normalization to obtain the th... Global semantic commonsense reasoning layer output ; Residual connections help train deep neural networks better; problem context updates representation. The calculation process is as follows: （21） in, Indicates the first Problem context representation in the global semantic commonsense reasoning layer This indicates that the problem context is updated. Update representation in the context of the problem Perform the same operation on top to produce the output of the first... Dynamic knowledge subgraph update representation of the global semantic commonsense reasoning layer ; Finally, the first Output of the global semantic commonsense reasoning layer and It serves as the input for the next layer of global semantic commonsense reasoning.

7. The commonsense question-answering method based on dynamic global semantic fusion as described in claim 1, characterized in that, The update representation of the dynamic knowledge subgraph is pooled to obtain a pooled representation of the dynamic knowledge subgraph. The answer to the commonsense question is obtained based on the pooled representation of the dynamic knowledge subgraph. Specifically, this includes: After N layers of global semantic commonsense reasoning, the final updated version of the question representation is obtained. Knowledge subgraph representation with bidirectional information fusion Calculate the score for a correct answer. as follows: （22） in, yes Mean pooling representation, yes Graph pooling representation, utilizing To standardize all questions - select the correct ones to predict the most reasonable answer.

8. A commonsense question-answering system based on dynamic global semantic fusion, employing the commonsense question-answering method based on dynamic global semantic fusion as described in any one of claims 1-7, characterized in that, include: The acquisition module is configured to acquire common sense questions to be answered and an initial knowledge subgraph. The question-answering module is configured to: input the common sense question to be answered and the initial knowledge subgraph into the trained common sense question-answering model, and output the answer corresponding to the common sense question; The trained commonsense question-answering model is used for: Feature extraction is performed on the common sense questions to be answered to obtain an initial representation of the question context; Based on the correlation between the initial representation of the question context and the entity representation of the initial knowledge subgraph, a soft mask is predicted; based on the soft mask, entity nodes related to common sense questions are identified from the initial knowledge subgraph, and then a dynamic knowledge subgraph is generated. Feature extraction is performed on the dynamic knowledge subgraph to obtain its initial representation; The initial representation of the problem context and the initial representation of the dynamic knowledge subgraph are fused with word-level local information to obtain the word-level local information fused representation of the problem context and the word-level local information fused representation of the dynamic knowledge subgraph. Multi-head attention learning is performed on the word-level local information fusion representation of the problem context and the word-level local information fusion representation of the dynamic knowledge subgraph to obtain the updated representation of the problem context and the updated representation of the dynamic knowledge subgraph. The updated representation of the dynamic knowledge subgraph is pooled to obtain the pooled representation of the dynamic knowledge subgraph, and the answer to the common sense question is obtained based on the pooled representation of the dynamic knowledge subgraph.

9. An electronic device, characterized in that it comprises: Memory is used to store computer-readable instructions in a non-transitory manner. as well as Processor, for executing the computer-readable instructions, When the computer-readable instructions are executed by the processor, they perform the method described in any one of claims 1-7.

10. A storage medium, characterized in that, Non-transitory storage of computer-readable instructions, wherein, when executed by a computer, the instructions of the method according to any one of claims 1-7 are executed.

Images