A causal common sense knowledge base construction method based on graph attention mechanism

Abstract

The application discloses a kind of causal common sense knowledge base construction methods based on graph attention mechanism, comprising the following steps: 1) obtain causal common sense knowledge triples from open knowledge base as training resources;2) using knowledge graph embedding technology, obtain the vector representation of causal common sense knowledge triples;3) combined with background knowledge graph, construct causal common sense knowledge base construction model based on graph attention mechanism, select appropriate loss function to optimize model parameters;4) using causal common sense knowledge base construction model to score the confidence of missing triples, the highest score triples are added to the knowledge base as new learning causal common sense knowledge.The application is based on background knowledge graph, and constructs causal common sense knowledge base construction model using graph attention mechanism, intends to learn new causal common sense knowledge through a small amount of training resources, expands causal common sense knowledge base, so that the knowledge base can better support intelligent question and answer system.

Description

Technical Field

[0001] This invention belongs to the field of natural language processing technology, specifically a method for constructing a causal common sense knowledge base based on graph attention mechanism. Background Technology

[0002] Common-sense knowledge of cause and effect refers to a universal cognitive foundation accumulated by people in daily life. With this knowledge, humans can better understand and adapt to their environment, making rational judgments and decisions. In the field of artificial intelligence, a common-sense knowledge base of cause and effect can provide intelligent question-answering systems with rich background knowledge and reasoning capabilities. Such intelligent question-answering systems based on common-sense cause and effect have potential applications in many fields, including education, healthcare, and scientific research.

[0003] Causal common-sense knowledge can be stored and displayed using knowledge graphs. A knowledge graph is a structured representation of knowledge, presenting the relationships (edges) between entities (nodes) in a graphical form, used to describe the connections between knowledge and concepts in the real world. In a knowledge graph, causal common-sense knowledge is described in the form of triples (head entity, relation, tail entity), forming a vast network. These triples can be linked to form a complex knowledge graph structure. Through triples, knowledge graphs can accurately represent the relationships and attributes between concepts. The triple storage format for common-sense knowledge is concise, scalable, and easy to understand. It enables the structured organization and retrieval of knowledge, providing a foundation for knowledge reasoning and application. Based on the constructed causal common-sense knowledge base, intelligent question-answering systems can better understand user questions, identify causal relationships within questions, deduce answers, and provide explanation and inference capabilities.

[0004] Currently, there are several methods for building knowledge bases: ① Manual construction: This is the most traditional and basic method, where knowledge is collected, organized, and written manually to build the knowledge base. Experts, domain experts, or knowledge workers can manually create knowledge bases by writing documents, rules, and defining concepts. This method is suitable for small-scale knowledge bases or knowledge bases in specific domains. ② Automatic extraction: This method uses Natural Language Processing (NLP) and information extraction techniques to automatically extract knowledge from large-scale text data. This method can use entity recognition, relation extraction, event extraction, and other techniques to automatically identify and extract knowledge. For example, information such as company names, product characteristics, and event occurrence times can be extracted from news articles to build a relevant knowledge base. ③ Semantic Web technology: The Semantic Web is a method for organizing and representing structured knowledge. It uses RDF (Resource Description Framework) as a knowledge representation language to build knowledge bases by defining entities, attributes, and relationships. Semantic Web technology also provides reasoning mechanisms and query languages ​​(such as SPARQL), enabling knowledge bases to have logical reasoning and flexible query capabilities. ④ Machine Learning Methods: This method utilizes machine learning algorithms to learn from large-scale data and build a knowledge base. It can use techniques such as clustering, classification, and topic modeling to automatically discover and organize knowledge. For example, text classification algorithms can be used to identify different topics or domains from news articles and build corresponding knowledge bases.

[0005] Despite the achievements of these methods, constructing a causal commonsense knowledge base still faces the following challenges: First, the completeness of known causal knowledge cannot be guaranteed, as some relations often lack sufficient triples for training. Causal commonsense knowledge data comes from diverse domains and cultural backgrounds, involving a rich variety of entities and relations. However, existing methods fail to fully exploit the attribute features and relational patterns associated with entities within the knowledge, neglecting neighborhood information when modeling semantic structure. Therefore, the semantic information of the obtained entity pairs may be incomplete, especially when the triple data for a particular causal relation is sparse. Failure to fully utilize neighbor information can lead to unreliable vector representations of causal relations, thus limiting the predictive power of the causal commonsense knowledge base construction model. Second, existing methods typically do not consider that each entity has different attribute features, and different features will have different impacts on handling different causal relation tasks. Summary of the Invention

[0006] The present invention addresses the shortcomings of the prior art by proposing a method for constructing a causal common sense knowledge base based on graph attention mechanism. This method aims to learn new causal common sense knowledge with a small amount of training resources, expand the causal common sense knowledge base, and enable the constructed knowledge base to better support intelligent question answering systems.

[0007] To achieve the above-mentioned objectives, the present invention adopts the following technical solution:

[0008] The present invention provides a method for constructing a causal commonsense knowledge base based on graph attention mechanism, characterized by the following steps:

[0009] Step 1: Collect causal common sense knowledge from the knowledge base and represent it in a structured manner, thereby constructing a set of triples of causal common sense knowledge. Where (h,r,t) represents a causal common sense, h represents the head entity, r represents the causal relationship, t represents the tail entity, E is the entity set, and R is the causal relationship set;

[0010] Step 2: Collect the high-frequency causal relationships associated with the triples in the causal common sense knowledge triple set TP, and use prior knowledge to store and model the high-frequency triples in the form of a directed graph, thereby constructing a background knowledge graph G′, where the head entity and tail entity of the triples are respectively the nodes of the background knowledge graph G′, and the causal relationships of the triples are the edges between the nodes in the background knowledge graph G′.

[0011] Step 3: Use a knowledge graph embedding method to represent each node and edge in the background knowledge graph G′ as a vector, thereby obtaining the vector representation V of the head entity node. h The vector representation V of the tail entity node t Vector representation of causal relationship edges V r ;

[0012] Step 4: Randomly select a causal relationship, denoted as r, from the set of causal common-sense knowledge triples TP. Select the partial triples corresponding to causal relationship r to construct the support set S of causal relationship r. r Using the remaining triples of causal relation r, construct the query set Q of causal relation r. r Then query set Q r All tail entities are replaced with entities that have no causal relationship with their head entities, thus constructing a negative query set.

[0013] Step 5: Develop the support set S corresponding to the causal relationship r. r and query set Q r Composition of training task T r ={S r Q r}; Select other causal relationships r′ from the causal relationship set R, and construct the support set S of causal relationship r′ according to the process in step four. r′ and query set Q r′ Composed of test task T r′ ={S r′ Qr′};

[0014] Step Six: Construct a feature vector learning module using a graph attention mechanism to support the set S. r Each entity pair in the vector representation is generated;

[0015] Step 6.1: Extract useful information from the background knowledge graph G′ to obtain the information set of the i-th neighbor corresponding to the causal relationship r. Among them, (h i ,t i S represents the support set S for the causal relationship r. r The i-th head entity-tail entity pair in the middle, This indicates the extraction of the i-th head entity h from the background knowledge graph G′. i The corresponding direct neighbor information, This indicates that the i-th head entity h is extracted from the background knowledge graph G′. i The k-th direct neighbor entity Related neighbor information, i.e., indirect neighbor information. Represents the tail entity t i Direct neighbor information, Represents the tail entity t i Indirect neighbor information;

[0016] Step 6.2: The feature vector learning module learns causal common sense triples (h) i ,r,t i The i-th entity pair (h) in ) i ,t i The vector representation of ) is denoted as

[0017] Step 7: For query set Q r The confidence level of the causal common sense knowledge triplet in the text is scored;

[0018] Step 7.1, for query set Q r Let q be any query triple in the set. r The vector representation P of the current causal relationship r is obtained by using equations (10)-(11). r :

[0019]

[0020]

[0021] In equations (10)-(11), · represents the dot product operation. The query triple q for causal relation r r The vector representation of , The support set S of the causal relationship r is obtained by learning through equation (9). r The i-th head entity-tail entity pair (h) i ,t i The vector representation of ) For the i-th entity pair (h) i ,t i ) represents the corresponding attention weight;

[0022] Step 7.2: Evaluate the query set Q using the metric function shown in equation (12). r Query knowledge triple q r confidence score d(q) r ,S r ):

[0023]

[0024] Step 7.3, for negative query sets Any negative query triple in the set is denoted as Following steps 7.1-7.2, calculate the confidence score of the negative query triplet.

[0025] Step 8: Optimize the graph attention network that generates the vector representation of the causal relationship r using the backpropagation algorithm, and calculate the loss function L of equation (13). When the loss function L continuously decreases until convergence, the optimal causal commonsense knowledge base construction model is obtained.

[0026]

[0027] In equation (13), γ is a hyperparameter;

[0028] Step 9: Input several candidate entities into the optimal causal common sense knowledge base construction model, calculate the confidence score of the corresponding query triples, and take the triple with the highest confidence score as the new causal common sense knowledge learned by the model.

[0029] The method for constructing a causal commonsense knowledge base based on graph attention mechanism described in this invention is also characterized in that step 6.1 is performed as follows:

[0030] Step 6.1.1: For the support set S of causal relationship r r The i-th head entity-tail entity pair (h) i ,t i Extract the i-th head entity h from the background knowledge graph G′. i The corresponding direct neighbor information in, Represents the i-th head entity h iA direct neighbor relationship; Represents the i-th head entity h i A direct neighbor entity; Represents the i-th head entity h i The number of direct neighbors associated with it. Represents the i-th head entity h i The associated direct neighbor causal relationship - entity pair;

[0031] Step 6.1.2: Extract the i-th head entity h from the background knowledge graph G′. i The k-th direct neighbor entity Related neighbor information is denoted as indirect neighbor information. in, Represents the i-th head entity h i An indirect neighbor relationship; Represents the i-th head entity h i An indirect neighbor entity; Represents the i-th head entity h i The number of indirect neighbors associated with it. Represents the i-th head entity h i The nth indirect neighbor relationship-entity pair;

[0032] Step 6.1.3, the direct neighbor information and indirect neighbor information Together they form the i-th head entity h i Neighbor information set

[0033] Step 6.1.4: Obtain the tail entity t following the process of steps 6.1.1-6.1.3. i Direct neighbor information Indirect Neighbor Information and neighbor information set

[0034] Step 6.1.5: Obtain the information set of the i-th neighbor corresponding to the causal relationship r.

[0035] Step 6.2 is performed as follows:

[0036] Step 6.2.1: Using equation (1), model the causal relationship r of the current self-learning task of causal common sense knowledge as the i-th head entity node h. i vector representation The i-th tail entity node t i vector representation Translation vectors between

[0037]

[0038] Step 6.2.2: Calculate the causal common sense knowledge triple (h) using formula (2). i ,r,t i The i-th head entity h in ) i direct neighbors Correlation between causal relationship r

[0039]

[0040] In equation (2), || represents vector concatenation, T is the transpose, and U1 and W1 are two weight matrices. direct neighbor entities The vector representation of , direct neighbor relationship Vector representation of;

[0041] Step 6.2.3: Calculate the causal common sense knowledge triple (h) using formula (3). i ,r,t i The i-th head entity h in ) i indirect neighbors Correlation between causal relationship r

[0042]

[0043] In equation (3), U2 and W2 are two weight matrices. Indirect neighbor entities The vector representation of , Indirect neighbor relationship Vector representation of;

[0044] Step 6.2.4: Aggregate direct and indirect neighbor information with different attention weights using equations (4)-(7), and obtain the i-th head entity h using equation (8). i Augmented vector representation

[0045]

[0046]

[0047]

[0048]

[0049]

[0050] In equations (4)-(8), σ is the activation function. This indicates that direct neighbor relationships will be represented. and direct neighbor entity representation The concatenated vector representation, This indicates that indirect neighbor relationships are represented. and indirect neighbor entity representation The concatenated vector representation, W, W nbr W self There are three weight matrices, where b is the bias. The i-th head entity h represents the aggregated direct neighbor information. i Enhanced vector representation, The i-th head entity h represents the aggregated indirect neighbor information. i The enhanced vector representation, g(·) is the gate mechanism;

[0051] Step 6.2.5: Obtain the i-th tail entity t following the process of steps 6.2.2-6.2.4. i Augmented vector representation

[0052] Step 6.2.6: Use equation (9) to represent the enhancement vector of the i-th head and tail entities. Connect them to generate a causal relationship r corresponding to the i-th entity pair (h) i ,t i The final vector representation of ) is used as the causal commonsense knowledge triple (h) learned by the feature vector learning module. i ,r,t i The i-th entity pair (h) in ) i ,t i The vector representation of ) is denoted as

[0053]

[0054] In equation (9), Given two weight matrices, b z This is a deviation.

[0055] The present invention provides an electronic device, including a memory and a processor, wherein the memory is used to store a program that supports the processor in executing the causal common sense knowledge base construction method, and the processor is configured to execute the program stored in the memory.

[0056] The present invention discloses a computer-readable storage medium on which a computer program is stored, wherein the computer program, when executed by a processor, performs the steps of the causal common sense knowledge base construction method.

[0057] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0058] 1. This invention fully explores the relationship structure and semantic information between entities by constructing a background knowledge graph, captures richer features from the direct and indirect neighbor information of causal common sense knowledge triples, enhances the vector representation of entity pairs, and uses the enhanced reference entity pair representation to generate a more reliable prototype representation of causal relationship, thereby improving the accuracy of the causal common sense knowledge base construction.

[0059] 2. This invention uses a graph neural network based on an attention mechanism to fuse rich semantic and relational information in the knowledge graph. It can simultaneously model information such as entities, entity attributes, and entity relationships involved in causal knowledge, adaptively calculate the influence weight of different entity features on the current causal relationship task, and integrate the information into the vector representation of the relationship. This allows for a more effective comparison of the similarity between query triples and relation prototypes, thereby improving the quality of knowledge base construction.

[0060] 3. This invention aggregates richer entity and neighbor information to learn vector representations of common sense relationships, and considers the different influences of entity features under different task relationships, which can increase the accuracy of causal common sense knowledge base construction.

[0061] 4. A causal common-sense knowledge base can support an intelligent question-answering system. The system can receive questions from users and search the knowledge base to find relevant causal relationships and knowledge. This invention uses a graph attention mechanism, allowing the system to accurately find the most relevant answer or explanation based on the semantics and context of the question. Attached Figure Description

[0062] Figure 1 This is a flowchart of the present invention. Detailed Implementation

[0063] In this embodiment, a method for constructing a causal commonsense knowledge base based on graph attention mechanism involves obtaining causal commonsense knowledge triples from an existing knowledge base as training resources. A neural network based on graph attention mechanism learns vector representations of causal relationships from corresponding reference triples. The causal relationships, head and tail entities, and their neighbors are projected into a latent semantic vector space. The vector representations of head and tail entities and their neighbors are aggregated through feature propagation and weighted combination in the graph attention network, thereby generating causal relationship vector representations that incorporate rich semantic information. This makes the causal commonsense knowledge base construction model more accurate. The confidence of the triples is measured by calculating the feature similarity between the causal relationship vector representation and the query triples. Specifically, for example... Figure 1 As shown, the procedure is as follows:

[0064] Step 1: Collect causal common sense knowledge from the knowledge base and represent it in a structured manner, thereby constructing a set of triples of causal common sense knowledge. Where (h,r,t) represents a causal common sense, h represents the head entity, r represents the causal relationship, t represents the tail entity, E is the set of entities, and R is the set of causal relationships;

[0065] Taking the common sense knowledge triple of cause and effect (lung cancer, due to, smoking addiction) as an example, "lung cancer" is the head entity, "due to" is the causal relationship, and "smoking addiction" is the tail entity.

[0066] Step 2: Collect the high-frequency causal relationships associated with the causal common sense knowledge triples set TP, and use prior knowledge to store and model the high-frequency triples in the form of a directed graph to construct a background knowledge graph G′. In this graph, the head entity and tail entity of the triple are the nodes of the background knowledge graph G′, and the causal relationship of the triple is the edge between the nodes in the background knowledge graph G′. The background knowledge graph G′ provides rich neighbor semantic information to enhance the vector representation of the head and tail entity pairs of the causal relationship.

[0067] Step 3: Use the knowledge graph embedding method TransE to represent each node and edge in the background knowledge graph G′ as a vector, thereby obtaining the vector representation V of the head entity node. h The vector representation V of the tail entity node t Vector representation of causal relationship edges V r ;

[0068] Step 4: Randomly select a causal relationship, denoted as r, from the set of causal common-sense knowledge triples TP. Select the partial triples corresponding to causal relationship r to construct the support set S of causal relationship r. r Using the remaining triples of causal relation r, construct the query set Q of causal relation r. r Then query set Q r All tail entities are replaced with entities that have no causal relationship with their head entities, thus constructing a negative query set.

[0069] Taking the extraction of the causal relationship "due to" as an example, (lung cancer, due to, smoking addiction) can be used as a reference triple of the current causal relationship, (lung cancer, due to, air pollution) can be used as a query triple of the current causal relationship, and (lung cancer, due to, crustal movement) can be used as a pseudo triple of the current causal relationship.

[0070] Step 5: Develop the support set S corresponding to the causal relationship r. r and query set Q r Composition of training task T r ={S r Qr}; Select other causal relationships r′ from the causal relationship set R, and construct the support set S of causal relationship r′ according to the process in step four. r′ and query set Q r′ Composed of test task T r′ ={S r′ Q r′};

[0071] The causal relationship r′ used in the current test task is a causal relationship other than "due to", such as "caused", "caused", "therefore" etc.

[0072] Step Six: Construct a feature vector learning module using a graph attention mechanism. This module integrates rich neighbor semantic information from the background knowledge graph, enabling simultaneous modeling of the head and tail entity nodes of causal relationship r, the direct neighbor information of entity nodes, and the indirect neighbor information of entity nodes. It adaptively calculates the attention weights between different semantic information and causal relationship r, aggregates vector representations with different attention weights, and finally learns the support set S. r Each entity in the vector is represented by an enhanced vector;

[0073] Step 6.1: Extract useful information from the background knowledge graph G′:

[0074] Step 6.1.1: For the support set S of causal relationship r r The i-th head entity-tail entity pair (h) i ,t i Extract the i-th head entity h from the background knowledge graph G′. i The corresponding direct neighbor information in, Represents the i-th head entity h i A direct neighbor relationship; Represents the i-th head entity h i A direct neighbor entity; Represents the i-th head entity h i The number of direct neighbors associated with it. Represents the i-th head entity h i The associated direct neighbor causal relationship - entity pair;

[0075] For example, for the causal relationship "due to" in the head-tail entity pair (lung cancer, smoking addiction) in the support set, there is a direct neighbor causal relationship-entity pair (based on, carcinogen) associated with the head entity "lung cancer" in the background knowledge graph;

[0076] Step 6.1.2: Extract the i-th head entity h from the background knowledge graph G′. i The k-th direct neighbor entity Related neighbor information is denoted as indirect neighbor information. in, Represents the i-th head entity h i An indirect neighbor relationship; Represents the i-th head entity h i An indirect neighbor entity; Represents the i-th head entity h i The number of indirect neighbors associated with it. Represents the i-th head entity h i The nth indirect neighbor relationship-entity pair;

[0077] Taking the direct neighbor of the head entity "lung cancer" in (lung cancer, due to, smoking addiction) (lung cancer, based on, carcinogen) as an example, there is an indirect neighbor causal relationship associated with the head entity "lung cancer" in the background knowledge graph - entity pair (caused by, cigarettes);

[0078] Step 6.1.3, the direct neighbor information and indirect neighbor information Together they form the i-th head entity h i Neighbor information set

[0079] Both the direct neighbor entity node "carcinogen" and the indirect neighbor entity node "cigarette" have semantic relevance to the reference triple (lung cancer, due to, nicotine addiction) under the current causal relationship, thus utilizing neighbor information to enhance the vector representation of head-tail entity pairs;

[0080] Step 6.1.4: Obtain the tail entity t following the process of steps 6.1.1-6.1.3. i Direct neighbor information Indirect Neighbor Information and neighbor information set

[0081] Step 6.1.5: Obtain the information set of the i-th neighbor corresponding to the causal relationship r.

[0082] Step 6.2: The feature vector learning module learns the vector representations of entity pairs in causal commonsense triples.

[0083] Step 6.2.1: Using equation (1), model the causal relationship r of the current self-learning task of causal common sense knowledge as the i-th head entity node h. i vector representation The i-th tail entity node t i vector representation Translation vectors between

[0084]

[0085] Step 6.2.2: Calculate the causal common sense knowledge triple (h) using formula (2). i ,r,t i The i-th head entity h in ) i direct neighbors Correlation between causal relationship r

[0086]

[0087] In equation (2), || represents vector concatenation, T is the transpose, and U1 and W1 are two weight matrices. direct neighbor entities The vector representation of , direct neighbor relationship Vector representation of;

[0088] To aggregate different neighbor information, an attention mechanism is used to calculate the relevance weight between different neighbor information and causal relationships - entities; the greater the relevance weight, the greater the influence of the corresponding neighbor information on the finally learned entity pair vector representation;

[0089] Step 6.2.3: Calculate the causal common sense knowledge triple (h) using formula (3). i ,r,t i The i-th head entity h in ) i indirect neighbors Correlation between causal relationship r

[0090]

[0091] In equation (3), U2 and W2 are two weight matrices. Indirect neighbor entities The vector representation of , Indirect neighbor relationship Vector representation of;

[0092] Step 6.2.4: Aggregate direct and indirect neighbor information with different attention weights using equations (4)-(7), and obtain the i-th head entity h using equation (8). i Augmented vector representation

[0093]

[0094]

[0095]

[0096]

[0097]

[0098] In equations (4)-(8), σ is the activation function. This indicates that a feedforward neural network is used to represent direct neighbor relationships. and direct neighbor entity representation The concatenated vector representation, This indicates that indirect neighbor relationships are represented using a feedforward neural network. and indirect neighbor entity representation The concatenated vector representation, W, W nbr W self There are three weight matrices, where b is the bias. The i-th head entity h represents the aggregated direct neighbor information. i Enhanced vector representation, The i-th head entity h represents the aggregated indirect neighbor information. i The augmented vector representation is obtained by g(·), which is a gate mechanism that can adaptively aggregate vector representations from direct and indirect neighbors.

[0099] Step 6.2.5: Obtain the i-th tail entity t following the process of steps 6.2.2-6.2.4. i Augmented vector representation

[0100] Step 6.2.6: Use equation (9) to represent the enhancement vector of the i-th head and tail entities. Connect them to generate a causal relationship r corresponding to the i-th entity pair (h) i ,t i The final vector representation of ) is used as the causal commonsense knowledge triple (h) learned by the feature vector learning module. i ,r,t i The i-th entity pair (h) in ) i ,t i The vector representation of ) is denoted as

[0101]

[0102] In equation (9), Given two weight matrices, b z For deviation;

[0103] Step 7: For query set Q r The confidence level of the causal common sense knowledge triplet in the text is scored;

[0104] Step 7.1, for query set Q r Let q be any query triple in the set. r By aggregating reference entity pairs with different attention weights, the vector representation P of the current causal relationship r is obtained using equations (10)-(11). r :

[0105]

[0106]

[0107] In equations (10)-(11), · represents the dot product operation. The query triple q for causal relation r r The vector representation of , The support set S of the causal relationship r is obtained by learning through equation (9). r The i-th head entity-tail entity pair (h) i ,t i The vector representation of ) For the i-th entity pair (h) i ,t i ) represents the corresponding attention weight;

[0108] An attention mechanism is used to encode reference triples and query triples in an interactive manner. Higher attention weights are assigned to reference entity pairs that are more relevant to the query triples, which helps to improve the accuracy of query prediction.

[0109] Step 7.2: Evaluate the query knowledge triple q in the query set using the metric function shown in equation (12). r confidence score d(q) r ,S r ):

[0110]

[0111] In equation (12), P r The vector representation of the causal relationship r generated by aggregating reference entity pairs with different attention weights in equation (11); the confidence score d(q) r ,S r The higher the value, the more likely the query triple is to be a fact triple.

[0112] Step 7.3, for negative query sets Any negative query triple in the set is denoted as Following steps 7.1-7.2, calculate the confidence score of the negative query triplet.

[0113] Step 8: Optimize the graph attention network that generates the vector representation of the causal relationship r using the backpropagation algorithm, and calculate the loss function L of equation (13). Minimize the loss function L based on the meta-training procedure of batch sampling. When the loss function L continuously decreases until convergence, the optimal causal commonsense knowledge base construction model is obtained.

[0114]

[0115] In equation (13), γ>0 is the marginal hyperparameter separating positive and negative examples;

[0116] Step 9: Input several candidate entities into the optimal causal common sense knowledge base construction model, calculate the confidence score of the corresponding query triples, and take the triple with the highest confidence score as the new causal common sense knowledge learned by the model.

[0117] In this embodiment, an electronic device includes a memory and a processor. The memory stores a program that supports the processor in executing the above-described method, and the processor is configured to execute the program stored in the memory.

[0118] In this embodiment, a computer-readable storage medium stores a computer program, which is executed by a processor to perform the steps of the above method.

Claims

1. A method for constructing a causal commonsense knowledge base based on graph attention mechanism, characterized in that, Follow these steps: Step 1: Collect causal common sense knowledge from the knowledge base and represent it in a structured manner, thereby constructing a set of triples of causal common sense knowledge. ,in, This represents a basic principle of cause and effect. Indicates the head entity. Indicates a cause-and-effect relationship. Indicates the tail entity. For a collection of entities, It is a set of causal relationships; Step Two: Collect a set of causal common sense knowledge triplets The high-frequency causal relationships associated with triples are identified, and prior knowledge is used to store and model these high-frequency triples in the form of a directed graph, thereby constructing a background knowledge graph. In this context, the head entity and tail entity of the triple serve as the background knowledge graph. The nodes and causal relationships of triples serve as the background knowledge graph. Edges between nodes; Step 3: Employ a knowledge graph embedding method to embed the background knowledge graph. Each node and edge in the vector representation is used to obtain the vector representation of the head entity node. Vector representation of tail entity nodes Vector representation of causal relationship edges ; Step 4: From the set of causal common sense knowledge triples A causal relationship is randomly selected from the data and denoted as . Selecting causal relationships The corresponding partial triples are used to construct causal relationships. support set Using causal relationships The remaining triples establish causal relationships. query set Then the query set All tail entities are replaced with entities that have no causal relationship with their head entities, thus constructing a negative query set. ; Step 5: From causal relationships Corresponding support set and query set Composition of training tasks From the set of causal relationships Select other causal relationships And construct causal relationships according to the process in step four. support set and query set To form a test task ; Step Six: Construct a feature vector learning module using a graph attention mechanism to support the set. Each entity pair in the vector representation is generated; Step 6.1: Extract background knowledge graph Useful information to obtain causal relationships The corresponding number Neighbor information set ;in, Indicates causal relationship support set The Middle Head entity - tail entity pair Indicates from background knowledge graph Extract the first Entity The corresponding direct neighbor information, Indicates from background knowledge graph Extracting from the first Entity The direct neighbor entities Related neighbor information, i.e., indirect neighbor information. Represents tail entity Direct neighbor information, Represents tail entity Indirect neighbor information; Step 6.1.1, Regarding causal relationships support set The Middle Head entity - tail entity pair From background knowledge graph Extract the first Entity The corresponding direct neighbor information ,in, Indicates the first Entity A direct neighbor relationship; Indicates the first Entity A direct neighbor entity; Indicates the first Entity The number of direct neighbors associated with it. Indicates the first Entity The associated direct neighbor causal relationship - entity pair; Step 6.1.2: From the background knowledge graph Extracting from the first Entity The direct neighbor entities Related neighbor information is denoted as indirect neighbor information. ,in, Indicates the first Entity An indirect neighbor relationship; Indicates the first Entity An indirect neighbor entity; Indicates the first Entity The number of indirect neighbors associated with it. Indicates the first Entity The associated first An indirect neighbor relationship - entity pair; Step 6.1.3, the direct neighbor information and indirect neighbor information Together they form the first Entity Neighbor information set ; Step 6.1.4: Obtain the tail entity following the process from Step 6.1.1 to Step 6.1.

3. Direct neighbor information Indirect neighbor information and neighbor information set ; Step 6.1.5: Obtain the causal relationship The corresponding number Neighbor information set ; Step 6.2: The feature vector learning module learns causal common sense triples. The Middle Entity pair The vector representation of is denoted as ; Step 7: Process the query set The confidence level of the causal common sense knowledge triplet in the text is scored; Step 7.1, for the query set Any query triple in the set is denoted as The current causal relationship is obtained using equations (10) and (11). vector representation : (10) (11) In equations (10)-(11), For dot product operation, For causal relationship query triples The vector representation of , Causal relationships are learned through equation (9) support set The Middle Head entity - tail entity pair The vector representation of , For the first Entity pair This represents the corresponding attention weight; Step 7.2: Evaluate the query set using the metric function shown in equation (12). Query knowledge triples confidence score : (12) Step 7.3, for negative query sets Any negative query triple in the set is denoted as Following steps 7.1-7.2, calculate the confidence score of the negative query triplet. ; Step 8: Use the backpropagation algorithm to generate causal relationships. The graph attention network represented by the vector is optimized, and the loss function of equation (13) is calculated. When the loss function When the values ​​continue to decrease until convergence, the optimal causal commonsense knowledge base construction model is obtained. (13) In equation (13), For hyperparameters; Step 9: Input several candidate entities into the optimal causal common sense knowledge base construction model, calculate the confidence score of the corresponding query triples, and take the triple with the highest confidence score as the new causal common sense knowledge learned by the model.

2. The method for constructing a causal commonsense knowledge base based on graph attention mechanism according to claim 1, characterized in that, Step 6.2 is performed as follows: Step 6.2.1: Use equation (1) to establish the causal relationship in the current task of self-directed learning of causal common sense knowledge. Modeling as the first Individual entity node vector representation , No. Tail entity node vector representation Translation vectors between : (1) Step 6.2.2: Calculate the causal common sense knowledge triplet using formula (2). The Middle Entity direct neighbors With causality Correlation between : (2) In equation (2), Connect vectors. For transpose, and There are two weight matrices. direct neighbor entities The vector representation of , direct neighbor relationship Vector representation of; Step 6.2.3: Calculate the causal common sense knowledge triplet using formula (3). The Middle Entity indirect neighbors With causality Correlation between : (3) In equation (3), and There are two weight matrices. Indirect neighbor entities The vector representation of , Indirect neighbor relationship Vector representation of; Step 6.2.4: Aggregate direct and indirect neighbor information with different attention weights using equations (4)-(7), and obtain equation (8) to obtain the first... Entity Augmented vector representation : (4) (5) (6) (7) (8) In equations (4)-(8), For activation function, This indicates that direct neighbor relationships will be represented. and direct neighbor entity representation The concatenated vector representation, This indicates that indirect neighbor relationships are represented. and indirect neighbor entity representation The concatenated vector representation, , , There are three weight matrices. For deviation, The first one represents the aggregation of direct neighbor information. Entity Enhanced vector representation, The first indirect neighbor information aggregation Entity Enhanced vector representation, For gate mechanism; Step 6.2.5: Following the process of steps 6.2.2-6.2.4, the following steps are obtained: Tail Entity Augmented vector representation ; Step 6.2.6: Use equation (9) to... Augmented vector representation of head and tail entities , Connect them to form a causal relationship. Corresponding to the Entity pair The final vector representation of the causal commonsense knowledge triplet learned by the feature vector learning module is used as the final vector representation of the causal commonsense knowledge triplet. The Middle Entity pair The vector representation of is denoted as ; (9) In equation (9), , There are two weight matrices. This is a deviation.

3. An electronic device, comprising a memory and a processor, characterized in that, The memory is used to store programs that support the processor in executing the causal common sense knowledge base construction method of any one of claims 1-2, and the processor is configured to execute the programs stored in the memory.

4. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is run by the processor, it executes the steps of the causal common sense knowledge base construction method according to any one of claims 1-2.

Images