A method for completing a logical graph based on reinforcement learning and related equipment

Abstract

The embodiment of the present application belongs to the field of knowledge graphs and relates to a method for completing a causal graph based on reinforcement learning and related equipment, including receiving event corpus, performing event entity recognition operation on the event corpus, obtaining event entities, performing event relationship recognition operation on the event training corpus, obtaining event relationships, and constructing a causal graph based on event entities and event relationships; inputting each entity node and edge node in the causal graph into the Bert model respectively, obtaining entity embedding vectors and edge embedding vectors respectively; taking any entity node in the causal graph as the starting node, calculating the state vector of each step of reasoning of the starting node in the causal graph based on the entity embedding vector and the edge embedding vector; performing step-by-step reasoning in the causal graph based on the state vector and the pre-trained reasoning model, obtaining the target causal relationship, and completing the causal graph according to the target causal relationship. The present application improves the efficiency and accuracy of completing the causal graph.

Description

Technical Field

[0001] The present application relates to the field of knowledge graph technology, and in particular to a method for completing a knowledge graph based on reinforcement learning and related equipment. Background Art

[0002] In recent years, knowledge graphs have been widely used in fields such as finance, e-commerce, and healthcare. However, traditional knowledge graphs also have limitations. First, the knowledge in knowledge graphs is static and fixed, while knowledge in reality is dynamic and evolving. Second, knowledge graphs can only answer questions such as "What is XX?" but struggle to answer questions such as "Why" and "How?" This is because entities lack a temporal, logical relationship between them. An emerging type of knowledge graph, the causal graph, studies predicate events and the logical relationships between them over time. Therefore, the causal graph is a new tool that addresses the shortcomings of traditional knowledge graphs.

[0003] Due to the complexity of the real world, building a causal graph requires a lot of effort, and there may still be problems such as missing logical relationships between entities. Therefore, knowledge completion (or reasoning) is also needed in practical applications. Knowledge completion refers to the triples that need to be completed in a given causal graph.<eh,r,?> , it is necessary to search in the entire event graph, and after passing through several intermediate entities, finally find the tail entity t that meets the requirements of the relationship r, thus forming a complete triple<eh,r,et> The entire search sequence can be summarized as: {(eh,r1,e1),...,(en,rn,et)} (n represents the nth entity in the search sequence). For example, if the following sequences exist: <Rising Brazilian iron ore prices lead to rising Chinese steel prices>, <Rising Chinese steel prices lead to rising domestic car prices>, <Rising domestic car prices, while residents' incomes have not increased>, <Residents' incomes have not increased, while overtime hours have increased>, <Increased overtime hours lead to an increase in subway ridership at night>, then the missing relationship can be found: <Rising Brazilian iron ore prices lead to an increase in subway ridership at night>.

[0004] However, the existing knowledge completion method of the causal graph has large errors in feature extraction, resulting in low accuracy in the final judgment of the relationship between events and poor completeness of the causal graph completion. Summary of the Invention

[0005] The purpose of the embodiments of the present application is to propose a method for completing a causal graph based on reinforcement learning and related equipment, which improves the efficiency and accuracy of completing the causal graph.

[0006] In order to solve the above technical problems, the present application provides a method for completing a causal graph based on reinforcement learning, which adopts the following technical solutions:

[0007] A method for completing a causal graph based on reinforcement learning includes the following steps:

[0008] receiving event corpus, performing event entity recognition on the event corpus to obtain event entities, performing event relationship recognition on the event corpus to obtain event relationships, and constructing an event graph based on the event entities and event relationships;

[0009] Calling the pre-trained Bert model, inputting each entity node and edge node in the event graph into the pre-trained Bert model, respectively, to obtain the entity embedding vector and edge embedding vector respectively;

[0010] Taking any entity node in the event graph as a starting node, and calculating a state vector of each step of reasoning of the starting node in the event graph based on the entity embedding vector and the edge embedding vector;

[0011] Based on the state vector and the pre-trained reasoning model, step-by-step reasoning is performed in the transaction graph to obtain the target transaction relationship between the two entity nodes, and the transaction graph is completed according to the target transaction relationship.

[0012] Furthermore, the step of performing an event relationship recognition operation on the event corpus to obtain event relationships includes:

[0013] receiving a plurality of relational dictionaries, and classifying sentences in the event training corpus based on the relational dictionaries to determine relational dictionaries corresponding to the sentences, wherein the relational dictionaries include a plurality of relation names;

[0014] Identify the part of speech of each word in the sentence and determine whether there is only one verb in the sentence;

[0015] If so, determining whether the word with the verb part of speech exists in the relationship dictionary, and if so, taking the word with the verb part of speech as the event relationship corresponding to the event entity;

[0016] If there is not only one verb, a relationship name is selected from the relationship dictionary corresponding to the sentence in a preset manner as the event relationship corresponding to the event entity.

[0017] Furthermore, the step of selecting a relationship name from the relationship dictionary corresponding to the sentence in a preset manner as the event relationship corresponding to the event entity includes:

[0018] Inputting the words whose part of speech is verb in the sentence into the pre-trained word vector conversion model respectively, obtaining the output first word embedding vectors respectively, and calculating the average value of the first word embedding vectors to obtain the average vector;

[0019] Inputting the relationship names of the relationship dictionary into the word vector conversion model respectively, and obtaining output second word embedding vectors respectively;

[0020] The semantic similarity between the average vector and the second word embedding vector is calculated, and the relationship name corresponding to the maximum semantic similarity is used as the event relationship.

[0021] Furthermore, the step of calculating the state vector of each reasoning step of the central node in the event graph based on the entity embedding vector and the edge embedding vector includes:

[0022] Calculating the semantic distance between the entity nodes based on the entity embedding vector and the edge embedding vector to obtain the entity semantic distance;

[0023] In the event graph, a first region of interest is formed with the starting node as the center and a first semantic distance preset as the radius, and an entity node within the first region of interest is determined based on the entity semantic distance as a first neighbor node of the starting node;

[0024] With the first neighbor node as the center and a second semantic distance as the radius, a second region of interest is formed, the entity node in the second region of interest is used as the second neighbor node of the first neighbor node, and the second neighbor node connected to the first neighbor node in the event graph is used as the target neighbor node;

[0025] The state vector of the start node is calculated based on an edge embedding vector of an edge between the target neighbor node and the first neighbor node and an entity embedding vector of the first neighbor node.

[0026] Furthermore, the step of calculating the state vector of the start node based on the edge embedding vector of the edge between the target neighbor node and the first neighbor node and the entity embedding vector of the first neighbor node includes:

[0027] The state vector of the starting node in the t-step reasoning is calculated based on the following formula:

[0028] in, represents the state vector of the starting node at step t, “:” represents the vector concatenation operation, represents the entity embedding vector of the first neighbor node reached at step t from the starting node, represents the sum of the edge embedding vectors from the starting node to the first neighbor node at the tth step, J represents the number of the target neighbor nodes corresponding to the first neighbor node at the tth step, It represents the sum of the edge embedding vectors of the edges passing from the target neighbor node corresponding to the first neighbor node in the t-th step to the first neighbor node in the t-th step.

[0029] Furthermore, the step of performing step-by-step reasoning in the transaction graph based on the state vector and the pre-trained reasoning model to obtain the target transaction relationship between the two entity nodes includes:

[0030] Inputting the state vector corresponding to the starting node at each step of inference into the pre-trained inference model, the inference model processes the state vector through a first hidden layer, outputs a first probability sequence, processes the first probability sequence through a second hidden layer, and outputs a second probability sequence;

[0031] Processing the second probability sequence through a softmax layer to obtain a third probability sequence;

[0032] Taking the action corresponding to the maximum probability value in the third probability sequence as the inference action, and performing an inference to obtain an inference node and an inference relationship;

[0033] After multiple reasoning actions, until it is inferred that the starting node is the same as the reasoning node, the reasoning relationship is used as the target logical relationship.

[0034] Furthermore, after the step of constructing a cause-and-effect graph based on the event entities and event relationships, the method further includes:

[0035] Extracting training data from the event graph, wherein the training data is a sequence consisting of entities and relationships;

[0036] Reward-based reinforcement learning is performed on the preset inference model based on the training data until a preset stopping condition is reached, thereby obtaining the pre-trained inference model.

[0037] In order to solve the above technical problems, the embodiment of the present application further provides a device for completing a causal graph based on reinforcement learning, which adopts the following technical solution:

[0038] A reinforcement learning-based event graph completion device, comprising:

[0039] A receiving module is configured to receive event corpus, perform event entity recognition on the event corpus to obtain event entities, perform event relationship recognition on the event corpus to obtain event relationships, and construct an event graph based on the event entities and event relationships;

[0040] A first computing module is configured to call a pre-trained Bert model, input each entity node and edge node in the event graph into the pre-trained Bert model, and obtain an entity embedding vector and an edge embedding vector, respectively;

[0041] A second calculation module is configured to take any entity node in the event graph as a starting node and calculate a state vector of each step of reasoning of the starting node in the event graph based on the entity embedding vector and the edge embedding vector;

[0042] An inference module is used to perform step-by-step reasoning in the transaction graph based on the state vector and a pre-trained inference model, obtain a target transaction relationship between two entity nodes, and complete the transaction graph according to the target transaction relationship.

[0043] In order to solve the above technical problems, the embodiment of the present application further provides a computer device, which adopts the following technical solution:

[0044] A computer device includes a memory and a processor, wherein the memory stores computer-readable instructions, and when the processor executes the computer-readable instructions, it implements the steps of the above-mentioned reinforcement learning-based causal graph completion method.

[0045] In order to solve the above technical problems, the embodiment of the present application further provides a computer-readable storage medium, which adopts the following technical solution:

[0046] A computer-readable storage medium having computer-readable instructions stored thereon, wherein the computer-readable instructions, when executed by a processor, implement the steps of the above-mentioned reinforcement learning-based causal graph completion method.

[0047] Compared with the prior art, the embodiments of the present application have the following beneficial effects:

[0048] In the process of completing the causal graph, this application uses state vectors for reasoning and considers the next action. The state vector is calculated based on the entity embedding vector and the edge embedding vector, so that the reasoning of each entity node is not only based on its own perspective. The entity embedding vector and the edge embedding vector are combined to find the optimal path, thereby improving the efficiency and accuracy of completing the causal graph. BRIEF DESCRIPTION OF THE DRAWINGS

[0049] In order to more clearly illustrate the solutions in this application, a brief introduction will be given below to the drawings required for use in the description of the embodiments of this application. Obviously, the drawings described below are some embodiments of this application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0050] Figure 1 is an exemplary system architecture diagram to which the present application may be applied;

[0051] Figure 2 This is a flowchart of an embodiment of a method for completing a causal graph based on reinforcement learning according to the present application;

[0052] Figure 3 1 is a schematic structural diagram of an embodiment of a device for completing a causal graph based on reinforcement learning according to the present application;

[0053] Figure 4 It is a structural diagram of an embodiment of a computer device according to the present application.

[0054] Figure numerals: 100, system architecture; 101-103, terminal device; 104, network; 105, server; 200, computer device; 201, memory; 202, processor; 203, network interface; 300, causal graph completion device based on reinforcement learning; 301, receiving module; 302, first computing module; 303, second computing module; 304, reasoning module. DETAILED DESCRIPTION

[0055] Unless otherwise defined, all technical and scientific terms used herein have the same meanings as commonly understood by those skilled in the art to which this application belongs. The terms used in the specification of the application are for the purpose of describing specific embodiments only and are not intended to limit this application. The terms "including" and "having" and any variations thereof in the specification and claims of this application and the above-mentioned drawings are intended to cover non-exclusive inclusions. The terms "first", "second", etc. in the specification and claims of this application or the above-mentioned drawings are used to distinguish different objects, not to describe a specific order.

[0056] References herein to "embodiments" mean that a particular feature, structure, or characteristic described in connection with the embodiments may be included in at least one embodiment of the present application. The appearance of this phrase in various places in the specification does not necessarily refer to the same embodiment, nor does it constitute an independent or alternative embodiment that is mutually exclusive of other embodiments. It is understood, both explicitly and implicitly, by those skilled in the art that the embodiments described herein may be combined with other embodiments.

[0057] In order to enable those skilled in the art to better understand the solution of the present application, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the accompanying drawings.

[0058] like Figure 1 As shown, system architecture 100 may include terminal devices 101, 102, 103, a network 104, and a server 105. Network 104 is a medium for providing communication links between terminal devices 101, 102, 103 and server 105. Network 104 may include various connection types, such as wired or wireless communication links or fiber optic cables.

[0059] Users can use terminal devices 101, 102, and 103 to interact with server 105 via network 104 to receive or send messages, etc. Various communication client applications can be installed on terminal devices 101, 102, and 103, such as web browser applications, shopping applications, search applications, instant messaging tools, email clients, social platform software, etc.

[0060] Terminal devices 101, 102, and 103 can be various electronic devices with display screens and support web browsing, including but not limited to smartphones, tablet computers, e-book readers, MP3 players (Moving Picture Experts Group Audio Layer III), MP4 (Moving Picture Experts Group Audio Layer IV), laptop computers, desktop computers, etc.

[0061] The server 105 may be a server that provides various services, such as a background server that provides support for web pages displayed on the terminal devices 101 , 102 , and 103 .

[0062] It should be noted that the reinforcement learning-based causal graph completion method provided in the embodiment of the present application is generally executed by a server / terminal device. Accordingly, the reinforcement learning-based causal graph completion device is generally set in the server / terminal device.

[0063] It should be understood that Figure 1 The number of terminal devices, networks and servers in the embodiment is merely illustrative. Any number of terminal devices, networks and servers may be provided as required.

[0064] Continue to refer Figure 2 , shows a flow chart of an embodiment of a method for completing a logical graph based on reinforcement learning according to the present application. The method for completing a logical graph based on reinforcement learning comprises the following steps:

[0065] S1: Receive event corpus, perform event entity recognition operation on the event corpus to obtain event entities, perform event relationship recognition operation on the event corpus to obtain event relationships, and construct an event graph based on the event entities and event relationships.

[0066] In this embodiment, the construction of the event graph is as follows: given the event training corpus used to train the model, an event recognition operation is performed on the event training corpus, which is specifically divided into an event entity recognition operation and an event relationship recognition operation, to obtain event entities and event relationships respectively. The event entity recognition operation is performed on the event training corpus, and the specific steps of obtaining event entities include: using an event extraction method to identify event entities from each sentence of the event training corpus, where the event entity is in the form of a phrase, for example: the price of iron ore has risen. Among them, there are many types of event extraction methods. This application adopts a remote supervision method, which is a semi-supervised learning algorithm. Among them, the remote supervision method can specifically use the PCNN (Piecewise Convolutional Neural Networks) model, which includes two layers of meaning: Piecewise max pooling layer and Convolutional Neural Networds, corresponding to the maximum pooling layer and the convolution layer. Using the powerful feature extraction function of the convolutional neural network, rich features can be automatically extracted, and the errors caused by manually designed features and feature extraction from the NLP tool library can be reduced. For all events, the identified event entities and their event relations (i.e., edges connecting event entities) are used to form a causal graph G. The set of all entity nodes (i.e., event entities) in the causal graph G is E, and the set of all edge nodes (i.e., event relations) is R.

[0067] In this embodiment, the electronic device (e.g. Figure 1 The server / terminal device shown in the figure can receive the event corpus via a wired connection or a wireless connection. It should be noted that the wireless connection method may include but is not limited to 3G / 4G / 5G connection, WiFi connection, Bluetooth connection, WiMAX connection, Zigbee connection, UWB (ultra wideband) connection, and other wireless connection methods currently known or to be developed in the future.

[0068] Specifically, the step of performing an event relationship recognition operation on the event corpus to obtain event relationships includes:

[0069] receiving a plurality of relational dictionaries, and classifying sentences in the event training corpus based on the relational dictionaries to determine relational dictionaries corresponding to the sentences, wherein the relational dictionaries include a plurality of relation names;

[0070] Identify the part of speech of each word in the sentence and determine whether there is only one verb in the sentence;

[0071] If so, determining whether the word with the verb part of speech exists in the relationship dictionary, and if so, taking the word with the verb part of speech as the event relationship corresponding to the event entity;

[0072] If there is not only one verb, a relationship name is selected from the relationship dictionary corresponding to the sentence in a preset manner as the event relationship corresponding to the event entity.

[0073] In this embodiment, the present application defines seven types of relationships based on mathematical logic, including: causal relationships, conditional relationships, reversal relationships, sequential relationships, hierarchical relationships, composition relationships, and concurrent relationships. Common descriptive words for each relationship type are collected, and a relationship dictionary corresponding to each relationship type is constructed, namely, a causal relationship dictionary, a conditional relationship dictionary, a reversal relationship dictionary, a sequential relationship dictionary, a hierarchical relationship dictionary, a composition relationship dictionary, and a concurrent relationship dictionary. For example, the causal relationship dictionary: [Relationship type: causal relationship, relationship name: "bring", "lead to"]. Each relationship dictionary in the present application corresponds to a relationship type. A classification algorithm is used to identify the relationship implied in the sentence where each event is located as one of the seven relationship types mentioned above, that is, based on the relationship dictionary and the classification algorithm, the sentences in the event training corpus are classified to determine the relationship dictionary corresponding to the sentence. Specifically, the classification algorithm is trained according to the relationship dictionary to obtain the trained classification algorithm, and the sentence is input into the trained classification algorithm to obtain the relationship dictionary corresponding to the sentence. The sentences in this application refer to sentences in the event training corpus that contain the event entity.

[0074] Identify the word types of each word in the sentence where the event entity is located. If there is only one verb, check whether the verb exists in the relation dictionary. If it exists, use it as the event relation. Since the pre-defined relation dictionary may not list all the relation names in life, if it does not exist, the word embedding vectors of the verb and the relation names in the relation dictionary are obtained through the word2vec idea respectively, and the semantic similarity of the word embedding vectors between them is calculated. Select the relation name with the highest semantic similarity as the event relation. For example, for a certain event, through the classification algorithm, it is identified as a causal事理 relation, and the verb in the sentence is "致使". This word does not appear in the dictionary. Then, through the above method, the semantic similarity between "致使" and "导致" is calculated to be 0.95, and the semantic similarity with "带来" is 0.91. Then, "导致" is used as the event relation. If there is more than one verb, select a relation name from the relation dictionary corresponding to the relation type as the event relation corresponding to the event entity. If there is no verb in the sentence, select a relation name from the corresponding relation dictionary as the event relation.

[0075] It should be noted that the classification algorithms adopted in this application include but are not limited to the NBC (Naive Bayesian Classifier) algorithm, the LR (Logistic Regress) algorithm, the ID3 (Iterative Dichotomiser 3) decision tree algorithm, the SVM (Support Vector Machine) algorithm, the KNN (K-Nearest Neighbor) algorithm, and the ANN (Artificial Neural Network) algorithm. In practical applications, any one or more of the above classification algorithms can be selected according to actual needs, or other classification algorithms can be selected according to actual needs as long as they are applicable.

[0076] Among them, the step of selecting a relation name from the relation dictionary corresponding to the sentence as the event relation corresponding to the event entity according to the preset method includes:

[0077] Input the words with the word type of verb in the sentence into the pre-trained word vector conversion model respectively, obtain the first word embedding vectors output respectively, calculate the average value of the first word embedding vectors, and obtain the average vector;

[0078] Input the relation names of the relation dictionary into the word vector conversion model respectively, and obtain the second word embedding vectors output respectively;

[0079] The semantic similarity between the average vector and the second word embedding vector is calculated, and the relationship name corresponding to the maximum semantic similarity is used as the event relationship.

[0080] In this embodiment, the word embedding vectors of all verbs in the sentence and the relationship names in the relationship dictionary are obtained respectively by a word vector conversion model (word2vec model), and the average value of all the above verbs is obtained. The word vector conversion model of this application adopts the word2vec model. The word2vec model is a tool for word vector calculation open sourced by Google. Word2vec can be efficiently trained on dictionaries of the order of millions and data sets of hundreds of millions. The training result obtained by this tool - word embedding, can well measure the similarity between words. The semantic similarity between the average value and the word embedding vector of the relationship name is calculated, that is, the cosine similarity between the vectors; the relationship name with the highest semantic similarity is selected as the event relationship.

[0081] S2: Call the pre-trained Bert model, input each entity node and edge node in the event graph into the pre-trained Bert model respectively, and obtain the entity embedding vector and edge embedding vector respectively.

[0082] In this embodiment, all entity nodes and edge nodes of the event graph G are respectively input into the pre-trained Bert model to obtain the output entity embedding vectors corresponding to the entity nodes and the edge embedding vectors corresponding to the edge nodes.

[0083] S3: Take any entity node in the event graph as the starting node, and calculate the state vector of each step of reasoning of the starting node in the event graph based on the entity embedding vector and the edge embedding vector.

[0084] In this embodiment, the state vector is calculated based on the entity embedding vector and the edge embedding vector, which facilitates the reasoning model to perform reasoning in the event graph. When each entity node in the event graph G completes one step of reasoning, it reports the change of its current state (that is, the edges it currently passes through and the entity nodes it reaches) to the reasoning module (the module that executes the reasoning program). Before the next step of reasoning, each entity node in the event graph G obtains the neighbor node information of the current reasoning state from the reasoning module, including the embedding vector of the neighbor node and the reasoning path taken to reach the neighbor node, and then calculates the state vector, which is then input into the reasoning model to determine the reasoning action for the next step of reasoning.

[0085] Specifically, the step of calculating the state vector of each step of reasoning of the central node in the event graph based on the entity embedding vector and the edge embedding vector includes:

[0086] Calculating the semantic distance between the entity nodes based on the entity embedding vector and the edge embedding vector to obtain the entity semantic distance;

[0087] In the event graph, a first region of interest is formed with the starting node as the center and a first semantic distance preset as the radius, and an entity node within the first region of interest is determined based on the entity semantic distance as a first neighbor node of the starting node;

[0088] With the first neighbor node as the center and a second semantic distance as the radius, a second region of interest is formed, the entity node in the second region of interest is used as the second neighbor node of the first neighbor node, and the second neighbor node connected to the first neighbor node in the event graph is used as the target neighbor node;

[0089] The state vector of the start node is calculated based on an edge embedding vector of an edge between the target neighbor node and the first neighbor node and an entity embedding vector of the first neighbor node.

[0090] In this embodiment, for any E i (i.e., the starting node), initially specifying a semantic distance as the radius to form an area of interest (AOI). The nodes within the AOI are E i (starting node)'s neighbor node. The radius of AOI can change dynamically, and every certain training cycle, the AOI range can be expanded or reduced according to the training effect, wherein the AOI range can be adjusted manually, or the computer can adjust it according to the reasoning accuracy. Specifically: if the ratio of the reasoning accuracy (the number of correct relationships discovered through reasoning / the total number of samples used for training) to the training overhead (such as memory consumed) increases, the AOI range can be expanded, otherwise it can be reduced. In the process of reinforcement learning and actual reasoning application, the present application considers the next action from a global perspective. The reasoning of each entity node is not only based on its own perspective, but also through the edge embedding vector of the edge between the target neighbor node and the first neighbor node and the entity embedding vector of the first neighbor node, it is realized to combine the information of neighboring events, gradually find the optimal path, and improve the accuracy of the completion of the event graph.

[0091] It should be noted that the specific values of the first semantic distance and the second semantic distance in this application may be the same or different and may be adjusted according to actual needs.

[0092] The step of calculating the state vector of the start node based on the edge embedding vector of the edge between the target neighbor node and the first neighbor node and the entity embedding vector of the first neighbor node includes:

[0093] The state vector of the starting node in the t-step reasoning is calculated based on the following formula:

[0094] in, represents the state vector of the starting node at step t, “:” represents the vector concatenation operation, represents the entity embedding vector of the first neighbor node reached at step t from the starting node, represents the sum of the edge embedding vectors from the starting node to the first neighbor node at the tth step, J represents the number of the target neighbor nodes corresponding to the first neighbor node at the tth step, It represents the sum of the edge embedding vectors of the edges passing from the target neighbor node corresponding to the first neighbor node in the t-th step to the first neighbor node in the t-th step.

[0095] In this embodiment, the reasoning state of step t includes: i (starting node) and arrives at E in step t k (first neighbor node) and its target neighbor node. Entity node state vector encoding: Define E i The state vector at step t is The state vector is used as an input model in the subsequent process to determine the reasoning action. i (starting node) and arrives at E after the tth step of reasoning k (first neighbor node), where each step refers to starting from the current entity node and following an edge to reach the next entity node. For E i (Starting Node) Departure and Arrival E k (first neighbor node) through the reasoning path (i.e., the edge in the reasoning graph G); find E k The second neighbor node of (the first neighbor node) starts from itself and reaches E through reasoning k Entity node E m (target neighbor node) (assuming there are J), For E m (Target neighbor node) departs and arrives at E k The sum of the embedding vectors of the inference path traversed by the first neighbor node. ":" denotes a vector concatenation operation. The Encoder is an encoding model based on a deep neural network, and can be implemented using the encoder module in the Transformer.

[0096] In addition, the step of calculating the semantic distance between the entity nodes based on the entity embedding vector and the edge embedding vector, and obtaining the entity semantic distance includes:

[0097] The entity semantic distance is calculated according to the following formula:

[0098] Ds(E i , E j )=sim(VE i , VE j )×sim(∑VR Ei ,∑VR Ej ), where Ds(E i , E j )

[0099] Represents the entity semantic distance, E i and E j Respectively represent the two entity nodes, VE i and VE j Represents the entity embedding vectors of the two entity nodes, ∑VR Ei and ∑VR Ej They represent the sum of the edge embedding vectors of all connected edges of the two entity nodes, and sim represents the cosine similarity.

[0100] In this embodiment, the entity semantic distance is calculated based on the entity embedding vectors of the two entity nodes and the edge embedding vector of the connected edge. i For E i Embedding vector of VR Ei is the edge embedding vector of the edge. sim(VE i , VE j ) is the cosine similarity of the entity embedding vectors of two entity nodes, ∑VR Ei For E i The sum of the edge embedding vectors of all connected edges. Define Ds(E i , E j ) is the entity node E i and E j The semantic distance between them.

[0101] S4: Based on the state vector and the pre-trained reasoning model, step-by-step reasoning is performed in the transaction graph to obtain the target transaction relationship between the two entity nodes, and the transaction graph is completed according to the target transaction relationship.

[0102] In this embodiment, the reasoning model is a multi-layer neural network with N categories of reasoning actions, where N is the total number of all relationship names in all the above-mentioned relationship dictionaries (seven types are given as an example in this application). The state vector of the starting node at the current step is input into the pre-trained reasoning model to obtain the reasoning action with the highest probability of output. In the event graph, reasoning is performed once according to the reasoning action with the highest probability, and finally the target event relationship is obtained. For example, in a certain event graph, the entity node "the price of iron ore in Brazil has risen" is used as the starting node, and an edge is selected through each step of reasoning to reach a node. Finally, it is found that the entity node "the price of iron ore in Brazil has risen" has a "bringing" relationship with "the increase in subway passenger flow at night".

[0103] Specifically, the step of performing step-by-step reasoning in the transaction graph based on the state vector and the pre-trained reasoning model to obtain the target transaction relationship between the two entity nodes includes:

[0104] Inputting the state vector corresponding to the starting node at each step of inference into the pre-trained inference model, the inference model processes the state vector through a first hidden layer, outputs a first probability sequence, processes the first probability sequence through a second hidden layer, and outputs a second probability sequence;

[0105] Processing the second probability sequence through a softmax layer to obtain a third probability sequence;

[0106] Taking the action corresponding to the maximum probability value in the third probability sequence as the inference action, and performing an inference to obtain an inference node and an inference relationship;

[0107] After multiple reasoning actions, until it is inferred that the starting node is the same as the reasoning node, the reasoning relationship is used as the target logical relationship.

[0108] In this embodiment, to maximize the value score of the inference action, this application uses the policy gradient method commonly used in the field of reinforcement learning to optimize the inference model, thereby obtaining a better-performing inference model. Specifically, the state vector is processed through the first hidden layer, the second hidden layer, and the softmax layer. In practice, more hidden layers can also be used to achieve better results.

[0109] The inference model processes the state vector through a first hidden layer to output a first probability sequence, and processes the first probability sequence through a second hidden layer to output a second probability sequence, including the following steps:

[0110] The first hidden layer processes the state vector using the following formula:

[0111] o1=relu(w1×v+b1), where v is the state vector, o1 is the first probability sequence, w1 is the weight of the first hidden layer, relu is the activation function, and b1 is the preset first bias;

[0112] The second hidden layer processes the first probability sequence using the following formula:

[0113] o2=relu(w2×o1+b2), where o2 is the second probability sequence, w2 is the weight of the second hidden layer, relu is the activation function, and b2 is the preset second bias.

[0114] In this embodiment, the input vector v (state vector) is set, the hidden layer weight matrix is set to w1, the relu activation function is adopted, the bias is b1, and the output o1 = relu (w1*v+b1); the second hidden layer weight matrix is set to w2, the bias is b2, the output o2 = relu (w2*o1+b2), and then o3 (the third probability sequence) is obtained through the softmax layer, and the action corresponding to the probability value (Pa) with the largest value in o3 is used as the inference action.

[0115] After the step of constructing a cause-and-effect graph based on the event entities and event relationships, the method further includes:

[0116] Extracting training data from the event graph, wherein the training data is a sequence consisting of entities and relationships;

[0117] Reward-based reinforcement learning is performed on the preset inference model based on the training data until a preset stopping condition is reached, thereby obtaining the pre-trained inference model.

[0118] In this embodiment, the training data consists of sequences of entities and relationships with correct relationships extracted from a logical graph. For example, a rise in Brazilian iron ore prices drives a rise in domestic car prices, while simultaneously stagnating household income. This application trains an inference model to discover relationships between entities. Once trained, the model can identify implicit relationships when presented with new data.

[0119] Specifically, the step of performing reward-based reinforcement learning on a preset inference model based on the training data includes:

[0120] After each step of reasoning, determine whether the reasoning is correct;

[0121] If it is correct, a first reward value is obtained, wherein the first reward value is

[0122] If it is wrong, a second reward value is obtained, wherein the second reward value is -len(path), wherein len(path) represents the number of edges passed by the reasoning.

[0123] In this embodiment, after each step of inference, judging whether the inference is correct means judging whether the inferred entities and relationships are correct, that is, whether they are the same as the preset real entities and real relationships. len(path) is the number of edges in the history. During the training process, after each inference is completed, if the entity is obtained correctly, a pre-defined reward value will be obtained. Otherwise, the reward value is -len(path). Obviously, if the reasoning fails after multiple steps, a larger penalty will be imposed. During training, the reasoning model may choose the wrong path (i.e., edge), so reinforcement learning is needed to train it, rewarding correct paths and penalizing incorrect paths. This allows the reasoning model to learn the connections between paths and find the correct one.

[0124] In addition, the step of obtaining the pre-trained inference model until a preset stopping condition is reached includes:

[0125] After each step of reasoning, the potential value score of the reasoning action performed during reasoning is calculated;

[0126] Obtaining a probability value of the inference action output by the inference model;

[0127] Calculate the negative value of the product of the logarithm of the probability value and the potential value score as an inference value;

[0128] Until the inference value is minimum and tends to be stable, it is determined that the multi-layer neural network has converged and the inference model is obtained.

[0129] In this embodiment, since each reasoning action will have an impact on subsequent actions, and this impact will gradually decay as the number of reasoning steps increases, it is also necessary to calculate the potential value score after each reasoning action. The probability value of the reasoning action output by the reasoning model is the probability value (Pa) with the largest value in the third probability sequence during the training process. This application uses the negative value of the product of the logarithm of Pa and the potential value score of the reasoning action as the training optimization target. When this value is minimized and almost no longer changes, that is, the multi-layer neural network converges, the training can be stopped. The optimization of multi-layer neural networks can adopt common optimization methods in the field of machine learning such as SGD (stochastic gradient descent) and Adam.

[0130] Furthermore, the step of calculating the potential value score of the reasoning action includes:

[0131] Calculating a reasoning action score according to the number of edges passed by the reasoning action;

[0132] The potential value score of the reasoning action is calculated according to the following formula:

[0133]

[0134] Where S′ represents the potential value score, γ represents the value attenuation coefficient, and S i-1 represents the reasoning action score for the i-1th time.

[0135] In this embodiment, S i-1 The calculation formula of is the same as the reward value. Specifically: If the reasoning is correct, then S i-1 is the i-1 reasoning action corresponding to (first reward value); if the reasoning is wrong, then S i-1 = -len(path) (second reward value) corresponding to i-1 reasoning actions. Wherein, len(path) represents the length of the reasoning path, that is, the number of historical edges passed during the reasoning process. In addition, this application also calculates the final total score S of the reasoning action. Specifically, After the reasoning model reaches the training optimization goal, that is, after the training is completed, the final reasoning action total score S reaches a maximum value.

[0136] In the process of completing the causal graph, this application uses state vectors for reasoning and considers the next action. The state vector is calculated based on the entity embedding vector and the edge embedding vector, so that the reasoning of each entity node is not only based on its own perspective. The entity embedding vector and the edge embedding vector are combined to find the optimal path, thereby improving the efficiency and accuracy of completing the causal graph.

[0137] Those skilled in the art will appreciate that all or part of the processes in the above-described method embodiments can be implemented by instructing related hardware via computer-readable instructions. The computer-readable instructions can be stored in a computer-readable storage medium. When executed, the computer-readable instructions can include the processes in the above-described method embodiments. The aforementioned storage medium can be a non-volatile storage medium such as a magnetic disk, an optical disk, a read-only memory (ROM), or a random access memory (RAM).

[0138] It should be understood that although the steps in the flowcharts of the accompanying drawings are shown in sequence as indicated by the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless otherwise specified herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some of the steps in the flowcharts of the accompanying drawings may include multiple sub-steps or multiple stages, and these sub-steps or stages are not necessarily executed at the same time, but can be executed at different times, and their execution order is not necessarily sequential, but can be executed in turn or alternately with other steps or at least a portion of the sub-steps or stages of other steps.

[0139] Further references Figure 3 , as a response to the above Figure 2 The present application provides an embodiment of a device for completing a logical graph based on reinforcement learning. Figure 2 Corresponding to the method embodiment shown, the device can be specifically applied to various electronic devices.

[0140] like Figure 3 As shown, the reinforcement learning-based causal graph completion device 300 described in this embodiment includes: a receiving module 301, a first calculation module 302, a second calculation module 303 and an inference module 304. Among them: the receiving module 301 is used to receive event corpus, perform event entity recognition operation on the event corpus to obtain event entities, perform event relationship recognition operation on the event corpus to obtain event relationships, and construct a causal graph based on the event entities and event relationships; the first calculation module 302 is used to call the pre-trained Bert model, input each entity node and edge node in the causal graph into the pre-trained Bert model respectively, and obtain entity embedding vectors and edge embedding vectors respectively; the second calculation module 303 is used to take any entity node in the causal graph as the starting node, and calculate the state vector of each step of reasoning of the starting node in the causal graph based on the entity embedding vector and the edge embedding vector; the reasoning module 304 is used to perform step-by-step reasoning in the causal graph based on the state vector and the pre-trained reasoning model, obtain the target causal relationship between the two entity nodes, and complete the causal graph according to the target causal relationship.

[0141] In this embodiment, in the process of completing the causal graph, the present application uses the state vector for reasoning and considers the next action, wherein the state vector is calculated based on the entity embedding vector and the edge embedding vector, so that the reasoning of each entity node is not only from its own perspective, but also combines the entity embedding vector and the edge embedding vector to find the optimal path, thereby improving the efficiency and accuracy of completing the causal graph.

[0142] The receiving module 301 includes a receiving submodule, an identifying submodule, a determining submodule, and a selecting submodule. The receiving submodule is configured to receive multiple relationship dictionaries, classify sentences in the event training corpus based on the relationship dictionaries, and determine the relationship dictionary corresponding to the sentences, wherein the relationship dictionary includes multiple relationship names; the identifying submodule is configured to identify the part of speech of each word in the sentence and determine whether there is one and only one verb in the sentence; the determining submodule is configured to determine whether the word with the part of speech of a verb exists in the relationship dictionary when there is one and only one verb in the sentence, and if so, use the word with the part of speech of a verb as the event relationship corresponding to the event entity; and the selecting submodule is configured to select a relationship name from the relationship dictionary corresponding to the sentence in a preset manner as the event relationship corresponding to the event entity when there is not one and only one verb in the sentence.

[0143] The selection submodule includes a first conversion unit, a second conversion unit and a calculation unit, wherein the first conversion unit is used to input the words with verb parts in the sentence into the pre-trained word vector conversion model respectively, obtain the output first word embedding vectors respectively, calculate the average value of the first word embedding vectors, and obtain the average vector; the second conversion unit is used to input the relationship names of the relationship dictionary into the word vector conversion model respectively, and obtain the output second word embedding vectors respectively; the calculation unit is used to calculate the semantic similarity between the average vector and the second word embedding vector, and take the relationship name corresponding to the maximum semantic similarity as the event relationship.

[0144] The second calculation module 303 includes a first calculation submodule, a first region forming submodule, a second region forming submodule and a second calculation submodule; the first calculation submodule is used to calculate the semantic distance between the entity nodes based on the entity embedding vector and the edge embedding vector to obtain the entity semantic distance; the first region forming submodule is used to form a first interest region in the matter graph with the starting node as the center and the first semantic distance preset as the radius, and determine the entity node in the first interest region as the first neighbor node of the starting node based on the entity semantic distance; the second region forming submodule is used to form a second interest region with the first neighbor node as the center and the second semantic distance preset as the radius, and use the entity node in the second interest region as the second neighbor node of the first neighbor node, and use the second neighbor node connected to the first neighbor node in the matter graph as the target neighbor node; the second calculation submodule is used to calculate the state vector of the starting node based on the edge embedding vector of the edge between the target neighbor node and the first neighbor node and the entity embedding vector of the first neighbor node.

[0145] In some optional implementations of this embodiment, the first calculation submodule is further configured to:

[0146] The entity semantic distance is calculated according to the following formula:

[0147] Ds(E i , E j )=sim(VE i , VE j )×sim(∑VR Ei ,∑VR Ej ), where Ds(E i , E j ) represents the entity semantic distance, E i and E j Respectively represent the two entity nodes, VE i and VE j Represents the entity embedding vectors of the two entity nodes, ∑VR Ei and ∑VR Ej They represent the sum of the edge embedding vectors of all connected edges of the two entity nodes, and sim represents the cosine similarity.

[0148] In some optional implementations of this embodiment, the second calculation submodule is further configured to calculate the state vector of the starting node at the t-th step of reasoning based on the following formula:

[0149] in, represents the state vector of the starting node at step t, “:” represents the vector concatenation operation, represents the entity embedding vector of the first neighbor node reached at step t from the starting node, represents the sum of the edge embedding vectors from the starting node to the first neighbor node at the tth step, J represents the number of the target neighbor nodes corresponding to the first neighbor node at the tth step, It represents the sum of the edge embedding vectors of the edges passing from the target neighbor node corresponding to the first neighbor node in the t-th step to the first neighbor node in the t-th step.

[0150] The reasoning module 304 includes a first input submodule, a second input submodule, a reasoning submodule and an acquisition submodule; the first input submodule is used to input the state vector corresponding to the starting node at each step of reasoning into the pre-trained reasoning model, and the reasoning model processes the state vector through the first hidden layer to output a first probability sequence, processes the first probability sequence through the second hidden layer, and outputs a second probability sequence; the second input submodule is used to process the second probability sequence through the softmax layer to obtain a third probability sequence; the reasoning submodule is used to take the action corresponding to the probability value with the largest numerical value in the third probability sequence as the reasoning action, and perform one reasoning to obtain the reasoning node and the reasoning relationship; the acquisition submodule is used to perform multiple reasoning actions until it is inferred that the starting node is the same as the reasoning node, and then take the reasoning relationship as the target matter-of-fact relationship.

[0151] In some optional implementations of this embodiment, the above-mentioned first input submodule is further used to: the first hidden layer processes the state vector through the following formula: o1=relu(w1×v+b1), wherein v is the state vector, o1 is the first probability sequence, w1 is the weight of the first hidden layer, relu is the activation function, and b1 is the preset first bias; the second hidden layer processes the first probability sequence through the following formula: o2=relu(w2×o1+b2), wherein o2 is the second probability sequence, w2 is the weight of the second hidden layer, relu is the activation function, and b2 is the preset second bias.

[0152] In some optional implementations of this embodiment, the above-mentioned device 300 also includes: an extraction module and a learning module, the extraction module is used to extract training data from the matter graph, wherein the training data is a sequence composed of entities and relationships; the learning module is used to perform reward-based reinforcement learning on a preset reasoning model based on the training data until a preset stopping condition is reached, thereby obtaining the pre-trained reasoning model.

[0153] The extraction module includes a judgment submodule, a first reward submodule and a second reward submodule; the judgment submodule is used to judge whether the reasoning is correct after each step of reasoning; the first reward submodule is used to obtain a first reward value when it is correct, wherein the first reward value is The second reward submodule is used to obtain a second reward value when an error occurs, wherein the second reward value is -len(path), wherein len(path) represents the number of edges passed by the reasoning.

[0154] The learning module includes a value calculation submodule, a probability value acquisition submodule, an inference value calculation submodule and a convergence submodule, wherein the value calculation submodule is used to calculate the potential value score of the inference action performed during the inference after each step of inference; the probability value acquisition submodule is used to obtain the probability value of the inference action output by the inference model; the inference value calculation submodule is used to calculate the negative value of the product of the logarithm of the probability value and the potential value score as the inference value; the convergence submodule is used to determine that the multi-layer neural network converges and obtain the inference model until the inference value is minimum and tends to be stable.

[0155] The value calculation submodule includes a score calculation unit and a value calculation unit. The score calculation unit is used to calculate the reasoning action score according to the number of edges passed by the reasoning action; the value calculation unit is used to calculate the potential value score of the reasoning action according to the following formula:

[0156]

[0157] Where S′ represents the potential value score, γ represents the value attenuation coefficient, and S i-1 represents the reasoning action score for the i-1th time.

[0158] In the process of completing the causal graph, this application uses state vectors for reasoning and considers the next action. The state vector is calculated based on the entity embedding vector and the edge embedding vector, so that the reasoning of each entity node is not only based on its own perspective. The entity embedding vector and the edge embedding vector are combined to find the optimal path, thereby improving the efficiency and accuracy of completing the causal graph.

[0159] To solve the above technical problems, the present application also provides a computer device. Figure 4 , Figure 4 This is a basic structural block diagram of the computer device in this embodiment.

[0160] The computer device 200 includes a memory 201, a processor 202, and a network interface 203 that are interconnected through a system bus. It should be noted that the figure only shows a computer device 200 having components 201-203, but it should be understood that it is not required to implement all the components shown, and more or fewer components can be implemented instead. Among them, those skilled in the art can understand that the computer device here is a device that can automatically perform numerical calculations and / or information processing according to pre-set or stored instructions, and its hardware includes but is not limited to microprocessors, application specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), digital signal processors (DSPs), embedded devices, etc.

[0161] The computer device may be a desktop computer, notebook computer, PDA, cloud server, etc. The computer device may interact with the user via a keyboard, mouse, remote control, touchpad, or voice control device.

[0162] The memory 201 includes at least one type of readable storage medium, including flash memory, hard disk, multimedia card, card-type memory (e.g., SD or DX memory), random access memory (RAM), static random access memory (SRAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), programmable read-only memory (PROM), magnetic memory, magnetic disk, optical disk, etc. In some embodiments, the memory 201 may be an internal storage unit of the computer device 200, such as a hard disk or memory of the computer device 200. In other embodiments, the memory 201 may also be an external storage device of the computer device 200, such as a plug-in hard disk equipped on the computer device 200, a smart media card (SMC), a secure digital (SD) card, a flash card, etc. Of course, the memory 201 may also include both the internal storage unit of the computer device 200 and its external storage device. In this embodiment, the memory 201 is generally used to store an operating system and various application software installed on the computer device 200, such as computer-readable instructions for a reinforcement learning-based causal graph completion method. In addition, the memory 201 can also be used to temporarily store various types of data that have been output or are about to be output.

[0163] In some embodiments, the processor 202 may be a central processing unit (CPU), a controller, a microcontroller, a microprocessor, or other data processing chip. The processor 202 is generally used to control the overall operation of the computer device 200. In this embodiment, the processor 202 is used to execute computer-readable instructions or process data stored in the memory 201, such as computer-readable instructions for executing the reinforcement learning-based causal graph completion method.

[0164] The network interface 203 may include a wireless network interface or a wired network interface. The network interface 203 is generally used to establish a communication connection between the computer device 200 and other electronic devices.

[0165] The present application also provides another embodiment, namely, providing a computer-readable storage medium, which stores computer-readable instructions, and the computer-readable instructions can be executed by at least one processor to enable the at least one processor to perform the steps of the above-mentioned reinforcement learning-based causal graph completion method.

[0166] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus the necessary general hardware platform, and of course can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk), and includes a number of instructions for enabling a terminal device (which can be a mobile phone, computer, server, air conditioner, or network device, etc.) to execute the methods described in each embodiment of the present application.

[0167] Obviously, the embodiments described above are only some of the embodiments of the present application, rather than all of the embodiments. The preferred embodiments of the present application are given in the accompanying drawings, but they do not limit the patent scope of the present application. The present application can be implemented in many different forms. On the contrary, the purpose of providing these embodiments is to make the understanding of the disclosure of the present application more thorough and comprehensive. Although the present application has been described in detail with reference to the aforementioned embodiments, for those skilled in the art, it is still possible to modify the technical solutions described in the aforementioned specific embodiments, or to make equivalent replacements for some of the technical features therein. Any equivalent structure made using the contents of the present application specification and the accompanying drawings, directly or indirectly used in other related technical fields, is also within the scope of patent protection of the present application.

Claims

1. A method for completing a causal graph based on reinforcement learning, characterized in that: The steps include: Receive event training corpus, perform event entity recognition operation on the event training corpus to obtain event entities, perform event relationship recognition operation on the event training corpus to obtain event relationships, and construct a matter graph based on the event entities and event relationships, extract training data from the matter graph, wherein the training data is a sequence of entities and relationships, perform reward-based reinforcement learning on a preset reasoning model based on the training data, calculate the potential value score of the reasoning action performed by the preset reasoning model during reasoning after each step of reasoning, obtain the probability value of the reasoning action output by the preset reasoning model, calculate the negative value of the product of the logarithm of the probability value and the potential value score as the reasoning value, until the reasoning value is minimum and tends to be stable, determine that the multi-layer neural network of the preset reasoning model has converged, and obtain the reasoning model; Call the pre-trained Bert model, input each entity node and edge node in the event graph into the pre-trained Bert model, and obtain the entity embedding vector and edge embedding vector respectively; Taking any entity node in the affair graph as the starting node, calculating the semantic distance between the entity nodes based on the entity embedding vector and the edge embedding vector to obtain the entity semantic distance, in the affair graph, taking the starting node as the center and presetting a first semantic distance as the radius to form a first region of interest, and determining the entity node in the first region of interest as the first neighbor node of the starting node based on the entity semantic distance, and taking the first neighbor node as the center and presetting a second semantic distance as the radius to form a second region of interest, taking the entity node in the second region of interest as the second neighbor node of the first neighbor node, and taking the second neighbor node connected to the first neighbor node in the affair graph as the target neighbor node, and calculating the state vector of the starting node based on the edge embedding vector of the edge between the target neighbor node and the first neighbor node and the entity embedding vector of the first neighbor node; Based on the state vector and the pre-trained reasoning model, step-by-step reasoning is performed in the transaction graph to obtain the target transaction relationship between the two entity nodes, and the transaction graph is completed according to the target transaction relationship.

2. The method for completing the event graph based on reinforcement learning according to claim 1 is characterized in that: The step of performing an event relationship recognition operation on the event training corpus to obtain event relationships includes: receiving a plurality of relational dictionaries, and classifying sentences in the event training corpus based on the relational dictionaries to determine relational dictionaries corresponding to the sentences, wherein the relational dictionaries include a plurality of relation names; Identify the part of speech of each word in the sentence and determine whether there is only one verb in the sentence; If so, determining whether the word with the verb part of speech exists in the relationship dictionary, and if so, taking the word with the verb part of speech as the event relationship corresponding to the event entity; If there is not only one verb, a relationship name is selected from the relationship dictionary corresponding to the sentence in a preset manner as the event relationship corresponding to the event entity.

3. The method for completing the causal graph based on reinforcement learning according to claim 2 is characterized in that: The step of selecting a relationship name from the relationship dictionary corresponding to the sentence in a preset manner as the event relationship corresponding to the event entity includes: Inputting the words whose part of speech is verb in the sentence into the pre-trained word vector conversion model respectively, obtaining the output first word embedding vectors respectively, and calculating the average value of the first word embedding vectors to obtain the average vector; Inputting the relationship names of the relationship dictionary into the word vector conversion model respectively, and obtaining output second word embedding vectors respectively; The semantic similarity between the average vector and the second word embedding vector is calculated, and the relationship name corresponding to the maximum semantic similarity is used as the event relationship.

4. The method for completing a logical graph based on reinforcement learning according to claim 1, characterized in that: The step of calculating the state vector of the start node based on the edge embedding vector of the edge between the target neighbor node and the first neighbor node and the entity embedding vector of the first neighbor node includes: The state vector of the starting node in the t-step reasoning is calculated based on the following formula: ,in, represents the state vector of the starting node at step t, ":" represents the vector concatenation operation, represents the entity embedding vector of the first neighbor node reached at step t from the starting node, represents the sum of the edge embedding vectors from the starting node to the first neighbor node at the tth step, J represents the number of the target neighbor nodes corresponding to the first neighbor node at the tth step, It represents the sum of the edge embedding vectors of the edges passing from the target neighbor node corresponding to the first neighbor node in the t-th step to the first neighbor node in the t-th step.

5. The method for completing the event graph based on reinforcement learning according to claim 1, characterized in that: The step of performing step-by-step reasoning in the transaction graph based on the state vector and the pre-trained reasoning model to obtain the target transaction relationship between the two entity nodes includes: Inputting the state vector corresponding to the starting node at each step of inference into the pre-trained inference model, the inference model processes the state vector through a first hidden layer, outputs a first probability sequence, processes the first probability sequence through a second hidden layer, and outputs a second probability sequence; Processing the second probability sequence through a softmax layer to obtain a third probability sequence; Taking the action corresponding to the maximum probability value in the third probability sequence as the inference action, and performing an inference to obtain an inference node and an inference relationship; After multiple reasoning actions, until it is inferred that the starting node is the same as the reasoning node, the reasoning relationship is used as the target logical relationship.

6. A device for completing a causal graph based on reinforcement learning, characterized in that: include: A receiving module is used to receive event training corpus, perform event entity recognition operation on the event training corpus to obtain event entities, perform event relationship recognition operation on the event training corpus to obtain event relationships, and construct a matter graph based on the event entities and event relationships, and extract training data from the matter graph, wherein the training data is a sequence of entities and relationships, and a preset reasoning model is subjected to reward-based reinforcement learning based on the training data. After each step of reasoning, the potential value score of the reasoning action performed by the preset reasoning model during reasoning is calculated, the probability value of the reasoning action output by the preset reasoning model is obtained, and the negative value of the product of the logarithm of the probability value and the potential value score is calculated as the reasoning value, until the reasoning value is minimized and tends to be stable, and it is determined that the multi-layer neural network of the preset reasoning model has converged, and an reasoning model is obtained; The first computing module is used to call the pre-trained Bert model, input each entity node and edge node in the event graph into the pre-trained Bert model, and obtain the entity embedding vector and edge embedding vector respectively; A second calculation module is used to take any entity node in the affair graph as the starting node, calculate the semantic distance between the entity nodes based on the entity embedding vector and the edge embedding vector, and obtain the entity semantic distance. In the affair graph, with the starting node as the center and a first semantic distance preset as the radius, a first region of interest is formed, and based on the entity semantic distance, the entity node in the first region of interest is determined as the first neighbor node of the starting node, with the first neighbor node as the center and a second semantic distance preset as the radius, a second region of interest is formed, and the entity node in the second region of interest is used as the second neighbor node of the first neighbor node, and the second neighbor node connected to the first neighbor node in the affair graph is used as the target neighbor node, and the state vector of the starting node is calculated based on the edge embedding vector of the edge between the target neighbor node and the first neighbor node and the entity embedding vector of the first neighbor node; An inference module is used to perform step-by-step reasoning in the transaction graph based on the state vector and a pre-trained inference model, obtain a target transaction relationship between two entity nodes, and complete the transaction graph according to the target transaction relationship.

7. A computer device, characterized in that: It includes a memory and a processor, wherein the memory stores computer-readable instructions, and when the processor executes the computer-readable instructions, it implements the steps of the reinforcement learning-based causal graph completion method as described in any one of claims 1 to 5.

8. A computer-readable storage medium, characterized in that The computer-readable storage medium stores computer-readable instructions, which, when executed by a processor, implement the steps of the reinforcement learning-based causal graph completion method according to any one of claims 1 to 5.

Images