An event extraction method and device based on causal knowledge constraints

Abstract

Provided in the embodiments of the present disclosure is a method and device for event extraction based on causal knowledge constraints, which comprises constructing a causal graph of an event text to obtain a causal strength matrix of feature data of the event text; constructing a question template of an event argument according to a type of the event; combining the question template and the event text and encoding, and inputting the encoded sequence into a BERT model to take the hidden state of the last layer as a semantic representation of the input sequence; applying a self-attention weight matrix to weight the semantic representation of the input sequence to obtain a weighted output; comparing the causal strength matrix and the self-attention weight matrix to obtain a causal loss term; and combining the causal loss term to define a loss function to obtain an answer to a question according to the weighted output. Through the processing scheme of the present disclosure, the interpretability of the model is improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of deep learning, and particularly relates to an event extraction model based on causal knowledge constraint and a device. BACKGROUND

[0002] At present, deep learning technology has achieved certain results in event extraction due to its strong fitting and generalization capabilities. An end-to-end model has achieved remarkable results in the event extraction task. However, the original intention of artificial intelligence is to make the machine imitate human intelligence when making decisions. Most neural network models are not a transparent and explainable model for decision makers. The model can only give a prediction result, but cannot give the basis behind the prediction. Explainability will be an indispensable part of machine learning models. Effectively explaining the model and increasing the explainability of the deep learning model so that the neural network can think according to the same logic as the human brain when making decisions is gradually becoming a hot issue in the academic and industrial circles. SUMMARY

[0003] Therefore, the present application provides an event extraction method and device based on causal knowledge constraint, which at least partially solves the problems in the prior art.

[0004] In a first aspect, the present application provides an event extraction method based on causal knowledge constraint, which is used to obtain an answer to a question. The method comprises the following steps:

[0005] constructing a causal graph of an event text to obtain a causal strength matrix of feature data of the event text;

[0006] constructing a question template of an argument of the event according to the type of the event;

[0007] combining the question template and the event text and encoding, and inputting the encoded sequence into a BERT model to take the hidden state of the last layer as a semantic representation of the input sequence;

[0008] applying a self-attention weight matrix to weight the semantic representation of the input sequence to obtain a weighted output;

[0009] comparing the causal strength matrix and the self-attention weight matrix to obtain a causal loss term; and

[0010] defining a loss function in combination with the causal loss term to obtain an answer to a question according to the weighted output.

[0011] In one specific implementation, constructing a causal graph of an event text comprises the following steps:

[0012] extracting feature data from the event text to form a feature matrix X;

[0013] computing a covariance matrix S of the feature matrix X to represent the correlation between different feature data; and

[0014] minimizing a loss function to obtain a causal graph of the event text, wherein the loss function is the difference between the covariance matrix S and a predicted covariance matrix under a given causal graph structure, denoted as follows:

[0015]

[0016] wherein Θ is a causal strength matrix of the causal graph, is a predicted covariance matrix under a given graph structure Θ, is the Frobenius norm, denotes the loss function of the causal graph.

[0017] In one specific implementation, the method further comprises obtaining an optimized causal strength matrix :

[0018]

[0019] wherein, is an L1 regularization term to ensure the sparsity of the graph, λ is a regularization parameter, and is a hyperparameter learned by the model during the training process, is the L1 norm of the adjacency matrix.

[0020] In one specific implementation, the method further comprises thresholding the optimized causal strength matrix using the following method:

[0021] setting a threshold, and setting the elements in the causal strength matrix whose absolute values are less than the threshold to 0, and retaining the elements greater than or equal to the threshold.

[0022] In one specific implementation, combining the question template and the event text and encoding comprises:

[0023] combining the question template and the event text in the following manner:

[0024] [CLS] question template [SEP] event text [SEP]

[0025] wherein [CLS] is a special classification mark of BERT, [SEP] is a special mark representing different sentences, and the encoding sequence is obtained by combining wherein wherein qi represents the vector form of the ith split word in the question template, xi represents the vector form of the ith split word in the event text, m is the length of the question template, and N is the length of the event text.

[0026] In one specific implementation, applying a self-attention weight matrix to the semantic representation of the input sequence to obtain a weighted output includes obtaining the self-attention weight matrix a using the following equation:

[0027]

[0028] wherein Q is a query vector in the attention mechanism, K is a key vector, V is a value vector, and in the self-attention mechanism and the weighted output is obtained by the following equation :

[0029] .

[0030] In one specific implementation, comparing the causal strength matrix and the self-attention weight matrix to obtain a causal loss term includes calculating the causal loss term according to the following equation:

[0031]

[0032] wherein

[0033]

[0034] is an element of the causal strength matrix, is an element of the self-attention weight matrix.

[0035] In one specific implementation, defining a loss function in combination with the causal loss term to obtain an answer to the question from the weighted output includes defining the loss function according to the following equation:

[0036]

[0037] wherein is a loss for the start position, is a loss for the end position, and is the causal loss term, and the start index and end index matrices of the answer are calculated according to the following equation:

[0038]

[0039] wherein represents a probability that the ith part in the input is selected as the start position of the predicted argument role, represents a probability that the ith part in the input is selected as the end position of the predicted argument role, is a weight hyperparameter matrix and is obtained during model training, and the loss of the start position and the loss of the end position are calculated by:

[0040] ,

[0041] wherein, is the cross-entropy between the predicted start position and the real start position , is the cross-entropy between the predicted end position and the real end position .

[0042] In one specific implementation, the method further comprises employing a dynamic threshold algorithm to filter out reasonable candidate answer intervals, comprising:

[0043] By calculating the start position probability and the end position probability of each word, all possible candidate answer intervals are generated;

[0044] For each candidate answer interval , the following conditions are verified:

[0045] and to ensure that both the start and end positions are within the sentence, wherein N

[0046] is the length of the sentence;

[0047] to ensure that the length of the answer does not exceed the preset maximum value;

[0048] The joint probability of the start position probability and the end position probability must be greater than the dynamic threshold Tt, wherein wherein, is the output probability of the [CLS] label, represents the overall probability of no answer, and a t is an adjustment parameter, is the average probability of all no-answer candidates; and

[0049] The candidate answer is retained , wherein represents the joint probability of the start position and the end position; and

[0050] All candidate answers with a relative no-answer score less than 0 are selected as the final answer selection.

[0051] In a second aspect, an event extraction device based on causal knowledge constraints is provided, the device being configured to obtain answers to questions, the device comprising:

[0052] a causal graph construction module configured to construct a causal graph of an event text to obtain a causal strength matrix of feature data of the event text;

[0053] a question template construction module configured to construct a question template of an event argument according to a type of the event; and

[0054] an event extraction model, the event extraction model comprising:

[0055] an encoding layer configured to combine the question template and the event text and encode, and input an encoded sequence into a BERT model to take a hidden state of a last layer as a semantic representation of the input sequence;

[0056] a self-attention layer configured to apply a self-attention weight matrix to the semantic representation of the input sequence to obtain a weighted output;

[0057] a causal self-attention constraint layer configured to compare the causal strength matrix and the self-attention weight matrix to obtain a causal loss term; and

[0058] a prediction layer configured to define a loss function in combination with the causal loss term to obtain answers to questions according to the weighted output.

[0059] In the embodiments of the present disclosure, an event extraction method and device based on causal knowledge constraints are provided, the method comprising constructing a causal graph of an event text to obtain a causal strength matrix of feature data of the event text; constructing a question template of an event argument according to a type of the event; combining the question template and the event text and encoding, and inputting an encoded sequence into a BERT model to take a hidden state of a last layer as a semantic representation of the input sequence; applying a self-attention weight matrix to the semantic representation of the input sequence to obtain a weighted output; comparing the causal strength matrix and the self-attention weight matrix to obtain a causal loss term; and defining a loss function in combination with the causal loss term to obtain answers to questions according to the weighted output. Through the processing scheme of the present disclosure, the interpretability of the model is improved. BRIEF DESCRIPTION OF DRAWINGS

[0060] In order to more clearly illustrate the technical solutions of the embodiments of the present disclosure, the drawings needed in the embodiments will be briefly introduced as follows. Obviously, the drawings in the following description are only some embodiments of the present disclosure, and other drawings can be obtained by those skilled in the art without creative labor.

[0061] Figure 1 A schematic diagram of the structure of an event extraction model based on causal knowledge constraints provided in an embodiment of this disclosure;

[0062] Figure 2 A flowchart of an event extraction method based on causal knowledge constraints provided in an embodiment of this disclosure;

[0063] Figure 3 A schematic diagram of the process for constructing a causal graph of event text to obtain a causal intensity matrix of feature data of the event text, provided in an embodiment of this disclosure;

[0064] Figure 4 This is a schematic diagram of the structure of the event extraction model provided in the embodiments of this disclosure;

[0065] Figure 5 A flowchart for effectively filtering reasonable candidate answer ranges using a dynamic threshold algorithm provided in this embodiment of the disclosure;

[0066] Figure 6 This is a schematic diagram of an event extraction device based on causal knowledge constraints provided in an embodiment of this disclosure. Detailed Implementation

[0067] The embodiments of this disclosure will now be described in detail with reference to the accompanying drawings.

[0068] The following specific examples illustrate the implementation of this disclosure. Those skilled in the art can easily understand other advantages and effects of this disclosure from the content disclosed in this specification. Obviously, the described embodiments are only a part of the embodiments of this disclosure, and not all of them. This disclosure can also be implemented or applied through other different specific embodiments, and the details in this specification can also be modified or changed based on different viewpoints and applications without departing from the spirit of this disclosure. It should be noted that, in the absence of conflict, the following embodiments and features in the embodiments can be combined with each other. Based on the embodiments in this disclosure, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this disclosure.

[0069] It should be noted that various aspects of embodiments within the scope of the appended claims are described below. It will be apparent that the aspects described herein can be embodied in a wide variety of forms, and any particular structure and / or function described herein is merely illustrative. Based on this disclosure, those skilled in the art will understand that one aspect described herein can be implemented independently of any other aspect, and two or more of these aspects can be combined in various ways. For example, any number of aspects set forth herein can be used to implement the device and / or practice the method. Additionally, this device and / or method can be implemented using structures and / or functionalities other than one or more of the aspects set forth herein.

[0070] It should also be noted that the illustrations provided in the following embodiments are only schematic representations of the basic concept of this disclosure. The drawings only show the components related to this disclosure and are not drawn according to the number, shape and size of the components in actual implementation. In actual implementation, the form, quantity and proportion of each component can be arbitrarily changed, and the layout of the components may also be more complex.

[0071] Furthermore, specific details are provided in the following description to facilitate a thorough understanding of the examples. However, those skilled in the art will understand that the described aspects can be practiced without these specific details.

[0072] This application proposes an event extraction model that uses prior causal knowledge to constrain an attention network. Experimental results show that the model has good interpretability without significantly compromising performance, and its interpretation results are consistent with prior knowledge. Next, with reference to the accompanying figures, a detailed description of this application's event extraction model based on causal knowledge constraints is provided.

[0073] First, refer to Figure 1 This describes an event extraction model for performing the event extraction method based on causal knowledge constraints of this application. The model includes a causal graph and an event extraction model, wherein the upper part is the causal graph and the lower part is the event extraction model.

[0074] A cause-effect graph is a graphical representation used to visualize and analyze causal relationships between variables. It uses nodes to represent variables and directed edges to indicate the causal relationships between them. In a cause-effect graph, each node represents a variable, and an arrow pointing from one node to another indicates that the former is the cause of the latter.

[0075] Event extraction models are a natural language processing technique designed to automatically identify and extract specific types of events and their associated attributes from text data. These models are typically built on machine learning or deep learning algorithms, enabling them to understand text content and identify event triggers and participants. Event extraction models first analyze the text through preprocessing steps (such as word segmentation, part-of-speech tagging, and named entity recognition), then use techniques like pattern matching, dependency parsing, or sequence labeling to determine the type and boundaries of the event. Next, the model further extracts detailed event information, such as time, location, participants, and the event's cause, process, and outcome. Event extraction has significant application value in information retrieval, intelligence analysis, automatic summarization, and question-answering systems, helping systems to understand text content more deeply and thus provide richer and more accurate information.

[0076] This application utilizes, for example Figure 1 The model shown is executed as follows Figure 2 The event extraction method based on causal knowledge constraints shown specifically includes:

[0077] S201: Construct a causal graph of the event text to obtain a causal strength matrix of the feature data of the event text;

[0078] S202: Construct a problem template for event arguments based on the type of the event;

[0079] S203: Combine the question template and the event text and encode them, and input the encoded sequence into the BERT model to use the hidden state of the last layer as the semantic representation of the input sequence;

[0080] S204: Apply a self-attention weight matrix to weight the semantic representation of the input sequence to obtain a weighted output;

[0081] S205: Compare the causal strength matrix with the self-attention weight matrix to obtain the causal loss term; and

[0082] S206: Define a loss function based on the causal loss term to obtain the answer to the question based on the weighted output.

[0083] Next, refer to Figure 3This application describes in detail the process of constructing a causal graph of event text to obtain a causal intensity matrix of the feature data of the event text. In this application, the NO-TEARS (No TEquilibriumAssumptionsRequired) algorithm is used to construct the causal graph. It is mainly used to learn causal structure graphs from observed data, where nodes represent variables and edges represent causal relationships. Unlike traditional methods, NO-TEARS does not rely on equilibrium assumptions or structural equation model assumptions. However, it should be understood that other methods for constructing causal graphs can also be used, such as expert knowledge-based methods, statistical algorithms, structural learning algorithms (such as LiNGAM), and hybrid methods (such as combining the Fuzzy Delphi Method and Fuzzy DEMATEL techniques), as long as the method can construct a causal graph. The specific process of constructing the causal graph in this application includes:

[0084] S301: Extract feature data from the event text to form a feature matrix X. First, extract relevant feature data, such as argument roles, trigger words, and event types, from the training set of multiple event texts based on the labeled corpus. Second, preprocess the data to convert it into binary numerical features, marking 0 for the presence of trigger words and 1 for the absence of trigger words, and so on. Finally, form the feature matrix from the processed text data. , where n is the number of samples and p is the feature dimension.

[0085] S302: Calculate the covariance matrix S of the feature matrix X to represent the correlation between different feature data; define the covariance matrix, and calculate the covariance matrix S of the feature matrix to represent the correlation between different features (or events), the calculation formula is as follows:

[0086]

[0087] Where n is the number of samples and X is the feature matrix.

[0088] S303: Minimize the loss function to obtain the causal graph of the event text; the loss function of the NO-TEARS algorithm is based on the covariance matrix. Specifically, the loss function is typically defined as the difference between the covariance matrix of the data and the predicted covariance matrix of the model. The form of the loss function is:

[0089]

[0090] Where Θ is the causality strength matrix (adjacency matrix) of the causal graph, that is, the edge weight matrix of the causal graph. It is the prediction covariance matrix given the graph structure Θ. It is the Frobenius norm. Let Θ represent the loss function.

[0091] S304: Since the NO-TEARS algorithm finds a graph structure Θ that minimizes a loss function to make the predicted covariance matrix as close as possible to the actual covariance matrix, the optimization problem is to obtain the optimized causal strength matrix. :

[0092]

[0093] in, The L1 regularization term ensures the sparsity of the graph, and λ is the regularization parameter, a hyperparameter learned by the model during training. It is the L1 norm of the adjacency matrix.

[0094] The optimized result is an adjacency matrix. The elements of this matrix represent the direct influence relationships between nodes, i.e., the causal strength matrix. The value of each element indicates the strength of the influence relationship. To obtain a clear causal graph, it is usually necessary to... Thresholding is performed, which involves setting a threshold value, setting elements whose absolute value is less than the threshold to 0, and keeping elements whose absolute value is greater than or equal to the threshold:

[0095]

[0096] The thresholded adjacency matrix (causality strength matrix) Convert to a directed graph (directed edges represent causal relationships), and the non-zero elements in the matrix. This represents the causal relationship from node i to node j. Furthermore, graph visualization tools (such as Graphviz) can be used to convert the adjacency matrix into a graphical causal graph, clarifying the causal relationships between argument roles, trigger words, and event types, and analyzing the influence chain between elements in an event.

[0097] The above has been referenced. Figure 3 This describes how to construct a causal graph to obtain a causal strength matrix of feature data from event text. Next, refer to... Figure 4 ,describe Figure 1 The lower part is the event extraction model, which is a question-answering framework used to obtain answers to questions or identify event arguments.

[0098] Before identifying event arguments, a question template for argument roles needs to be constructed. Then, the answer to the question is obtained through an encoding layer, a self-attention layer, a causal-self-attention constraint layer, and a prediction layer. The encoding layer uses the BERT model to generate word vectors for each word in the event text; the self-attention layer utilizes the interaction information between input texts to focus on the semantic relevance between different parts of sentences; the causal-self-attention constraint layer ensures that the self-attention allocation of the self-attention model follows causal relationships, constraining the self-attention learning process during training and optimizing the attention mechanism; the prediction layer receives feature vectors from the self-attention layer and generates responses to the question and the answer. Next, the event extraction process of this application will be described in detail.

[0099] First, construct a problem template for event arguments based on the type of event, i.e. Figure 2 In step S202, event extraction depends on the event type obtained from event detection. Based on the argument roles defined by the event type, a question template based on argument roles is designed.

[0100] Leveraging the power of the MNEE and ACE2005 datasets, this study first identifies argument roles, categorizing them into time-related, location-related, person-related, and general argument roles. Secondly, it designs appropriate questioning strategies based on these four argument role types. Finally, different Chinese and English argument role questioning templates are designed for the MNEE dataset (a dataset used for neurophysiological signal analysis, typically containing neurophysiological data such as electroencephalograms (EEGs) and magnetoencephalograms (MEGs)) and the ACE2005 dataset (a widely used dataset for event extraction research, containing data for tasks such as entity recognition, values, event expressions, relations, and events). These templates are shown in Table 1.

[0101] Table 1. Templates for the argument role problem in different datasets

[0102]

[0103] Building upon this, and considering that the semantic information of event types can aid in the extraction of argument roles, "in [event type]" is added to the end of the question template to encode event type information. Here, [event type] (or [event type]) is instantiated from the prediction results obtained during the event detection model phase. This constructs a question template that enhances the semantic information of event types. The final question templates for the MNEE and ACE2005 datasets are shown in Tables 2 and 3 below.

[0104] Table 2. Argument Role Question Template for Augmenting Event Type Semantic Information in the MNEE Dataset

[0105]

[0106] Table 3. Argument Role Question Template for Enhancing Event Type Semantic Information in the ACE Dataset

[0107]

[0108] After constructing the question template for event arguments, the question template and the event text are combined and encoded. The encoded sequence is then input into the BERT model, using the hidden states of the last layer as the semantic representation of the input sequence. Figure 2 S203 in the model. The encoding layer uses the BERT model to generate word vectors for each word in the event text. Specifically, when extracting event arguments based on a predefined question template, each input can be instantiated, i.e.:

[0109]

[0110] Here, [CLS] is a special classification marker in BERT, and [SEP] is a special marker representing different statements. The model determines whether each character in the input sentence is the start or end position of the answer for that argument role. More specifically, the model determines whether each character in the input sentence is the start or end position of the answer to the query statement.

[0111] For an input sequence, it can be encoded to obtain an encoded sequence. Where qi represents the vector form of the i-th word in the question template, xi represents the vector form of the i-th word in the event text, m is the length of the question template, and N is the length of the event text. This encoded sequence is input into the BERT model to obtain the hidden state of the last layer. As a semantic representation of the input sequence, N represents the total length of the input text sequence, that is:

[0112] H = BERT(Input)

[0113] After obtaining the semantic representation of the input sequence, the self-attention layer utilizes the interaction information between the input texts to focus on the semantic relevance between different parts of the sentences. That is, it applies a self-attention weight matrix to weight the semantic representation of the input sequence to obtain a weighted output. Figure 2S204 in this paper. Specifically, in an event, argument roles and trigger words influence and interact with each other. This application uses a self-attention mechanism to treat the sentence itself as a query vector, effectively learning the correlation between words, especially the relationship between trigger words and argument roles, and between argument roles themselves. This strengthens the parts useful for predicting argument roles, giving them higher weights and improving the model's extraction performance. The specific implementation is as follows:

[0114]

[0115] Where α is the self-attention weight matrix, Q is the query vector in the attention mechanism, K is the key vector, and V is the value vector, and in the self-attention mechanism... .

[0116] The input sequence H is weighted by a self-attention weight matrix α to enhance the semantic relevance between words, thereby obtaining a weighted output. :

[0117]

[0118] To enhance interpretability, it is also necessary to ensure that the allocation of self-attention in the self-attention layer model follows causal relationships in the causal-self-attention constraint layer, constraining the self-attention learning process during the training phase and optimizing the attention mechanism. Specifically, the causal strength matrix can be compared with the self-attention weight matrix to obtain the causal loss term, i.e. Figure 2 S205 in the document. During the training phase, the self-attention mechanism learns the correlations between words to enhance the semantic representation of sentences. To make the process of generating self-attention weights identical to human thinking logic, focusing on the relationships between key components such as trigger words and argument roles, this application uses causal knowledge to guide its learning process.

[0119] Specifically, in the above references Figure 3 The description already includes a causal graph and a causal strength matrix for the event. Next, we describe the learning process of using the causal knowledge of trigger word-argument roles and argument roles-argument roles in the causal graph, and using its constrained self-attention mechanism to strengthen the part related to predicting argument roles.

[0120] Specifically, for an input sequence H, a certain part of which is H a H b Corresponding argument character or trigger

[0121] The word is , Based on the causal strength matrix, the magnitudes of the causal strengths of these two parts are... The calculation formula is as follows:

[0122]

[0123] By performing a causal strength query on the entire input, the causal strength matrix of that input can be obtained. To ensure comparability of different causal intensities at the same scale, a normalization operation is performed on the causal intensity matrix:

[0124]

[0125] In order to enable the model to effectively learn the causal relationship between trigger words and argument roles in calculating self-attention weights, this application designs a causal loss term. The learning process of the self-attention layer is constrained by causal knowledge, and this loss term is determined by comparing causal strength. and attention weight To achieve this, the specific definition is as follows:

[0126]

[0127] The introduction of the causal loss term aims to enhance the model's understanding of the causal relationships between argument roles and between argument roles and trigger words.

[0128] After obtaining the causal loss term, a loss function can be defined based on the causal loss term to obtain the answer to the question according to the weighted output, i.e. Figure 2 S206 in the example. Specifically, the answer prediction layer receives feature vectors from the self-attention layer and generates a response to the question and an answer. Specifically, the answer prediction layer uses the global representation of the input sequence... Two weight hyperparameter matrices are introduced. (Obtained during model training), calculate the start and end index matrices for the predicted argument roles respectively:

[0129]

[0130] in, This represents the probability that the i-th part of the input is selected as the starting position of the predicted argument role. Let represent the probability that the i-th part of the input is selected as the ending position of the predicted argument role. Therefore, the loss function of the event argument recognition model consists of two parts: the loss at the start position and the loss at the end position, calculated as follows:

[0131]

[0132] in, To predict the starting position and the actual starting position Cross-entropy between To predict the end position and the actual end position Cross-entropy between them.

[0133]

[0134] By incorporating a causal loss term, the loss function is redefined, enabling the model to focus on learning the self-attention weight matrix during the learning process, thereby increasing the model's interpretability without sacrificing performance.

[0135]

[0136] By minimizing the loss function This will give you the start and end positions of the correct answer.

[0137] The current model can calculate the starting position of each word as the correct answer. and the end

[0138] Location The probability is known; however, some questions may have no valid answers or have an unreasonable range of predictions. Therefore, such as... Figure 5 As shown, this application proposes a dynamic threshold algorithm to effectively filter out reasonable candidate answer ranges. The specific steps are as follows:

[0139] S501: Candidate answer generation. First, the probability of the starting position of each word is calculated. and the probability of the ending position Generate all possible candidate answer intervals. .

[0140] S502: Validation of constraints: For each candidate answer interval The following conditions need to be verified:

[0141] and To ensure that both the beginning and end positions are within the sentence, where N

[0142] Sentence length;

[0143] To ensure that the length of the answer does not exceed the preset maximum value;

[0144] Starting position probability and the probability of the ending position The joint probability must be greater than the dynamic threshold Tt.

[0145] S503: Dynamic Threshold Design: The dynamic threshold T can be calculated using the following formula:

[0146]

[0147] in, The output probability for the [CLS] label. Let α represent the overall probability of having no answer. t To adjust the parameters, It is the average probability of all candidates with no answer.

[0148] S504: Filter candidate answers; ultimately, only when... candidate answers Only then will it be retained, among which This represents the joint probability of the starting and ending positions.

[0149] S505: Final answer selection can choose all candidate answers with a relative score of less than 0 for no answer, simplified to:

[0150]

[0151] By using this dynamic threshold design, the algorithm can not only effectively filter out reasonable candidate answers, but also improve the interpretability and adaptability of the model.

[0152] The above has been referenced. Figures 1-5 The content describes the event extraction model based on causal knowledge constraints in this application. Next, refer to... Figure 6 This application describes an event extraction device 600 based on causal knowledge constraints. The event extraction device based on causal knowledge constraints includes:

[0153] A causal graph construction module 601 is used to construct a causal graph of the event text to obtain a causal strength matrix of the feature data of the event text;

[0154] Problem template construction module 602 is used to construct problem templates for event arguments based on the type of the event; and

[0155] Event extraction model 603 includes an encoding layer, which is used to combine the question template and the event text and encode them, and input the encoded sequence into the BERT model so that the hidden state of the last layer is used as the semantic representation of the input sequence.

[0156] The event extraction model 603 also includes a self-attention layer, which applies a self-attention weight matrix to weight the semantic representation of the input sequence to obtain a weighted output;

[0157] The event extraction model 603 further includes a causal self-attention constraint layer, which compares the causal strength matrix with the self-attention weight matrix to obtain a causal loss term; and

[0158] The event extraction model 603 also includes a prediction layer, which defines a loss function in conjunction with the causal loss term to obtain the answer to the question based on the weighted output.

[0159] The functions and construction methods of the cause-effect graph construction module 601, the problem template construction module 602, and the event extraction model 603 have been referenced above. Figures 1-5 The description has already been provided, so I will not repeat it here.

[0160] The above description is merely a specific embodiment of this disclosure, but the scope of protection of this disclosure is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this disclosure should be included within the scope of protection of this disclosure. Therefore, the scope of protection of this disclosure should be determined by the scope of the claims.

Claims

1. An event extraction method based on causal knowledge constraints, the method being used to obtain answers to questions, characterized in that, The method includes: Construct a causal graph of the event text to obtain a causal strength matrix of the feature data of the event text; Construct a problem template for event arguments based on the type of the event; The question template and the event text are combined and encoded, and the encoded sequence is input into the BERT model so that the hidden state of the last layer is used as the semantic representation of the input sequence; The semantic representation of the input sequence is weighted using a self-attention weight matrix to obtain a weighted output; The causal strength matrix is ​​compared with the self-attention weight matrix to obtain the causal loss term; and A loss function is defined based on the causal loss term to obtain the answer to the question according to the weighted output; wherein The semantic representation of the input sequence is weighted using a self-attention weight matrix to obtain a weighted output, including obtaining the self-attention weight matrix α using the following formula: ； Where Q is the query vector in the attention mechanism, K is the key vector, and V is the value vector, and in the self-attention mechanism... And obtain the weighted output through the following formula. : ； Comparing the causal strength matrix with the self-attention weight matrix to obtain the causal loss term includes calculating the causal loss term according to the following formula: ； in, ； The elements of the causal strength matrix are... The elements of the self-attention weight matrix are N, where N is the length of the event text; and Defining a loss function based on the causal loss term to obtain the answer to the question according to the weighted output includes defining the loss function as follows: ； in For the loss at the starting position, The loss is for the ending position, and This represents the causal loss term, and the starting and ending index matrices of the answer are calculated according to the following formula: ； in, This represents the probability that the i-th part of the input is selected as the starting position of the predicted argument role. This represents the probability that the i-th part of the input is selected as the ending position of the predicted argument role. The weight hyperparameter matrix is ​​obtained during model training, and the loss at the start and end positions is calculated using the following formula: ， in, To predict the starting position and the actual starting position Cross-entropy between To predict the end position and the actual end position Cross-entropy between them.

2. The event extraction method based on causal knowledge constraints according to claim 1, characterized in that, Constructing a cause-effect graph of event text includes: Feature data is extracted from the event text to form a feature matrix X; Calculate the covariance matrix S of the feature matrix X to represent the correlation between different feature data; and Minimize the loss function to obtain the causal graph of the event text, wherein the loss function is the difference between the covariance matrix S and the predicted covariance matrix under the given causal graph structure, expressed as follows: ； Where Θ is the causality strength matrix of the causal graph. It is the prediction covariance matrix given the graph structure Θ. It is the Frobenius norm. The loss function represents the cause-effect graph.

3. The event extraction method based on causal knowledge constraints according to claim 2, characterized in that, The method also includes obtaining the optimized causality strength matrix using the following formula. : ； in, The L1 regularization term ensures the sparsity of the graph, and λ is the regularization parameter, a hyperparameter learned by the model during training. It is the L1 norm of the adjacency matrix.

4. The event extraction method based on causal knowledge constraints according to claim 3, characterized in that, The method also includes using the following method to optimize the causality strength matrix. Perform thresholding: Set a threshold for the causal strength matrix Elements whose absolute value is less than the threshold are set to 0, and elements that are greater than or equal to the threshold are retained.

5. The event extraction method based on causal knowledge constraints according to claim 1, characterized in that, Combining the question template and the event text and encoding them includes: Combine the question template and the event text in the following manner: [CLS] Problem Template [SEP] Event Text [SEP] [CLS] is a special classification marker in BERT, and [SEP] is a special marker representing different statements. The encoded sequence is obtained by combining them. , where qi represents the vector form of the i-th word in the question template, xi represents the vector form of the i-th word in the event text, m is the length of the question template, and N is the length of the event text.

6. The event extraction method based on causal knowledge constraints according to claim 1, characterized in that, The method also includes employing a dynamic threshold algorithm to filter out reasonable candidate answer ranges, including: By calculating the probability of the starting position of each word and the probability of the ending position Generate all possible candidate answer intervals. ; For each candidate answer range Verify the following conditions: and To ensure that both the beginning and end positions are within the sentence, where N Sentence length; This is to ensure that the length of the answer does not exceed the preset maximum value; Starting position probability and the probability of the ending position The joint probability must be greater than the dynamic threshold Tt, where ,in, The output probability for the [CLS] label. Let α represent the overall probability of having no answer. t To adjust the parameters, It is the average probability of all candidates with no answer; and reserve candidate answers ,in Represent the joint probability of the start and end positions; and All candidate answers with a relative no-answer score less than 0 are selected as the final answer, where N is the length of the event text.

7. An event extraction device based on causal knowledge constraints, the device being used to obtain answers to questions, characterized in that, The device includes: The causal graph construction module is used to construct a causal graph of the event text to obtain the causal strength matrix of the feature data of the event text; The question template building module is used to construct question templates for event arguments based on the type of the event; and Event extraction model, the event extraction model includes: The encoding layer is used to combine the question template and the event text and encode them, and input the encoded sequence into the BERT model so that the hidden state of the last layer is used as the semantic representation of the input sequence; The self-attention layer applies a self-attention weight matrix to weight the semantic representation of the input sequence to obtain a weighted output; A causal self-attention constraint layer compares the causal strength matrix with the self-attention weight matrix to obtain a causal loss term; and The prediction layer defines a loss function based on the causal loss term to obtain the answer to the question according to the weighted output; wherein The semantic representation of the input sequence is weighted using a self-attention weight matrix to obtain a weighted output, including obtaining the self-attention weight matrix α using the following formula: ； Where Q is the query vector in the attention mechanism, K is the key vector, and V is the value vector, and in the self-attention mechanism... And obtain the weighted output through the following formula. : ； Comparing the causal strength matrix with the self-attention weight matrix to obtain the causal loss term includes calculating the causal loss term according to the following formula: ； in, ； The elements of the causal strength matrix are... The elements of the self-attention weight matrix are N, where N is the length of the event text; and Defining a loss function based on the causal loss term to obtain the answer to the question according to the weighted output includes defining the loss function as follows: ； in For the loss at the starting position, The loss is for the ending position, and This represents the causal loss term, and the starting and ending index matrices of the answer are calculated according to the following formula: ； in, This represents the probability that the i-th part of the input is selected as the starting position of the predicted argument role. This represents the probability that the i-th part of the input is selected as the ending position of the predicted argument role. The weight hyperparameter matrix is ​​obtained during model training, and the loss at the start and end positions is calculated using the following formula: ， in, To predict the starting position and the actual starting position Cross-entropy between To predict the end position and the actual end position Cross-entropy between them.

Images