Event extraction method and device based on causal knowledge constraint

Abstract

The embodiment of the invention provides a causal knowledge constraint-based event extraction method and device. The method comprises the following steps of: constructing a causal graph of an event text to obtain a causal intensity matrix of feature data of the event text; constructing a problem template of an event argument according to the type of the event; the question template and the event text are combined and coded, and a coded sequence is input into a BERT model so that the hidden state of the last layer can serve as semantic representation of the input sequence; weighting the semantic representation of the input sequence by applying a self-attention weight matrix to obtain a weighted output; comparing the causal intensity matrix with the self-attention weight matrix to obtain a causal loss item; and defining a loss function in combination with the causal loss item to obtain an answer to the question according to the weighted output. Through the processing scheme disclosed by the invention, the interpretability of the model is improved.

Description

Technical Field

[0001] The present invention relates to the field of deep learning technology, and in particular to an event extraction model and device based on causal knowledge constraints. Background Art

[0002] At present, deep learning technology has achieved certain results in event extraction due to its powerful fitting and generalization capabilities. The end-to-end model has achieved remarkable results in event extraction tasks. However, the original intention of artificial intelligence design is to make machines imitate human intelligence when making decisions. Most neural network-based models are not transparent and explainable to decision makers. The model can only give prediction results, but cannot provide the basis behind such predictions. Interpretability will be an indispensable part of machine learning models. Effectively explaining the model, increasing the interpretability of deep learning models, and making neural networks think according to the same logic as the human brain when making decisions are gradually becoming hot issues in academia and industry. Summary of the invention

[0003] In view of this, the embodiments of the present disclosure provide an event extraction method and device based on causal knowledge constraints, which at least partially solve the problems existing in the prior art.

[0004] In a first aspect, an embodiment of the present disclosure provides an event extraction method based on causal knowledge constraints, the method being used to obtain an answer to a question, the method comprising:

[0005] Constructing a causal graph of the event text to obtain a causal intensity matrix of feature data of the event text;

[0006] Construct a question template of event arguments according to the type of event described;

[0007] Combining the question template and the event text and encoding them, and inputting the encoded sequence into a BERT model to use the hidden state of the last layer as a semantic representation of the input sequence;

[0008] Applying a self-attention weight matrix to weight the semantic representation of the input sequence to obtain a weighted output;

[0009] comparing the causal strength matrix with the self-attention weight matrix to obtain a causal loss term; and

[0010] A loss function is defined in conjunction with the causal loss term to obtain an answer to the question based on the weighted output.

[0011] In a specific implementation, constructing a causal graph of an event text includes:

[0012] Extract feature data from the event text to form a feature matrix X;

[0013] Calculating the covariance matrix S of the feature matrix X to express the correlation between different feature data; and

[0014] Minimize the loss function to obtain the causal graph of the event text, where the loss function is the difference between the covariance matrix S and the predicted covariance matrix under a given causal graph structure, expressed as follows:

[0015]

[0016] where Θ is the causal strength matrix of the causal graph, is the predicted covariance matrix under a given graph structure Θ, is the Frobenius norm, Represents the loss function of the causal graph.

[0017] In a specific implementation, the method further includes obtaining an optimized causal strength matrix by the following formula: :

[0018]

[0019] in, is the L1 regularization term to ensure the sparsity of the graph, λ is the regularization parameter, which is a hyperparameter learned by the model during training. is the L1 norm of the adjacency matrix.

[0020] In a specific implementation, the method further includes using the following method to optimize the causal strength matrix Perform thresholding:

[0021] Set a threshold value and transform the causal strength matrix The elements whose absolute values ​​are less than the threshold are set to 0, and the elements whose absolute values ​​are greater than or equal to the threshold are retained.

[0022] In a specific implementation, combining the question template and the event text and encoding them includes:

[0023] The question template and the event text are combined in the following manner:

[0024] [CLS] Question template [SEP] Event text [SEP]

[0025] Where [CLS] is a special classification tag of BERT, [SEP] is a special tag representing different sentences, and the encoding sequence is obtained by combining , where qi represents the vector form of the i-th word segment in the question template, xi represents the vector form of the i-th word segment in the event text, m is the length of the question template, and N is the length of the event text.

[0026] In a specific implementation, applying a self-attention weight matrix to weight the semantic representation of the input sequence to obtain a weighted output includes obtaining a self-attention weight matrix α using the following formula:

[0027]

[0028] Among them, Q is the query vector in the attention mechanism, K is the key vector, V is the value vector, and in the self-attention mechanism , and the weighted output is obtained by :

[0029] .

[0030] In one specific implementation, comparing the causal strength matrix with the self-attention weight matrix to obtain a causal loss term includes calculating the causal loss term according to the following formula:

[0031]

[0032] in,

[0033]

[0034] are the elements of the causal strength matrix, are the elements of the self-attention weight matrix.

[0035] In a specific implementation, a loss function is defined in combination with the causal loss term to obtain an answer to the question according to the weighted output, including defining the loss function according to the following formula:

[0036]

[0037] in is the loss at the starting position, is the loss of the ending position, and is the causal loss term, and the starting and ending subscript matrices of the answer are calculated according to the following formula:

[0038]

[0039] in, represents the probability that the i-th part in the input is selected as the starting position of the predicted argument role, represents the probability that the i-th part in the input is selected as the end position of the predicted argument role, is the weight hyperparameter matrix and is obtained during model training, and the loss at the starting position and the loss at the ending position are calculated by the following formula:

[0040] ,

[0041] in, To predict the starting position and the actual starting position The cross entropy between Predict the end position and the actual end position The cross entropy between .

[0042] In a specific implementation, the method further includes using a dynamic threshold algorithm to screen out a reasonable candidate answer interval, including:

[0043] By calculating the starting position probability of each word and the end position probability , generate all possible candidate answer intervals ;

[0044] For each candidate answer interval , verify the following conditions:

[0045] and , to ensure that the start and end positions are both inside the sentence, where N

[0046] is the sentence length;

[0047] , to ensure that the answer length does not exceed the preset maximum;

[0048] Starting position probability and the end position probability The joint probability must be greater than the dynamic threshold T, where ,in, is the output probability of the [CLS] label, represents the overall probability of no answer, α is the adjustment parameter, is the average probability of all no-answer candidates; and

[0049] reserve Candidate answer for ,in represents the joint probability of the starting position and the ending position; and

[0050] All candidate answers with a relative no-answer score less than 0 are selected as the final answer choices.

[0051] In a second aspect, a device for event extraction based on causal knowledge constraints is provided, the device being used to obtain answers to questions, the device comprising:

[0052] A causal graph construction module, used to construct a causal graph of an event text to obtain a causal intensity matrix of feature data of the event text;

[0053] A question template construction module, used to construct a question template of event arguments according to the type of the event; and

[0054] An event extraction model, wherein the event extraction model comprises:

[0055] An encoding layer, for combining the question template and the event text and encoding them, and inputting the encoded sequence into a BERT model so as to use the hidden state of the last layer as a semantic representation of the input sequence;

[0056] A self-attention layer that applies a self-attention weight matrix to weight the semantic representation of the input sequence to obtain a weighted output;

[0057] a causal self-attention constraint layer that compares the causal strength matrix with the self-attention weight matrix to obtain a causal loss term; and

[0058] The prediction layer defines a loss function in combination with the causal loss term to obtain an answer to the question based on the weighted output.

[0059] In the disclosed embodiment, a method and device for event extraction based on causal knowledge constraints are provided, the method comprising constructing a causal graph of an event text to obtain a causal strength matrix of feature data of the event text; constructing a question template of event arguments according to the type of the event; combining the question template and the event text and encoding them, and inputting the encoded sequence into a BERT model to use the hidden state of the last layer as the semantic representation of the input sequence; applying a self-attention weight matrix to weight the semantic representation of the input sequence to obtain a weighted output; comparing the causal strength matrix with the self-attention weight matrix to obtain a causal loss term; and defining a loss function in combination with the causal loss term to obtain an answer to the question based on the weighted output. The processing scheme disclosed in the present invention improves the interpretability of the model. BRIEF DESCRIPTION OF THE DRAWINGS

[0060] In order to more clearly illustrate the technical solutions of the embodiments of the present disclosure, the drawings required for use in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present disclosure. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0061] Figure 1A schematic diagram of the structure of an event extraction model based on causal knowledge constraints provided in an embodiment of the present disclosure;

[0062] Figure 2 A flowchart of an event extraction method based on causal knowledge constraints provided in an embodiment of the present disclosure;

[0063] Figure 3 A schematic diagram of a process of constructing a causal graph of an event text to obtain a causal strength matrix of feature data of the event text provided by an embodiment of the present disclosure;

[0064] Figure 4 A schematic diagram of the structure of an event extraction model provided in an embodiment of the present disclosure;

[0065] Figure 5 A flow chart of effectively screening out reasonable candidate answer intervals through a dynamic threshold algorithm provided in an embodiment of the present disclosure;

[0066] Figure 6 A schematic diagram of an event extraction device based on causal knowledge constraints provided in an embodiment of the present disclosure. DETAILED DESCRIPTION

[0067] The embodiments of the present disclosure are described in detail below with reference to the accompanying drawings.

[0068] The following describes the embodiments of the present disclosure through specific examples, and those skilled in the art can easily understand other advantages and effects of the present disclosure from the contents disclosed in this specification. Obviously, the described embodiments are only a part of the embodiments of the present disclosure, rather than all of the embodiments. The present disclosure can also be implemented or applied through other different specific embodiments, and the details in this specification can also be modified or changed in various ways based on different viewpoints and applications without departing from the spirit of the present disclosure. It should be noted that the following embodiments and features in the embodiments can be combined with each other without conflict. Based on the embodiments in the present disclosure, all other embodiments obtained by ordinary technicians in the field without making creative work are within the scope of protection of the present disclosure.

[0069] It should be noted that various aspects of the embodiments within the scope of the appended claims are described below. It should be apparent that the aspects described herein may be embodied in a wide variety of forms, and any specific structure and / or function described herein is merely illustrative. Based on the present disclosure, it should be understood by those skilled in the art that an aspect described herein may be implemented independently of any other aspect, and two or more of these aspects may be combined in various ways. For example, any number of aspects described herein may be used to implement the device and / or practice the method. In addition, other structures and / or functionalities other than one or more of the aspects described herein may be used to implement this device and / or practice this method.

[0070] It should also be noted that the illustrations provided in the following embodiments are only schematic illustrations of the basic concept of the present disclosure. The drawings only show components related to the present disclosure rather than being drawn according to the number, shape and size of components in actual implementation. In actual implementation, the type, quantity and proportion of each component may be changed arbitrarily, and the component layout may also be more complicated.

[0071] Additionally, in the following description, specific details are provided to facilitate a thorough understanding of the examples. However, it will be understood by those skilled in the art that the aspects described may be practiced without these specific details.

[0072] This application proposes an event extraction model that uses prior causal knowledge to constrain the attention network. Experimental results show that the model has good interpretability without excessively compromising performance, and its interpretation results are consistent with prior knowledge. Next, with reference to the accompanying drawings, an event extraction model based on causal knowledge constraints in this application is described in detail.

[0073] First, refer to Figure 1 , describing an event extraction model for executing the event extraction method based on causal knowledge constraints of the present application, the model includes a causal graph and an event extraction model, wherein the upper part is the causal graph and the lower part is the event extraction model.

[0074] A causal graph is a graphical representation method for visualizing and analyzing the causal relationship between variables. It represents variables through nodes and indicates the causal relationship between these variables through directed edges. In a causal graph, each node represents a variable, and an arrow pointing from one node to another indicates that the former is the cause of the latter.

[0075] The event extraction model is a natural language processing technology that aims to automatically identify and extract specific types of events and their related attributes from text data. This model is usually built based on machine learning or deep learning algorithms, which can understand the text content and identify the trigger words and participants of the event. The event extraction model first analyzes the text through preprocessing steps (such as word segmentation, part-of-speech tagging, named entity recognition), and then uses techniques such as pattern matching, dependency syntax analysis, or sequence labeling to determine the type and boundaries of the event. Then, the model further extracts detailed information about the event, such as time, location, participants, and the cause, process, and result of the event. Event extraction has important application value in fields such as information retrieval, intelligence analysis, automatic summarization, and question-answering systems. It can help the system understand the text content more deeply, thereby providing richer and more accurate information.

[0076] This application uses Figure 1 The model shown is executed as Figure 2 The event extraction method based on causal knowledge constraints shown in FIG. 1 specifically includes:

[0077] S201: constructing a causal graph of an event text to obtain a causal strength matrix of feature data of the event text;

[0078] S202: constructing a question template of event arguments according to the type of the event;

[0079] S203: combining the question template and the event text and encoding them, and inputting the encoded sequence into a BERT model to use the hidden state of the last layer as the semantic representation of the input sequence;

[0080] S204: applying a self-attention weight matrix to weight the semantic representation of the input sequence to obtain a weighted output;

[0081] S205: Compare the causal strength matrix with the self-attention weight matrix to obtain a causal loss term; and

[0082] S206: Define a loss function in combination with the causal loss term to obtain an answer to the question based on the weighted output.

[0083] Next, refer to Figure 3, describes in detail the process of constructing a causal graph of an event text in the present application to obtain a causal intensity matrix of the characteristic data of the event text. In the present application, the NO-TEARS (No TEquilibriumAssumptions Required) algorithm is used to construct a causal graph, which is mainly used to learn a causal structure graph from observational data, where nodes represent variables and edges represent causal relationships. Unlike traditional methods, NO-TEARS does not rely on equilibrium assumptions or assumptions of structural equation models. However, it should be understood that other causal graph construction methods can also be used, such as expert knowledge-based construction methods, statistical algorithms, structural learning algorithms (such as LiNGAM), hybrid methods (such as combining fuzzy Delphi Method and fuzzy DEMATEL technology), etc., as long as the method can construct a causal graph. The process of constructing a causal graph in the present application specifically includes:

[0084] S301: Extract feature data from event text to form feature matrix X. First, extract relevant feature data from the training set of multiple event texts based on the annotated corpus, such as argument roles, trigger words, event types, etc.; second, preprocess and convert them into binary numerical features, where the presence of trigger words is recorded as 0, otherwise it is recorded as 1, and the same applies to the others. Finally, the processed text data forms a feature matrix , where n is the number of samples and p is the feature dimension.

[0085] S302: Calculate the covariance matrix S of the feature matrix X to express the correlation between different feature data; define the covariance matrix, calculate the covariance matrix S of the feature matrix to express the correlation between different features (or events), and the calculation formula is as follows:

[0086]

[0087] Where n is the number of samples and X is the feature matrix.

[0088] S303: Minimize the loss function to obtain the causal graph of the event text; the loss function of the NO-TEARS algorithm is based on the covariance matrix. Specifically, the loss function is usually defined as the difference between the covariance matrix of the data and the predicted covariance matrix of the model. The form of the loss function is:

[0089]

[0090] Among them, Θ is the causal strength matrix (adjacency matrix) of the causal graph, that is, the weight matrix of the edges of the causal graph, is the predicted covariance matrix under a given graph structure Θ, is the Frobenius norm, Denotes the loss function of Θ.

[0091] S304: Since the NO-TEARS algorithm finds a graph structure Θ by minimizing a loss function so that the predicted covariance matrix is ​​closest to the actual covariance matrix, the optimization problem is used to obtain the optimized causal strength matrix :

[0092]

[0093] in, is the L1 regularization term to ensure the sparsity of the graph, λ is the regularization parameter, which is a hyperparameter learned by the model during training. is the L1 norm of the adjacency matrix.

[0094] The result of optimization is an adjacency matrix , the elements of this matrix represent the direct influence relationship between nodes, that is, the causal strength matrix, and the value of the element represents the strength of the influence relationship. In order to obtain a clear causal diagram, it is usually necessary to Perform threshold processing, that is, set a threshold, set elements with absolute values ​​less than the threshold to 0, and retain elements greater than or equal to the threshold:

[0095]

[0096] The thresholded adjacency matrix (causal strength matrix) Converted to a directed graph (directed edges represent causal relationships), the non-zero elements in the matrix Indicates the causal relationship from node i to node j. And using graph visualization tools (such as Graphviz), the adjacency matrix can be converted into a graphical causal graph to clarify the causal relationship between argument roles, trigger words, and event types, and analyze the influence chain between elements in the event.

[0097] The above has been referenced Figure 3 Describes how to construct a causal graph to obtain a causal strength matrix for feature data of event text. Next, refer to Figure 4 ,describe Figure 1 The event extraction model in the lower middle part is a question-answering based framework, which is used to obtain answers to questions or identify event arguments.

[0098] Before recognizing event arguments, it is necessary to construct a question template of argument roles, and then obtain the answer to the question through the encoding layer, self-attention layer, causal-self-attention constraint layer and prediction layer. The encoding layer uses the BERT model to generate word vectors for each word in the event text; the self-attention layer uses the interactive information between the input texts to focus on the semantic relevance of different parts of the sentences; the causal-self-attention constraint layer ensures that the allocation of self-attention of the self-attention layer model follows the causal relationship, constrains the learning process of self-attention during the training phase, and optimizes the attention mechanism; the prediction layer accepts the feature vector from the self-attention layer and generates a response and answer to the question. Next, the event extraction process of this application is described in detail.

[0099] First, construct a question template for event arguments based on the type of event, namely Figure 2 Step S202 in the event extraction depends on the event type obtained by event detection, and according to the argument role defined by the event type, a question template based on the argument role is designed.

[0100] Based on the MNEE dataset and the ACE2005 dataset, firstly, the role type of the argument is determined, and the arguments are divided into time-related arguments, place-related argument roles, person-related argument roles, and ordinary argument roles; secondly, a reasonable questioning strategy is designed according to these four argument role types; finally, different Chinese and English argument role questioning templates are designed for the MNEE dataset (the MNEE dataset is a dataset used for neuroelectrophysiological signal analysis, usually containing neurophysiological data such as electroencephalogram (EEG) and magnetoencephalogram (MEG)) and the ACE2005 dataset (the ACE2005 dataset is a dataset widely used in event extraction research, which contains data for tasks such as entity recognition, value, event expression, relationship, and event), as shown in Table 1:

[0101] Table 1. Argument role question templates in different datasets

[0102]

[0103] On this basis, considering that the semantic information of event type can help extract argument roles, "in [event type]" is added to the end of the question template to encode the event type information, where [event type] (or [event type]) is instantiated from the prediction result obtained from the event detection model stage. In this way, a question template with enhanced event type semantic information is constructed. Finally, the question templates for the MNEE dataset and ACE2005 dataset are constructed as shown in Tables 2 and 3 below:

[0104] Table 2. Argument role question templates for enhancing event type semantic information in the MNEE dataset

[0105]

[0106] Table 3. Argument role question templates for enhancing event type semantic information in the ACE dataset

[0107]

[0108] After constructing the question template of the event argument, the question template and the event text are combined and encoded, and the encoded sequence is input into the BERT model to use the hidden state of the last layer as the semantic representation of the input sequence, that is, Figure 2 S203 in the code layer. The encoding layer uses the BERT model to generate word vectors for each word in the event text. Specifically, when extracting event arguments based on the defined question template, each input can be instantiated, that is:

[0109]

[0110] Among them, [CLS] is a special classification tag of BERT, and [SEP] is a special tag representing different sentences. The model judges whether each character in the sentence is the starting position or the ending position of the answer to the argument role based on the input sentence. More specifically, the model judges whether each character in the sentence is the starting position or the ending position of the answer to the query sentence based on the input Input.

[0111] For an input sequence, the input sequence can be encoded to obtain an encoded sequence , where qi represents the vector form of the i-th word in the question template, xi represents the vector form of the i-th word in the event text, m is the length of the question template, and N is the length of the event text. This encoding sequence is input into the BERT model to obtain the hidden state of the last layer , as the semantic representation of the input sequence, N represents the total length of the input text sequence, that is:

[0112] H=BERT(Input)

[0113] After obtaining the semantic representation of the input sequence, the self-attention layer uses the mutual information between the input texts to focus on the semantic relevance of different parts of the sentences, that is, the self-attention weight matrix is ​​applied to weight the semantic representation of the input sequence to obtain a weighted output, that is, Figure 2S204 in . Specifically, the argument roles and trigger words in the event influence and interact with each other. This application uses the self-attention mechanism to take the sentence itself as the query vector, effectively learning the correlation between words, especially the correlation between trigger words and argument roles, and between argument roles, so that the part useful for predicting argument roles is strengthened and gets a higher weight, thereby improving the extraction performance of the model. The specific implementation method is as follows:

[0114]

[0115] Among them, α is the self-attention weight matrix, Q is the query vector in the attention mechanism, K is the key vector, V is the value vector, and in the self-attention mechanism .

[0116] Apply the self-attention weight matrix α to weight the input sequence H to strengthen the semantic relevance between words, thereby obtaining a weighted output :

[0117]

[0118] In order to enhance interpretability, it is also necessary to ensure that the self-attention distribution of the self-attention layer model follows the causal relationship in the causal-self-attention constraint layer, constrain the learning process of self-attention in the training phase, and optimize the attention mechanism. Specifically, the causal strength matrix and the self-attention weight matrix can be compared to obtain the causal loss term, that is, Figure 2 S205 in the above. The self-attention mechanism learns the correlation between words during the training phase to strengthen the semantic representation of the sentence. In order to make the process of generating self-attention weights consistent with human thinking logic, and pay attention to the correlation between key parts such as trigger words and argument roles, this application uses causal knowledge to guide its learning process.

[0119] Specifically, in the above reference Figure 3 In the description of the event, the causal graph and causal strength matrix have been obtained. Next, we describe how to use the causal knowledge of trigger word-argument role and argument role-argument role in the causal graph and the learning process of its constrained self-attention mechanism to strengthen the part related to predicting argument roles.

[0120] Specifically, for an input sequence H, a certain part of which is H a , H b , corresponding to the argument role or trigger

[0121] Word , , query based on the causal strength matrix, the causal strength of these two parts is , the calculation formula is as follows:

[0122]

[0123] For the entire input part, a causal strength query is performed to obtain the causal strength matrix of the input , in order to make different causal strengths comparable on the same scale, the causal strength matrix is ​​normalized:

[0124]

[0125] In order to enable the model to effectively learn the causal relationship between trigger words and argument roles in calculating self-attention weights, this application designs a causal loss term , using causal knowledge to constrain the learning process of the self-attention layer. This loss term is calculated by comparing the causal strength and attention weights To achieve this, the specific definition is as follows:

[0126]

[0127] The introduction of the causal loss term aims to strengthen the model's understanding of the causal relationship between argument roles and argument roles, and between argument roles and trigger words.

[0128] After obtaining the causal loss term, a loss function can be defined in combination with the causal loss term to obtain the answer to the question according to the weighted output, that is, Figure 2 Specifically, the answer prediction layer receives the feature vector from the self-attention layer and generates a response to the question and an answer. Specifically, the answer prediction layer generates a response to the question and an answer based on the global representation of the input sequence. , introduce two weight hyperparameter matrices (obtained during model training), calculate the starting and ending subscript matrices of the predicted argument roles respectively:

[0129]

[0130] in, represents the probability that the i-th part in the input is selected as the starting position of the predicted argument role, It represents the probability that the i-th part in the input is selected as the end position of the predicted argument role. Therefore, the loss function of the event argument recognition model consists of two parts: the loss of the starting position and the loss of the ending position. The specific calculation method is as follows:

[0131]

[0132] in, To predict the starting position and the actual starting position The cross entropy between Predict the end position and the actual end position The cross entropy between .

[0133]

[0134] Combined with the causal loss term, the loss function is redefined so that the model can focus on the learning of the self-attention weight matrix during the learning process, increasing the interpretability of the model without losing model performance:

[0135]

[0136] By minimizing the loss function , get the starting and ending positions of the correct answer.

[0137] The current model can calculate the starting position of the correct answer for each word and end

[0138] Location However, some questions may have no valid answers or unreasonable prediction ranges. Figure 5 As shown, this application proposes a dynamic threshold algorithm to effectively screen out reasonable candidate answer intervals. The specific steps are as follows:

[0139] S501: Candidate answer generation, first, by calculating the starting position probability of each word and the end position probability , generate all possible candidate answer intervals .

[0140] S502: Verify constraints: For each candidate answer interval , the following conditions need to be verified:

[0141] and , to ensure that the start and end positions are both inside the sentence, where N

[0142] is the sentence length;

[0143] To ensure that the answer length does not exceed the preset maximum;

[0144] Starting position probability and the end position probability The joint probability must be greater than the dynamic threshold T.

[0145] S503: Dynamic threshold design: The dynamic threshold T can be calculated by the following formula:

[0146]

[0147] in, is the output probability of the [CLS] label, represents the overall probability of no answer, α is the adjustment parameter, is the average probability of all no-answer candidates.

[0148] S504: Screen candidate answers, and finally, only when Candidate answer for will be retained, among which represents the joint probability of the starting position and the ending position.

[0149] S505: Final answer selection, which can select all candidate answers with a relative no-answer score less than 0, can be simplified to:

[0150]

[0151] Through this dynamic threshold design, the algorithm can not only effectively screen out reasonable candidate answers, but also improve the interpretability and adaptability of the model.

[0152] The above has been referenced Figure 1-Figure 5 The content of this paper describes the event extraction model based on causal knowledge constraints of this application. Next, refer to Figure 6 , describing the event extraction device 600 based on causal knowledge constraints of the present application, the event extraction device based on causal knowledge constraints of the present application includes:

[0153] A causal graph construction module 601, wherein the causal graph construction module 601 is used to construct a causal graph of an event text to obtain a causal strength matrix of feature data of the event text;

[0154] A question template construction module 602 is used to construct a question template of event arguments according to the type of the event; and

[0155] The event extraction model 603 includes an encoding layer, which is used to combine the question template and the event text and encode them, and input the encoded sequence into the BERT model to use the hidden state of the last layer as the semantic representation of the input sequence.

[0156] The event extraction model 603 also includes a self-attention layer, which applies a self-attention weight matrix to weight the semantic representation of the input sequence to obtain a weighted output;

[0157] The event extraction model 603 also includes a causal self-attention constraint layer, which compares the causal strength matrix with the self-attention weight matrix to obtain a causal loss term; and

[0158] The event extraction model 603 also includes a prediction layer, which defines a loss function in combination with the causal loss term to obtain an answer to the question based on the weighted output.

[0159] The functions and construction methods of the causal graph construction module 601, the problem template construction module 602, and the event extraction model 603 have been referred to above. Figure 1-Figure 5 The description has been given and will not be repeated here.

[0160] The above is only a specific implementation of the present disclosure, but the protection scope of the present disclosure is not limited thereto. Any changes or substitutions that can be easily thought of by a person skilled in the art within the technical scope disclosed in the present disclosure should be included in the protection scope of the present disclosure. Therefore, the protection scope of the present disclosure should be based on the protection scope of the claims.

Claims

1. An event extraction method based on causal knowledge constraints, the method is used to obtain answers to questions, characterized in that: The method comprises: Constructing a causal graph of the event text to obtain a causal intensity matrix of feature data of the event text; Construct a question template of event arguments according to the type of event described; Combining the question template and the event text and encoding them, and inputting the encoded sequence into a BERT model to use the hidden state of the last layer as a semantic representation of the input sequence; Applying a self-attention weight matrix to weight the semantic representation of the input sequence to obtain a weighted output; comparing the causal strength matrix with the self-attention weight matrix to obtain a causal loss term; and A loss function is defined in conjunction with the causal loss term to obtain an answer to the question based on the weighted output.

2. The event extraction method based on causal knowledge constraints according to claim 1 is characterized in that: Constructing a causal graph for an event text involves: Extract feature data from the event text to form a feature matrix X; Calculating the covariance matrix S of the feature matrix X to express the correlation between different feature data; and Minimize the loss function to obtain the causal graph of the event text, where the loss function is the difference between the covariance matrix S and the predicted covariance matrix under a given causal graph structure, expressed as follows: ； where Θ is the causal strength matrix of the causal graph, is the predicted covariance matrix under a given graph structure Θ, is the Frobenius norm, Represents the loss function of the causal graph.

3. The event extraction method based on causal knowledge constraints according to claim 2 is characterized in that: The method also includes obtaining an optimized causal strength matrix by the following formula : ； in, is the L1 regularization term to ensure the sparsity of the graph, λ is the regularization parameter, which is a hyperparameter learned by the model during training. is the L1 norm of the adjacency matrix.

4. The event extraction method based on causal knowledge constraints according to claim 3 is characterized in that: The method also includes the following method to optimize the causal strength matrix Perform thresholding: Set a threshold value and transform the causal strength matrix The elements whose absolute values ​​are less than the threshold are set to 0, and the elements whose absolute values ​​are greater than or equal to the threshold are retained.

5. The event extraction method based on causal knowledge constraints according to claim 1 is characterized in that: Combining the question template and the event text and encoding them includes: The question template and the event text are combined in the following manner: [CLS] Question template [SEP] Event text [SEP] Where [CLS] is a special classification tag of BERT, [SEP] is a special tag representing different sentences, and the encoding sequence is obtained by combining , where qi represents the vector form of the i-th word segment in the question template, xi represents the vector form of the i-th word segment in the event text, m is the length of the question template, and N is the length of the event text.

6. The event extraction method based on causal knowledge constraints according to claim 1 is characterized in that: Applying a self-attention weight matrix to weight the semantic representation of the input sequence to obtain a weighted output includes obtaining a self-attention weight matrix α using the following formula: ； Among them, Q is the query vector in the attention mechanism, K is the key vector, V is the value vector, and in the self-attention mechanism , and the weighted output is obtained by : 。 7. The event extraction method based on causal knowledge constraints according to claim 6 is characterized in that: Comparing the causal strength matrix with the self-attention weight matrix to obtain a causal loss term includes calculating the causal loss term according to the following formula: ； in, ； are the elements of the causal strength matrix, is the element of the self-attention weight matrix.

8. The event extraction method based on causal knowledge constraints according to claim 7 is characterized in that: Defining a loss function in combination with the causal loss term to obtain an answer to the question according to the weighted output includes defining the loss function according to the following formula: ； in is the loss at the starting position, is the loss of the ending position, and is the causal loss term, and the starting and ending subscript matrices of the answer are calculated according to the following formula: ； in, represents the probability that the i-th part of the input is selected as the starting position of the predicted argument role, represents the probability that the i-th part in the input is selected as the end position of the predicted argument role, is the weight hyperparameter matrix and is obtained during model training, and the loss at the starting position and the loss at the ending position are calculated by the following formula: ， in, To predict the starting position and the actual starting position The cross entropy between Predict the end position and the actual end position The cross entropy between .

9. The event extraction method based on causal knowledge constraints according to claim 8 is characterized in that: The method further includes using a dynamic threshold algorithm to screen out a reasonable candidate answer interval, including: By calculating the starting position probability of each word and the end position probability , generate all possible candidate answer intervals ; For each candidate answer interval , verify the following conditions: and , to ensure that the start and end positions are both inside the sentence, where N is the sentence length; , to ensure that the answer length does not exceed the preset maximum; Starting position probability and the end position probability The joint probability must be greater than the dynamic threshold T, where ,in, is the output probability of the [CLS] label, represents the overall probability of no answer, α is the adjustment parameter, is the average probability of all no-answer candidates; and reserve Candidate answer for ,in represents the joint probability of the starting position and the ending position; and All candidate answers with a relative no-answer score less than 0 are selected as the final answer choices.

10. An event extraction device based on causal knowledge constraints, the device is used to obtain answers to questions, characterized in that: The device comprises: A causal graph construction module, used to construct a causal graph of an event text to obtain a causal intensity matrix of feature data of the event text; A question template construction module, used to construct a question template of event arguments according to the type of the event; and An event extraction model, wherein the event extraction model comprises: An encoding layer, for combining the question template and the event text and encoding them, and inputting the encoded sequence into a BERT model so as to use the hidden state of the last layer as a semantic representation of the input sequence; A self-attention layer that applies a self-attention weight matrix to weight the semantic representation of the input sequence to obtain a weighted output; a causal self-attention constraint layer that compares the causal strength matrix with the self-attention weight matrix to obtain a causal loss term; and The prediction layer defines a loss function in combination with the causal loss term to obtain an answer to the question based on the weighted output.

Images