Brain connection event causal relationship extraction method based on weighted joint constraint learning

Abstract

The invention discloses a brain connection event causal relationship extraction method based on weighted joint constraint learning. The method aims at a text mining task-brain connection event causal relationship extraction in an event-level neural image. According to the method, a novel model named as a brain connection event relation miner is developed by utilizing weighted joint constraint learning. A lightweight model is developed by integrating a BiLSTM (Bidirectional Long Short Term Memory) network and a multilayer perceptron (MLP), and is used for jointly extracting a plurality of event causal relationships from brain connection literatures. In view of scarcity of an annotation brain connection corpus, a weighted joint constraint learning framework is provided. The framework integrates dual consistency constraints, including common sense constraints and domain constraints, and combines the dual consistency constraints with adaptive weight learning to enhance the small sample learning ability of the model.

Description

Technical Field

[0001] The present invention belongs to the field of neuroimaging text mining. Background Art

[0002] Brain science research has developed into the era of brain connectivity, using neuroimaging and electrophysiological techniques to construct macroscopic networks of brain structure and functional connectivity. This approach systematically reveals the complexity of human brain function and connectivity. The surge in articles focusing on brain structure and functional connectivity is obvious. A search on the popular biomedical article database PubMed (https: / / pubmed.ncbi.nlm.nih.gov / ) shows that nearly 5,500 neuroimaging articles on brain connectivity were published in 2022 alone. These articles provide rich scientific evidence and reveal various aspects of brain science, including brain mechanism interpretation, neuroscience knowledge base, system brain fusion computing, etc.

[0003] Neuroimaging text mining is an important method for extracting scientific evidence from neuroimaging articles. A large amount of research work has developed effective neuroimaging text mining techniques, such as topic modeling-based methods and entity recognition-based methods. These studies have greatly promoted our understanding of brain mechanisms and have attracted widespread attention in the field of brain science. For example, as of September 2023, Neurosynth (https: / / www.neurosynth.org / ), an automated tool for scanning and analyzing thousands of articles reporting functional magnetic resonance imaging (fMRI) results, has accumulated 14,371 studies. A query on Web of Science in September 2023 showed that 2,036 academic articles cited Neurosynth's research results. However, these neuroimaging text mining techniques mainly extract terms and their binary relations at the term level. This approach may not fully capture the complexity of brain connectivity research, because compared with early functional area-level brain mechanism research, brain connectivity research has more complex experimental designs and more intricate results. However, extracting event relationships is still crucial to integrating extracted neuroimaging events into a cohesive chain of scientific evidence.

[0004] Event relation extraction is an important and arduous text mining task. A large number of complex models have been developed to extract various types of relations, including temporal relations, causal relations, etc. Most of them are complex deep learning models that rely on large-scale annotated corpora. In order to obtain reliable scientific evidence, the main purpose of event relation extraction in brain connectivity is to identify event causal relationships, such as the causal relationship between symptoms and neuroanatomy and the causal relationship between brain regions. However, event causal relationship extraction in brain connectivity remains a challenging task. As a task of event relation extraction, it lacks a public annotated corpus suitable for training deep neural networks. In addition, it lacks a variety of auxiliary resources for knowledge integration and transfer learning, which are currently the main strategies for event relation extraction in low-resource environments. Summary of the invention

[0005] The present invention proposes a method for extracting causal relationships of brain connection events based on weighted joint constraint learning. It mainly integrates bidirectional long short-term memory networks and multi-layer perceptrons to jointly extract multiple event causal relationships from brain connection literature. In response to the scarcity of annotated brain connection corpus, dual consistency constraint learning is integrated, including common sense constraints and domain constraints, and they are combined with adaptive weight learning to enhance the small sample learning ability of the model.

[0006] The creativity of the present invention is mainly reflected in:

[0007] (1) The present invention obtains a brain connectivity event causal relationship extraction method based on weighted joint constraint learning, which is used to extract event causal relationships of brain connectivity in a low-resource environment. Context embedding is combined with part of speech embedding (POS), and bidirectional long short-term memory (BiLSTM) is used to encode the context features of event pairs to reduce the impact of ambiguous or conflicting information in event mentions. A multi-layer perceptron (MLP) decoder is used to jointly extract the causal relationships of multiple events from brain connectivity articles.

[0008] (2) We develop a joint constraint learning framework to enhance the model's few-shot learning capability and reduce potential errors in relation extraction. Using declarative logic rules, we define common sense and domain consistency constraints and transform them into differentiable loss functions to guide model learning and avoid potential annotation errors. We introduce adaptive weight learning to dynamically adjust the weight of each constraint loss, balance the contributions of different consistency constraints, and reduce the overlap of constraints. BRIEF DESCRIPTION OF THE DRAWINGS

[0009] Figure 1 This is the brain connection event causal relationship extraction model diagram based on weighted joint constraint learning of the present invention. DETAILED DESCRIPTION

[0010] The specific implementation modes of the present invention are further described in detail with reference to the accompanying drawings and examples.

[0011] Step 1: Before implementing text vectorization, data set collection and corpus annotation are required. The specific process is as follows:

[0012] Dataset collection: Raw corpus collection is the first step. 228 article abstracts were crawled from the PubMed central repository using the terms "brain connectomics OR brain connectivity" and the following filtering criteria: article type = "Clinical Trial" & text availability = "Full Text" & dates = "2021 / 10 / 1-2023 / 3 / 31". Some review articles and articles not related to brain connectivity were excluded, and a total of 200 abstracts were selected as the raw corpus. These corpora were further optimized through some preprocessing operations, including merging paragraphs, removing garbled text, extra spaces, and redundant information.

[0013] Corpus annotation: six types of brain connection events and three types of relationships between them are annotated for the collected data sets. Event types include brain connection events (CON): refers to the communication and information interaction between brain regions facilitated by the neural fiber network. These interactions can be detected and depicted using a variety of sophisticated neuroimaging techniques, including functional magnetic resonance imaging (fMRI), electroencephalogram (EEG), electroencephalogram (MEG), etc. Effect events (EFF): refers to the state change of a specific brain activity when performing a specific experimental task, including positive and negative effects. Experimental events (EXP): refers to the entire process of the subject completing the cognitive task. It includes attributes related to the experiment, such as the subject's age and stimulus response pattern. Acquisition events (ACQ): refers to the behavior of the neuroimaging equipment to obtain physiological and psychological data from the subject. Analysis events (ANA): refers to the process of using tools or methods to analyze physiological and psychological data to produce a set of results, such as the default mode network. Result events (RES): refers to the generation or interpretation of analysis results. and causal relationships (CE), cause-effect relationships (EC) and no relationship (NoRel).

[0014] After the text is annotated, the text is embedded and the document is converted into an input vector. BioBERT[1] and part-of-speech encoding (POS encoding) are used to construct these input vectors. A document can be viewed as a sequence of tokens D = {x1, ···, e1, x 12 ,···,e2,···,x 31 ,e3,···,x n}, where some of the tokens E D ={e1,e2,···,e m}(m<n) belongs to the event trigger set of the mark, and the rest are irrelevant morphemes. For each token, its corresponding token embedding expression is:

[0015]

[0016] where {cv1,cv2,…,cv m} denotes the contextualized embedding generated using BioBERT, while {pv1,pv2,…,pv n} represents the POS embedding generated using the POS click vector. Combining all the tag embeddings, the document D can be transformed into the following input vector:

[0017]

[0018] Step 2: The feature extraction layer encodes the contextual features of the event pair based on the vector input from the previous layer. This layer uses a bidirectional long short-term memory network (BiLSTM) and various embedding operations to achieve this.

[0019] First, input vector EW D is fed into the BiLSTM, and then the last hidden state of the BiLSTM is concatenated with the event trigger word e to obtain the event embedding representation, denoted as h e For any event e The embedding can be expressed as:

[0020]

[0021] in Indicates event e i The input vector is represents the output of the forward hidden layer and the backward hidden layer.

[0022] Second, we obtain the event pair (e) by element-wise multiplication and subtraction. i ,e j )(i≠j≤m). The context features are expressed as follows:

[0023]

[0024] in represents the element-wise subtraction operator of the matrix, and ⊙ represents the element-wise multiplication operator.

[0025] We then input the contextual features of the event pairs into the encoder of a multi-layer perceptron (MLP) and train the encoder using contrastive loss to estimate the probability that a relationship holds.

[0026] Step 3 predicts event relations based on the contextual joint features of event pairs. The joint features of event pairs are input into the MLP decoder, and the decoder is trained to estimate the probability that a causal relationship exists between event pairs:

[0027]

[0028] Among them I e represents the hidden layer of the MLP encoder, A h and B h represents the weights and biases of the hidden layer, O e represents the output layer of the MLP encoder, A o and B o Represents the weights and biases of the output layer.

[0029] Based on the description of specific event pairs in the document, the learning objectives of relation prediction are elaborated. The purpose of learning is to enable the model to capture data annotations while standardizing the model to maintain the consistency of logical constraints. Drawing on the logic-driven framework for maintaining the consistency of neural models (Wang et al. [2]), the constraint requirements are clarified by combining common sense knowledge and domain expertise. Therefore, a joint dual-consistency constraint learning framework is introduced to incorporate logical constraints into the encoding learning objectives. The specific content of joint dual-consistency constraint learning will be discussed in detail in the following sections.

[0030] Step 4: Based on the feature description of the basic event pairs in the document, the model is designed to effectively capture data annotations through the learning process of event relationship extraction, and model regularization is implemented to ensure the consistency of logical constraints. There are many kinds of logical constraints between events. For example, if an event pair (e i ,e j ) between which there is a causal relationship (CE), then the event (e j ,e i ) must have a cause-effect relationship (EC) between them. Therefore, step 3 combines a weighted joint consistency constraint learning framework to integrate such logical constraints into the learning objectives. The specific constraint learning and related learning objectives are described as follows:

[0031] Common sense constraints,Common sense constraints are independent of specific domains, and they represent general logical constraints in event relation extraction.,In the task of extracting causal relations of brain connectivity events, two,consistency requirements are adopted as common sense constraints, including annotation consistency,and conjunction consistency. These requirements can be defined by declarative logic rules,and then transformed into differentiable loss functions.

[0032] a. Annotation consistency: If an event pair (e1, e2) is annotated with relation r, then the model should predict this relation:

[0033]

[0034] The symbol ∧ represents logical conjunction (i.e., logical “AND” operation), (e1, e2) represents an event pair, and E D Refers to the set of event triggers that have been annotated. In order to achieve consistency in annotation, the learning objective is set to maximize the probability of the true label and capture inconsistencies with the product t-norm by transforming to the negative logarithmic space. Therefore, the annotation loss can be equivalently defined as the following cross entropy form:

[0035]

[0036] Among them, L ann represents the annotation loss, and r(e1,e2) represents the probability that the event pair (e1,e2) predicted by the model belongs to relation r. By minimizing this loss function, we can train the model to better capture the relationship between events and improve the consistency of annotations.

[0037] b. Conjunction consistency: Suppose there are three related events e1, e2, and e3, which are combined into three event pairs, namely (e1, e2), (e2, e3), and (e1, e3). Not all possible relations can be assigned to these three pairs of events. More specifically, if two relations r1(e1, e2) and r2(e2, e3) apply to the event pairs (e1, e2) and (e2, e3), respectively, then in order to achieve logical consistency of the relations, the following two conjunction rules can be enforced:

[0038] Rule 1: The logical conjunction of relations r1(e1,e2) and r2(e2,e3) can be used to deduce the existence of another relation r3(e1,e3):

[0039]

[0040] in, represents a set of relations, and Represents a The set of all relations in that do not conflict with r1 and r2.

[0041] Rule 2: The logical conjunction of relations r1(e1,e2) and r2(e2,e3) can deduce the non-relation r3(e1,e3):

[0042]

[0043] Among them, the symbol Indicates logical NOT.

[0044] Table 1 is a summary table that describes all conjunction rules for extracting causal relationships between brain connection events. In the table, the leftmost column represents the relationship r1(e1,e2), while the top row represents the relationship r2(e2,e3), CE represents causal relationship, EC represents cause-effect relationship, NoRel represents no relationship, and the symbol - represents uncertainty. Indicates negation. Each cell of the table contains all relations that can be derived from the conjunction of these two relations and their negation, i.e., R(r1,r2), for the event pair (e1,e3). For example, if r1(e1,e2) is a causal relation (CE) and r2(e2,e3) is also a causal relation (CE), then it can be inferred that r3(e1,e3) is also a causal relation (CE).

[0045] Table 1 Summary table of conjunction constraint relations

[0046]

[0047] Similar to annotation consistency, the loss function dedicated to connection consistency can be derived as:

[0048] L con =L con1 +L con2 (11)

[0049]

[0050] Where L con1 and L con2 They represent the conjunction losses corresponding to rule 1 and rule 2 respectively.

[0051] ② Domain constraints: Domain constraints are some logical constraints in event relationship extraction based on prior knowledge in a specific domain. They are proposed based on the types of events and the relationships between event types, and can be defined as the following logical rules.

[0052] Rule 3: If event e1 belongs to event type E1 and event e2 belongs to event type E2, then some negated relation r(E1,E2) can be inferred:

[0053]

[0054] Where E i Indicates event e i Type, Is The set of all relations in which event types E1 and E2 do not conflict

[0055] Table 2 shows a summary table describing all the type exclusion rules needed to extract causal relationships of brain connection events. In the table, the leftmost column represents event type E1, while the top row represents event type E2. Each cell of the table contains all the relation negations that can be excluded based on the event type, i.e. For event pairs (e1, e2). For example, if e1 is an experimental event and e2 is an outcome event, then the possibility that r3(e1, e3) is a causal relationship (EC) can be ruled out. In the table, CON: brain connection event; EFF: effect event; EXP: experimental event; ACQ: acquisition event; ANA: analysis event; RES: outcome event; -: no constraint; Exclude relationship.

[0056] Table 2 Summary of relational constraints of domain constraints

[0057]

[0058] Similarly, the loss for negated consistency is defined as follows:

[0059]

[0060] We have shown above how to formulate logical constraints for the cross entropy loss in different scenarios. Now, we can combine them into a single loss function to jointly optimize the model parameters. The final loss function is:

[0061] L=λ a L ann +λ c L con +λ d L dom (16)

[0062] where λ ★ is a label weight and a non-negative dynamic coefficient that controls the impact of the loss term and can be learned using a dynamic adjustment method [2]. The goal of the dynamic coefficient is to seek to balance the loss of each relation training case. Through continuous iterations, each constraint loss weight will be dynamically adjusted to minimize the overall loss. This approach helps balance the contributions of different consistency constraints to improve the performance of the model.

[0063] References

[0064] [1]J.Lee,W.Yoon,S.Kim,et al.,“BioBERT:a pre-trained biomedicallanguage representationmodel forbiomedical text mining,”Bioinformatics,vol.36,no.4,pp.1234–1240,2020.

[0065] [2]H.Wang,M.Chen,H.Zhang,et al.,“Joint constrained learning forevent-event relationextraction,”in Proceedings of the 2020Conference onEmpirical Methods in NaturalLanguage Processing(EMNLP),pp.696–706,2020.

[0066] [3]Z.Chen,V.Badrinarayanan,C.-Y.Lee,et al.,“Gradnorm:Gradientnormalization foradaptive loss balancing in deep multitask networks,”inProceedings ofthe 35th InternationalConference on Machine Learning,ICML,vol.80,pp.793–802,2018.

Claims

1. A method for extracting causal relationships of brain connection events based on weighted joint constraint learning, characterized in that The following steps are involved: Step 1: To achieve text vectorization, the document is represented as a string of words D = {w1,…,e1,…,w i ,…,e2,…,w j }, some of which belong to the annotation event trigger set E D ={e1,e2,···,e m }(m<n), the others are different phrases, where m is the number of trigger words and n is the number of all phrases; for each word symbol w i , and its corresponding token embedding expression is where {cv1,cv2,…,cv m } represents the contextual embedding generated by the pre-trained BioBER model in the biomedical field, while {pv1,pv2,…,pv n } represents the POS embedding generated using the click vector of the part-of-speech (POS) encoding; combined with all the tag embeddings, the document D is transformed into the following input vector in and Embedded expression representing the corresponding event trigger word; Step 2: In order to extract the features from the text, feature extraction is performed on the vector constructed in step 1. First, the constructed vector V D Input to the bidirectional long short-term memory network (Bidirectional Long Short-Term Memory, BiLSTM), and then concatenate the last hidden state of BiLSTM with the event trigger word to obtain the event e i The embedding is represented as in Indicates event e i The input vector is and represents the output of the forward hidden layer and the backward hidden layer; secondly, the event pair (e i ,e j The context features of )(i≠j≤m) are obtained by element-wise multiplication and subtraction. in represents the element-by-element subtraction operator of the matrix, ⊙ represents the element-by-element multiplication operator; at the same time, the event pair (e i ,e j )’s contextual features are input into the encoder of a multi-layer perceptron (MLP), and the possibility of the relationship between events is estimated by training the encoder with contrastive loss; Step 3: In order to extract the relationship between event pairs, the context feature vector of the event pair is sent to the MLP decoder, and then the decoder is trained to estimate the possibility of a causal relationship between the event pairs: I e =f(h(e i ,e j )A h +B h ) O e =s(I e A o +B o ), Where f(*) represents the linear function of the hidden layer, s(*) represents the linear function of the output layer, and I e represents the hidden layer of the MLP encoder, A h and B h represents the weights and biases of the hidden layer, O e represents the output layer of the MLP encoder, A o and B o Represents the weights and biases of the output layer; Step 4: Based on the feature description of the basic event pairs in the document, the learning process of event relationship extraction is aimed at enabling the model to capture data annotations and implement model regularization to ensure the consistency of logical constraints.

2. The method according to claim 1: the logical constraints described in step 4 are described in detail as follows, implemented by the following constraint logic; defined by declarative logic rules, and then converted into a differentiable loss function; a. Annotation consistency: If an event pair (e1, e2) is annotated as a relation r, the model should predict the event pair as that relation: in, The symbol ∧ represents logical conjunction, i.e., logical "AND" operation, (e1, e2) represents an event pair, and E D Refers to the set of event triggers that have been labeled; in order to achieve consistency in the annotations, the learning objective is set to maximize the probability of the true label, and the inconsistency with the product t-norm is captured by transforming to the negative logarithmic space; b. Conjunction consistency: Assume that there are three related events e1, e2, and e3, which are combined into three event pairs, namely (e1, e2), (e2, e3), and (e1, e3); not all possible relations can be assigned to these three pairs of events; more specifically, if two relations r1(e1, e2) and r2(e2, e3) are applicable to the event pairs (e1, e2) and (e2, e3), respectively, then in order to achieve logical consistency of the relations, the following two conjunction rules are implemented: Rule 1: The logical conjunction of relations r1(e1,e2) and r2(e2,e3) can be used to deduce the existence of another relation r3(e1,e3): in, represents a set of relations, and Represents a The set of all relations in that do not conflict with r1 and r2; Rule 2: The logical conjunction of relations r1(e1,e2) and r2(e2,e3) can deduce the non-relation r3(e1,e3): Among them, the symbol Indicates logical negation; Some logical constraints in event relationship extraction based on prior knowledge in the field of neuroimaging; Develop logical rules based on event types and relationships between event types; Rule 3: If event e1 belongs to event type E1 and event e2 belongs to event type E2, then some negated relation r(E1,E2) can be inferred: Where E i Indicates event e i Type, Is The set of all relations in that do not conflict with event types E1 and E2.

Images