Biological Event Relationship Extraction Method and Electronic Device Integrating Structured Representation and Entity Relationship Reasoning

Abstract

A method and an electronic device for extracting biological event relationships by integrating structured representations and entity relationship reasoning belong to the field of natural language processing. To address issues such as the insufficient utilization of the hierarchical structure information of events in the task of extracting biological event relationships, the key points are to structurally represent the biological events layer by layer according to the three-level units of the biological events to obtain biological event feature representations; infer the relationship paths between the biological entities of the biological event pairs in the biological event relationship dataset; splice the relationship paths with the biological event data in the biological event relationship dataset to obtain a first spliced feature; extract the semantic features of the first spliced feature; splice the biological event feature representation and the semantic features to obtain a second spliced feature; and use a neural network classifier to map the second spliced feature to a relationship space to judge biological event relationships. The effect is that it can accurately extract biological event relationships.

Description

Technical Field

[0001] The present invention belongs to the field of natural language processing and relates to a method for event relationship extraction for biomedical texts, specifically a method for biological event relationship extraction that integrates structured event representation (SER) and entity relationship reasoning (ERR) information and adopts a meta-learning strategy for long-tail distribution. Background Art

[0002] Understanding the complex interactions between biological events at different levels, such as biological molecules, cells, tissues, and organs, is of great significance for revealing the essential laws of life activities, guiding the development of drugs, and treating diseases. These complex interactions are usually discretely distributed in a large amount of biomedical literature. For biomedical researchers, manually obtaining valuable research information from a large amount of literature is a complex and vast project, which seriously restricts the progress of related research. Therefore, information extraction technology for the biomedical field has emerged. In response to the actual needs of biomedical research, from biological named entity recognition, entity relationship extraction to biological event extraction, information extraction research in the biomedical field has gone through a process from easy to difficult, and has achieved relatively rich results in the extraction of biological entity relationships related to proteins, drugs, diseases, and the extraction of biological events occurring at multiple levels such as molecules, cells, tissues, and organs.

[0003] In recent years, with the continuous deepening of research and the improvement of information extraction capabilities in related fields, event relationship extraction and event graph construction, which can obtain richer knowledge, have become hot issues of concern to researchers. The event graph aims to build a knowledge base with event relationships as the core to describe the evolution laws and patterns between events, and high-precision event relationship extraction is the premise and important technical means for building event graphs. In the field of biomedicine, compared with biological entity relationships and biological events, biological event relationships contain more complex and deeper biomedical knowledge, and can more detailedly describe the complex interactive relationships between biological molecules, cells, tissues, organs, etc. Studying biological event relationship extraction and further building a large-scale biological event relationship knowledge base will not only enable researchers in the fields of biomedicine and pharmaceuticals to efficiently obtain accurate, rich, and valuable information, accelerate the progress of related research, but also lay a solid foundation for the final construction of event graphs in the biomedical field.

[0004] Event relation extraction is a difficult point in information extraction. Compared with entity relation extraction and event extraction, the task of event relation extraction is more complex and difficult. Event relation extraction can be understood as taking events as basic semantic units and extracting the logical relations (such as temporal relations, causal relations) occurring between events through the features of the events themselves and other information related to the events (such as the context information in the text where the events are located).

[0005] In the general domain, the early methods for event relation extraction mainly relied on manually defined pattern matching algorithms. However, these methods had high requirements for the researchers' language knowledge in related fields. Moreover, since the ways to express event relations in human language are very diverse and complex, it was difficult to fully generalize them with limited patterns. Therefore, this method could not achieve good results. PROTEUS (Domain Modeling for Language Analysis[J].DomainModeling for Language Analysis, 1988) and COATIS (Coatis, an nlp system to locateexpressions of actions connected by causality links[C].European Workshop onKnowledge Acquisition Springer-Verlag, 1997) were two early systems that used pattern matching methods. C.G Khoo et al. (Using cause-effect relations in text to improve informationretrieval precision[J].Information processing&management, 2001) further developed this method extensively in subsequent research, greatly reducing the dependence on domain knowledge. However, this pattern matching-based method still relied heavily on manually defined features. In subsequent related research, researchers gradually adopted various machine learning algorithms for event relation extraction to automatically extract features from text. In the research of Quoc-Chinh Bui et al. (Extracting causal relations on hiv drug resistance from literature[J].BMCbioinformatics, 2010), the method of logistic regression was used to extract the causal relationship between drugs and virus mutations. Zeng et al. (Relation classification via convolutional deep neural network[C].International Conference on Computational Linguistics, 2014) used convolutional neural networks for multi-relation extraction, and the experimental results showed that CNN was very efficient in such tasks.However, machine learning algorithms such as CNN and LSTM are actually highly dependent on the quality and scale of annotated corpora, which to some extent limits the performance of event relation extraction tasks. In recent years, the proposal of pre-trained language models has enabled researchers to train models using unannotated raw corpora and learn rich human language knowledge in the text, which has greatly reduced the human effort and time cost of data annotation and data acquisition in event relation extraction tasks. Zuo et al. (LearnDA: Learnable knowledge-guided data augmentation for event causality identification[J]. Association for Computational Linguistics, 2021) proposed a causal event relation detection model based on BERT that integrates a learnable data augmentation method, effectively reducing the dependence on high-quality annotated data in event relation extraction tasks. Man et al. (Event causality identification via generation of important context words[C]. North American Chapter of the Association for Computational Linguistics, 2022) proposed an event causality detection method based on the generative language model - T5, achieving the current best results on this task.

[0006] Biological event relation extraction is an extension and expansion of event relation extraction in the biomedical field, which is more complex than in the general domain. Liang et al. (Low Resource Causal Event Detection from Biomedical Literature[C].Association for Computational Linguistics,2022) used various models such as BERT, BioBERT, and BiLSTM to attempt the task of extracting biological event causal relations, and compared them with the rule-based method proposed by Hahn-Powell et al. (This before that:Causal precedence in the biomedical domain[C].Association for Computational Linguistics,2016). The results showed that the BERT model pre-trained with biomedical corpus achieved the best results. Abbas Akkasi and Mari-Francine Moens (Causal relationship extraction from biomedical text using deep neural models:A comprehensive survey[J].Journal of Biomedical Informatics,2021) implemented CNN classification models based on the pre-trained language models ELMO and BioBERT respectively in their research to extract biological causal event relations, and compared them with the LSTM model and the support vector machine model. Both pre-trained language models showed good results. Generally speaking, there are few studies on event relation extraction in the biomedical field at present, mainly focusing on biological causal event relations, and there is a severe lack of publicly available high-quality datasets.

[0007] Through a large number of analyses of biomedical literature, the present invention summarizes the following relevant characteristics of event relation extraction in the biomedical field:

[0008] (1) The structural complexity of biological events. Event relation extraction in the general domain generally only studies simple events, while biological events focus more on the complex interaction between deep biological processes and pathological processes. Therefore, most biological events contain complex structural information, but the event feature representation methods in the general domain cannot fully utilize this structural information.

[0009] (2) There is less available effective context information. Different from the general domain, the distribution of biological events in the literature shows the characteristics of "sparse overall and dense locally". "Sparse overall" means that two events with a relationship may span a relatively long document distance, that is, there will be a large amount of context noise between the two events; while "dense locally" means that multiple events in a sentence share the same context information, making it difficult to distinguish the relationships between events based on context.

[0010] (3) There is a "long-tail" problem in event relationships. Currently, in the general domain, single event relationships are mainly extracted, such as: "coreference", "causality", "temporal sequence", etc. However, there are many types of biological event relationships, and the number of instances of different types of event relationships varies greatly, with an obvious "long-tail" problem, which restricts the extraction accuracy of small-sample event relationship types.

[0011] The above three main characteristics of biological event relationship extraction make the biological event relationship extraction task more complex and difficult compared with the general domain. Therefore, it is necessary to study targeted event relationship extraction methods on the basis of fully analyzing its characteristics. Summary of the Invention

[0012] In order to solve problems such as the hierarchical structure information of events in the biological domain event relationship extraction task cannot be fully utilized and there is less available effective context information between events, according to some embodiments of the present invention, a biological event relationship extraction method that integrates structured representation and entity relationship reasoning includes

[0013] S10. According to the hierarchical structure of biological events, set three-level units for the biological events in the biological event relationship data set;

[0014] S20. Structurally represent the biological events level by level according to the three-level units of the biological events to obtain biological event feature representations;

[0015] S30. Infer the relationship path between the biological entities of the biological event pairs in the biological event relationship data set;

[0016] S40. Concatenate the relationship path with the biological event data in the biological event relationship data set to obtain a first concatenated feature;

[0017] S50. Extract the semantic features of the first concatenated feature;

[0018] S60. Concatenate the biological event feature representation and the semantic feature to obtain a second concatenated feature;

[0019] S70. Map the second concatenated feature to a relationship space through a neural network classifier to judge the biological event relationship.

[0020] A method for extracting biological event relationships that integrates structured representation and entity relationship reasoning according to some embodiments of the present invention, wherein the three-level unit in step S10 includes

[0021] A high-level unit, the high-level unit includes complex events, and the complex events include more than two simple events;

[0022] An elementary unit, including simple events, the simple events include trigger words and biological entities, and the biological entities include entity elements and / or trigger word elements;

[0023] A basic unit, including any one of a trigger word, an entity element, and a trigger word element.

[0024] A method for extracting biological event relationships that integrates structured representation and entity relationship reasoning according to some embodiments of the present invention, the method for structurally representing the biological event according to the three-level unit of the biological event level by level in step S20 to obtain a biological event feature representation includes:

[0025] S21. Encode the biological entities of the basic unit, and the encoded stack obtains a biological entity relationship network in the form of a third-order tensor. The first two dimensions of the tensor of the biological entity relationship network represent the encoding of the biological entity, and the third dimension of the tensor represents the encoding of the biological entity relationship type. Decompose the third-order tensor matrix to obtain three vectors of different dimensions of the biological entity containing the biological entity relationship network information, and encode the information of the three vectors of different dimensions to represent the feature e of the biological entity a , wherein, the decomposition of the third-order tensor matrix is represented by the formula:

[0026]

[0027] In the formula, χ i,j,k represents three vectors of different dimensions of the biological entity containing the biological entity relationship network information, A i,: represents the biological entity feature matrix, X' :,:,k represents the tensor decomposition parameter, represents the biological entity feature matrix, i represents the first dimension, j represents the second dimension, and k represents the third dimension;

[0028] Among them, encoding the information of the three vectors of different dimensions represents the biological entity feature e a of the basic unit, which is represented by the formula:

[0029] e a = χ j χ k χ ia

[0030] where the subscript a represents the a-th biological entity;

[0031] S22. Incorporate the biological entity feature e of the basic unit a into the biological event entity matrix encoded by the BERT network model to obtain the first biological event entity matrix. Input the first biological event entity matrix into the graph neural network to encode the simple events of the elementary units, and the graph neural network outputs the feature representation E of the elementary units s ;

[0032] S23. The complex event of the advanced unit includes i simple events of i elementary units. For the feature representation E i of the elementary unit E s , extract the biological entity matrix P s of the feature representation E i , the interaction relationship identification matrix R s of the event represented by the trigger word, and encode the implicit interaction relationship of the event through the tensor neural network to obtain the biological event feature representation Y. Among them, encoding the implicit interaction relationship of the event through the tensor neural network is represented by the formula:

[0033]

[0034] where F(·) represents the NTN network, T s represents the parameter matrix in the NTN network, W represents the transition probability matrix in the NTN network, and b represents the bias.

[0035] According to the biological event relationship extraction method that fuses structured representation and entity relationship reasoning according to some embodiments of the present invention, infer the relationship path between the biological entities of the biological event pairs in the biological event relationship dataset in step S30. Among them, infer the relationship path between the biological entities of the biological event pairs in the biological event relationship dataset through the method of reinforcement learning, and the reward function of the reinforcement learning model is represented by the formula:

[0036]

[0037]

[0038]

[0039] r all = r Global + r Efficiency + r Diversity

[0040] where r GLOBAL represents the global reward function, r Efficiency represents the efficiency reward function, r DIVERSITYDenote the diversity reward function as r all Denote the final reward function; denote the currently found path as p i Denote the historical path; F represents a hyperparameter, representing the number of paths that the current entity pair has obtained, and e target Denote the target entity.

[0041] According to the method for extracting biological event relationships by integrating structured representation and entity relationship reasoning according to some embodiments of the present invention, the method for splicing the relationship path with the biological event data of the biological event relationship data set in step S40 to obtain the first splicing feature includes: splicing the relationship path after the original text of the biological event data to obtain the first splicing feature, where the biological event data includes the original text, the head event, and the tail event, and the original text contains the context information of the two events.

[0042] According to the method for extracting biological event relationships by integrating structured representation and entity relationship reasoning according to some embodiments of the present invention, the method for extracting the semantic feature of the first splicing feature in step S50 includes: inputting the first splicing feature into the BERT network model for encoding, and the output of the BERT network model is the semantic feature of the first splicing feature.

[0043] According to the method for extracting biological event relationships by integrating structured representation and entity relationship reasoning according to some embodiments of the present invention, if the length of the first splicing feature exceeds 512 after word segmentation and adding [CLS] and [SEP] tags in step 40, the excess part is truncated, and a matrix of [sentence_length, 768] is encoded, where sentence_length <= 512, and each vector of length 768 represents the semantic encoding of a word.

[0044] According to the method for extracting biological event relationships by integrating structured representation and entity relationship reasoning according to some embodiments of the present invention, the method for splicing the biological event feature representation and the semantic feature to obtain the second splicing feature in step S60 includes: splicing the biological event feature representation Y and the semantic feature of the first splicing feature output by the BERT network model to obtain the second splicing feature.

[0045] A method for extracting biological event relationships that integrates structured representation and entity relationship reasoning according to some embodiments of the present invention. The training of the neural network classifier in step S70 includes: constructing a training set through a meta-learning strategy. Each time, randomly extract n categories from the training set, with K samples in each category, for a total of n×K data. Construct a Meta-Task as the input of the support set of the model. Then, randomly extract samples from the remaining data of these n categories as the prediction objects of the model. The model is required to learn to distinguish n categories from the n×K data. Through multiple rounds of training, the model obtains its common part from different Meta-Tasks, including extracting important features and comparing sample similarities, and discarding the task-related parts in the Meta-Tasks. An embodiment of the present application also provides an electronic device, which includes: one or more processors, a memory, and one or more programs; wherein, the one or more programs are stored in the memory, and the one or more programs include instructions. When the instructions are executed by the electronic device, the electronic device executes any possible technical solution designed in the embodiments of the present application.

[0046] Beneficial effects:

[0047] In the first aspect, for the biological event relationship dataset of relationship categories, the present invention processes it through structured event representation, entity relationship path reasoning, and context information feature extraction to obtain vector representations of these information. Next, these vector information are concatenated as the final features of each piece of data, and a training set is constructed using a meta-learning strategy to train the event relationship extraction model. The event relationship extraction task is regarded as a classification task, and a linear neural network is used as the final classification layer, realizing effective extraction of event relationships in the biomedical field.

[0048] In the second aspect, according to the hierarchical structure characteristics of biological events, through a structured event representation method, the obtained biological event feature representation can contain the hierarchical structure information of the event. Through tensor neural network processing, the obtained biological event feature representation contains complex interaction relationship features from the biological entity relationship network, structural features of elementary units of biological events, and structural features of high-level units of biological events.

[0049] In the third aspect, in the preliminary research work of corpus construction in the present invention, it is found that the relationship paths of the fine-grained units (elements) that make up biological events fully contain the relationship features between biological events and can enhance the information for event relationship extraction. Therefore, the present invention introduces high-precision entity relationship information to assist in event relationship reasoning and can solve the problem of less effective context information.

[0050] In the fourth aspect, the present invention adopts a meta-learning strategy for the "long tail" problem in the dataset, enabling the model to have good performance even when facing instances of small-sample biological event relationships.

[0051] In the fifth aspect, the present invention integrates entity path context information and structured event representation information, achieving a macro-average F1 value of 93.61% in the multi-classification task of event relationships, and the macro-average recall rate and precision rate reaching 93.07% and 94.95% respectively (the macro-average means calculating the metrics for each class separately and then taking the average). Brief Description of the Drawings

[0052] Figure 1 : A biological event relationship extraction model that integrates structured representation and entity relationship reasoning. Detailed Embodiments

[0053] Embodiment 1: The present invention provides an extraction method for the characteristics of biological event relationships, realizing the extraction of event relationships from a large number of biomedical texts with marked events, and better solving problems such as the inability to fully utilize the hierarchical structure information of events, the lack of available effective context information between events, and the obvious "long tail" distribution in the data in the event relationship extraction task of the biological field, and obtaining high extraction accuracy.

[0054] The present invention mainly consists of three parts: 1. A structured event representation module; 2. An event relationship extraction module based on entity relationship paths; 3. A meta-learning strategy for the long tail distribution. The steps are as follows:

[0055] Step 1: Dataset Introduction

[0056] The present invention uses the biological event dataset published by Frisoni et al. (Text-to-Text Extraction and Verbalization of Biomedical Event Graphs [C]. Proceedings of the 29th International Conference on Computational Linguistics, 2022), and annotates event relationships on this dataset as a supplement to support the research of the present invention. The specific relationship types and quantity distributions are shown in Table 1.

[0057] Table 1 Dataset Relationship Categories and Quantity Distributions

[0058] Category Quantity Category Quantity consists_of 5309 interconnects 102 co-occurs_with 1167 part_of 91 process_of 442 surrounds 76 precedes 418 affects 71 associated_with 416 location_of 71 produces 383 interacts_with 26 adjacent_to 159 causes 17 occurs_in 158 connected_to 7 result_of 143 tributary_of 4 prevents 143

[0059] Step 2: Construction of a Structured Biological Event Representation Model

[0060] (1) Hierarchy of Biological Events

[0061] Biological events are usually composed of finer-grained basic units, and different combinations of basic units constitute different types of biological events. The present invention defines the hierarchy of biological events as three hierarchical units: high-level units (including complex events, including more than two simple events), elementary units (including simple events, including at least one of a trigger word, an entity element, and a trigger word element), and basic units (including any one of a trigger word, an entity element, and a trigger word element), where a biological entity includes an entity element or a trigger word element.

[0062] (2) Structured Representation Method of Biological Events

[0063] In view of the hierarchical structure characteristics of biological events, the present invention designs a structured event representation method so that the obtained biological event feature representation can contain the hierarchical structure information of the event. The specific steps are as follows:

[0064] ① Characteristics of Basic Units (Biological Entities) of Biological Events

[0065] For basic units containing entity elements and trigger word elements, the entity elements or trigger word elements are encoded, and the encoded stacks form a biological entity relationship network in the form of a third-order tensor. According to the biological entity relationship network, the first two dimensions of the tensor represent the encoding of entity elements or trigger word elements, and the third dimension of the tensor represents the encoding of relationship types;

[0066] The third-order tensor matrix is decomposed using a tensor decomposition algorithm to obtain the information encoding of entity elements or trigger word elements, and the information encoding is the feature representation e of the biological entity a ;

[0067] That is, the present invention first introduces a biological entity relationship network and constructs it in the form of a third-order tensor (the first two dimensions of the tensor are both entity types, and the third dimension represents the relationship type). Then, a tensor decomposition algorithm is used to encode the complex interaction relationships of biological entities in the relationship network. The formal expression of tensor decomposition is shown in the following formula:

[0068]

[0069] χ i,j,k represents a vector of three different dimensions of biological entities containing biological entity relationship network information;

[0070] A i,: represents a biological entity feature matrix;

[0071] X' :,:,k represents tensor decomposition parameters;

[0072] Represents the biological entity feature matrix;

[0073] i represents the first dimension, j represents the second dimension, and k represents the third dimension;

[0074] Pair i,j,k Further calculation is performed, and the calculation formula is as follows:

[0075] e a = x j χ k χ ia

[0076] Among them, the subscript a represents the ath biological entity. Through the above method, the obtained entity feature representation can contain the complex relationship information in the entity relationship network. a .

[0077] ② Characteristics of elementary units (simple events) of biological events

[0078] In obtaining the feature representation of biological entities i After that, the biological event entity matrix encoded by BERT is inserted into the graph neural network (GAT, GCN, GNN) as the input to encode the elementary units (simple events) E that constitute the biological events. The output of the graph neural network is the obtained elementary unit feature representation E s

[0079] ③ Characteristics of higher-level units (complex events) of biological events

[0080] If the biological event to be processed is a complex event composed of two simple events E1 and E2, for the simple event E1, the elementary unit feature representation E2 of the simple event E1 is s Extract entity elements or trigger word element matrix P1, for simple event E2, express the elementary unit feature E2 s Extract entity elements or trigger word elements P2, interaction relationship identifier R between E1 event and E2 event s (the trigger word connecting the two events), the implicit interaction relationship between the three is encoded through the encoding of the tensor neural network, and the final biological event feature representation Y is obtained. s Taking the implicit interaction relationship learning of as an example, the encoding method of the tensor neural network is shown in the following formula:

[0081]

[0082]

[0083] Among them, T sLet \( \mathbf{X} \) represent the parameter matrix in the NTN network, \( F() \) represent the NTN network, the output of the NTN network is the biological event feature representation, \( \mathbf{W} \) represents the transition probability matrix in the NTN network, and \( \mathbf{b} \) represents the bias.

[0084] Compared with the existing biological event representation methods through the above formula, the biological event feature representation obtained by this method contains the complex interaction relationship features from the biological entity relationship network, the structural features of the elementary units of biological events, and the structural features of the advanced units of biological events.

[0085] Step 3: Construction of the entity relationship path inference model based on reinforcement learning

[0086] In the preliminary research work of the corpus construction of the present invention, it is found that the relationship paths of the fine-grained units (elements) that make up biological events fully contain the relationship features between biological events, and can enhance the information for event relationship extraction. Therefore, introducing high-precision entity relationship information to assist in event relationship reasoning is an effective way to solve the problem of less effective context information.

[0087] The present invention uses the method of reinforcement learning to infer the relationship paths between the entity elements of event pairs as auxiliary information for event relationship extraction. The present invention embeds and represents the entity relationship network as the input of the reinforcement learning model, and the reward function of the reinforcement learning model focuses on three aspects: the diversity, accuracy, and length of the found path results. The reward function is as follows:

[0088]

[0089]

[0090]

[0091] \( r \) all \( = r \) Global \( + r \) Efficiency \( + r \) Diversity

[0092] \( r \) GLOBAL refers to the global reward function, which focuses on the accuracy of the path result; \( r \) Efficiency refers to the efficiency reward function, which focuses on the length of the path result; \( r \) DIVERSITY refers to the diversity reward function, which focuses on the repeatability of the path result; \( r \) all refers to the final reward function; \( p \) refers to the currently found path, \( p \) i refers to the historical path; \( F \) is a hyperparameter representing the number of paths that have been obtained for the current entity pair; \( e \) target refers to the target entity.

[0093] After obtaining the entity relationship path, the present invention uses it as an enhanced context information, concatenates it after the original text, and then uses the BERT pre-trained language model to extract semantic features from this text. The BERT model needs to be fine-tuned on the event relationship classification task of the dataset of the present invention before this, so that it can capture more information about event relationships. The input feature used in the classification task is the word embedding representation corresponding to the [CLS] label in the output of the BERT model.

[0094] Step 4: Train the event relationship extraction model using the meta-learning strategy

[0095] For the above biomedical event relationship dataset containing 19 relationship categories, after being processed by the structured event representation, entity relationship path reasoning, and context information feature extraction modules, vector representations of this information will be obtained. Next, these vector information will be concatenated as the final feature of each piece of data, and a training set will be constructed using the meta-learning strategy to train the event relationship extraction model. The present invention regards the event relationship extraction task as a classification task and uses a linear neural network as the final classification layer.

[0096] The present invention provides a method for extracting event relationships from biomedical texts, that is, given a piece of biomedical text and the event pairs annotated in the text, the relationship of this event pair is output. The specific implementation manner of the model is as follows:

[0097] 1. Data example

[0098] Each piece of data in the dataset includes three items: "original text", "head event information", and "tail event information". The "original text" here provides the context information of two events, and the "event information" should at least include the trigger words of two events and their entity parameters. As shown in Table 2, Head Event and Tail Event are the formatted representations of "the expression of TKTL1" and "the expression of LDH5" respectively. Among them, the event category of expression is Gene expression, and the entity categories of TKTL1 and LDH5 are Gene_or_gene_product.

[0099] Table 2 Model input data sample

[0100]

[0101] 2. Structured event representation module

[0102] First, the features of biological entities in the entity relationship network are introduced. After tensor decomposition, they are concatenated into the event entity feature matrix A encoded by BERT. After inputting the entity matrix into the graph neural network, a matrix containing the interaction information of entities within the event is obtained. This matrix is sliced and input into the Neural Tensor Network (NTN) to identify the interaction relationships R between different simple biological events and between events within complex events. s Interaction calculations are performed to obtain the final event representation.

[0103] 3. Entity Relationship Path Reasoning Module

[0104] The basic principle of the reinforcement learning path-finding model adopted in the present invention is to input the entity relationship network node embedding representations of the source node Source and the target node Target, with the expectation of obtaining the entity relationship path between the source node Source and the target node Target. The reward function feeds back a positive or negative incentive to the model according to the success or failure of path-finding, and designs the reward function according to the diversity and length of the found path, with the expectation that the found path can be shorter and more diverse.

[0105] The method for constructing the training set used when training the path-finding model is to combine the entity parameters of the two for a single event pair (Event Pair) in the above-mentioned event relationship data set. Since the event pairs in the data set have semantic relationships, it is considered that there is also a strong correlation between their entity elements. Inputting such entity pairs (Entity Pair) into the reinforcement learning path-finding model can achieve better path-finding results. Negative samples are not set in the training set, that is, all entity pairs in the training set are considered to be able to find paths. Some examples of entity relationship paths are shown in Table 3.

[0106] Table 3 Examples of Entity Relationship Paths

[0107]

[0108] 4. BERT Context Semantic Encoding Module

[0109] After the entity relationship path reasoning is completed, if the reasoning is successful, the obtained path will be input into the "BERT context semantic encoding module" and concatenated after the "original text", and the information of the head event and the tail event will also be concatenated and input into BERT for encoding. Since the original text will generally obtain a relatively long text after concatenating the entity relationship path and event information, and the maximum length of the sentence input into the BERT model is 512, so if the length of this text exceeds 512 after word segmentation and adding [CLS] and [SEP] tags, the approach taken in the present invention is to truncate the excess part. After the encoding is completed, each sentence will obtain a matrix of [sentence_length, 768] (sentence_length <= 512), where each vector of length 768 represents the semantic encoding of a word.

[0110] 5. Train the event relation extraction model using the meta-learning strategy

[0111] The above processing can obtain event structure information (event representation) and context information (entity relationship path) beneficial to event relation extraction. Next, perform the event relation extraction task, concatenate these encodings, and use a neural network classifier to map them to the relation space, and determine which relation exists between two events according to these encoding information. When training this neural network classifier, use the meta-learning strategy to construct the training set to improve the classification performance of the model for small samples. Specifically, randomly select n categories from the training set each time, with K samples for each category (a total of n × K data), and construct a Meta-Task as the support set input of the model. Then randomly select samples from the remaining data of these n categories as the prediction objects of the model (that is, require the model to learn how to distinguish these n categories from n × K data), and through multiple rounds of training, enable the model to obtain the common part from different Meta-Tasks (such as extracting important features and comparing sample similarities, etc.), and discard the task-related part in the Meta-Task. Thus, when the model faces instances of small-sample biological event relations, it can also have good performance.

[0112] After adopting the meta-learning strategy for the "long tail" problem in the dataset and integrating the entity path context information and structured event representation information, finally, the macro-average F1 value of the present invention in the multi-classification task of event relationships reached 93.61%, and the macro-average recall rate and precision rate reached 93.07% and 94.95% respectively (the macro-average means taking the average after calculating the metrics for each category separately). The classification results for each specific category are shown in Table 4. Among them, "gold_num" refers to the number of samples in the test set, "pred_num" refers to the number of samples predicted as this category during the test, and "corr_num" refers to the number of correctly predicted samples.

[0113] Table 4 Evaluation Table of Event Relationship Classification Results

[0114] gold_num pred_num corr_num Precision Recall F1-score interconnects 1 1 1 1.00 1.00 1.00 location_of 4 4 4 1.00 1.00 1.00 part_of 6 6 5 0.83 0.83 0.83 result_of 7 9 7 0.78 1.00 0.88 occurs_in 8 8 8 1.00 1.00 1.00 prevents 10 8 8 1.00 0.80 0.89 associated_with 11 12 10 0.83 0.91 0.87 process_of 12 12 12 1.00 1.00 1.00 adjacent_to 13 10 10 1.00 0.77 0.87 produces 14 13 13 1.00 0.93 0.96 affects 16 16 16 1.00 1.00 1.00 precedes 19 21 19 0.91 1.00 0.95 co-occurs_with 45 36 36 1.00 0.80 0.89 consists_of 205 215 203 0.94 0.99 0.97 MacroAverage - - - 0.95 0.93 0.94

[0115] Based on the above embodiments, the embodiments of the present application further provide an electronic device, which includes: one or more processors, a memory, and one or more programs; wherein, the one or more programs are stored in the memory, and the one or more programs include instructions that, when executed by the electronic device, cause the electronic device to execute the method provided by the above embodiments.

[0116] Based on the above embodiments, the embodiments of the present application further provide a computer storage medium, in which a computer program is stored, and when the computer program is executed by a computer, the computer is caused to execute the method provided by the above embodiments.

[0117] Among them, the storage medium can be any available medium that can be accessed by a computer. Taking this as an example but not limited to: the computer-readable medium can include RAM, ROM, EEPROM, CD-ROM or other optical disc storage, magnetic disk storage medium or other magnetic storage devices, or any other medium that can be used to carry or store the desired program code in the form of instructions or data structures and can be accessed by a computer.

[0118] Those skilled in the art should understand that the embodiments of the present application can be provided as a method, a system, or a computer program product. Therefore, the present application can adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware aspects. Moreover, the present application can adopt the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk memory, CD-ROM, optical memory, etc.) containing computer-usable program code.

[0119] This application is described with reference to the flowcharts and / or block diagrams of methods, apparatus (systems), and computer program products according to the present application. It should be understood that each flow and / or block in the flowcharts and / or block diagrams, and the combinations of flows and / or blocks in the flowcharts and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to the processors of general-purpose computers, special-purpose computers, embedded processors, or other programmable data processing devices to generate a machine, such that the instructions executed by the processors of the computer or other programmable data processing devices produce means for implementing the functions specified in one or more of the flows Figure 1 one or more of the flows and / or blocks Figure 1 or one or more of the blocks.

[0120] These computer program instructions can also be stored in a computer-readable memory that can direct a computer or other programmable data processing device to work in a particular manner, such that the instructions stored in the computer-readable memory produce a manufactured article including instruction means that implement the functions specified in one or more of the flows Figure 1 one or more of the flows and / or blocks Figure 1 or one or more of the blocks.

[0121] These computer program instructions can also be loaded onto a computer or other programmable data processing device, such that a series of operational steps are executed on the computer or other programmable device to generate a computer-implemented process, and thus the instructions executed on the computer or other programmable device provide steps for implementing the functions specified in one or more of the flows Figure 1 one or more of the flows and / or blocks Figure 1 or one or more of the blocks.

[0122] Obviously, those skilled in the art can make various changes and modifications to this application without departing from the spirit and scope of this application. Thus, if these modifications and variations of this application fall within the scope of the claims of this application and their equivalent technologies, this application is also intended to include these changes and modifications.

Claims

1. A method for extracting biological event relationships that integrates structured representation and entity relationship reasoning, including S10. According to the hierarchical structure of biological events, set three-level units for the biological events in the biological event relationship dataset; S20. Structurally represent the biological events level by level according to the three-level units of the biological events to obtain biological event feature representations; S30. Infer the relationship paths between the biological entities of the biological event pairs in the biological event relationship dataset; S40. Concatenate the relationship paths with the biological event data in the biological event relationship dataset to obtain the first concatenated feature; S50. Extract the semantic features of the first concatenated feature; S60. Concatenate the biological event feature representation and the semantic features to obtain the second concatenated feature; S70. Use a neural network classifier to map the second concatenated feature to a relationship space to judge the biological event relationship.

2. The method for extracting biological event relationships that integrates structured representation and entity relationship reasoning according to claim 1, wherein The three-level units in step S10 include High-level units, where high-level units include complex events, and the complex events include more than two simple events; Elementary units, including simple events, where the simple events include trigger words and biological entities, and the biological entities include entity elements and / or trigger word elements; Basic units, including any one of trigger words, entity elements, and trigger word elements.

3. The method for extracting biological event relationships by integrating structured representation and entity relationship reasoning according to claim 2, characterized in that The method of structurally representing the biological events level by level according to the three-level units of the biological events in step S20 to obtain biological event feature representations includes: S21. Encode the biological entities of the basic unit, and the encoded stacks result in a biological entity relationship network in the form of a third-order tensor. The first two dimensions of the tensor of the biological entity relationship network represent the encoding of the biological entities, and the third dimension of the tensor represents the encoding of the biological entity relationship type. Decompose the third-order tensor matrix to obtain three vectors of different dimensions of the biological entity containing the information of the biological entity relationship network, and encode the information of the three vectors of different dimensions to represent the feature e of the biological entity a , where the decomposition of the third-order tensor matrix is represented by the formula: where χ i,j,k represents a vector of three different dimensions in which a biological entity contains biological entity relationship network information, A i,: represents a biological entity feature matrix, X' :,:,k represents tensor decomposition parameters, represents a biological entity feature matrix, i represents the first dimension, j represents the second dimension, and k represents the third dimension; Among them, the biological entity feature e that encodes and represents the basic unit of the vector information in the three different dimensions a is represented by the formula: e a = χ j χ k χ ia In the formula, the subscript a represents the a-th biological entity; S22. Incorporate the biological entity feature e of the basic unit a into the biological event entity matrix encoded by the BERT network model to obtain the first biological event entity matrix. Input the first biological event entity matrix into the graph neural network to encode the simple events of the elementary units, and the graph neural network outputs the feature representation E of the elementary units s ; S23. The complex event of the advanced unit includes i simple events of i elementary units. For the elementary unit E i 's feature representation E s , extract the biological entity matrix P s of feature representation E i , the interaction relationship identification matrix R of the event represented by the trigger word s , encode the implicit interaction relationship of the event through a tensor neural network to obtain the biological event feature representation Y. Among them, encoding the implicit interaction relationship of the event through a tensor neural network is represented by the formula: where \(F(\cdot)\) represents the NTN network, \(T\) s represents the parameter matrix in the NTN network, \(W\) represents the transition probability matrix in the NTN network, and \(b\) represents the bias.

4. The method for extracting biological event relationships by integrating structured representation and entity relationship reasoning according to claim 3, wherein In step S30, infer the relationship paths between the biological entities of the biological event pairs in the biological event relationship dataset. Among them, use the method of reinforcement learning to infer the relationship paths between the biological entities of the biological event pairs in the biological event relationship dataset, and the reward function of the reinforcement learning model is represented by the formula: r all ＝r Global +r Efficiency +r Diversity where r GLOBAL represents the global reward function, r Efficiency represents the efficiency reward function, r DIVERSITY represents the diversity reward function, r all represents the final reward function; p represents the currently found path, p i represents the historical path; F represents a hyperparameter, indicating the number of paths that the current entity pair has obtained, e target represents the target entity.

5. The method for extracting biological event relationships by integrating structured representation and entity relationship reasoning according to claim 3 or 4, characterized in that The method of concatenating the relationship paths with the biological event data in the biological event relationship dataset in step S40 to obtain the first concatenated feature includes: Concatenate the relationship paths after the original text of the biological event data to obtain the first concatenated feature. The biological event data includes the original text, the head event, and the tail event, and the original text contains the context information of the two events.

6. The method for extracting biological event relationships by integrating structured representations and entity relationship reasoning according to claim 5, wherein The method of extracting the semantic features of the first concatenated feature in step S50 includes: Input the first concatenated feature into the BERT network model for encoding, and the output of the BERT network model is the semantic feature of the first concatenated feature.

7. The method for extracting biological event relationships by integrating structured representation and entity relationship reasoning according to claim 6, wherein If the length of the first concatenated feature after word segmentation and adding [CLS] and [SEP] tags in step 40 exceeds 512, truncate the excess part, and encode the resulting matrix of [sentence_length, 768], where sentence_length <= 512, and each vector of length 768 represents the semantic encoding of a word.

8. The method for extracting biological event relationships by integrating structured representation and entity relationship reasoning according to claim 7, wherein, The method of concatenating the biological event feature representation and the semantic feature in step S60 to obtain a second concatenated feature includes: concatenating the biological event feature representation Y and the semantic feature of the first concatenated feature output by the BERT network model to obtain a second concatenated feature.

9. The method for extracting biological event relationships by integrating structured representation and entity relationship reasoning according to claim 7, characterized in that The training of the neural network classifier in step S70 includes: constructing a training set through a meta-learning strategy, randomly extracting n classes from the training set each time, with K samples in each class, for a total of n×K data, constructing a Meta-Task as the input of the support set of the model, and then randomly extracting samples from the remaining data of these n classes as the prediction objects of the model. The model is required to learn to distinguish n classes from the n×K data. Through multiple rounds of training, the model obtains its common part from different Meta-Tasks, including extracting important features and comparing sample similarities, and discarding the task-related parts in the Meta-Tasks.

10. An electronic device, the electronic device comprising: One or more processors, a memory, and one or more programs; wherein, the one or more programs are stored in the memory, and the one or more programs include instructions that, when executed by the electronic device, cause the electronic device to execute the method according to any one of claims 1-9.

Images