Model for joint extraction of Chinese electronic medical records

Abstract

The invention discloses a model for joint extraction of Chinese electronic medical records. The model comprises a coding layer encoder, a bidirectional entity pair extraction layer BiEPE and a relation extraction layer RE, wherein the input of the encoding layer encoder is used for inputting statements, the output end of the encoding layer encoder is connected with the input end of the bidirectional entity pair extraction layer BiEPE, the output end of the bidirectional entity pair extraction layer BiEPE is connected with the input end of the relation extraction layer RE, and the output end of the relation extraction layer RE outputs a relation triple. Compared with a traditional joint extraction model, the technical scheme provided by the model introduces Global context mechanism to help the model to obtain more detailed text information, the model can obtain deeper semantics by adding part-of-speech information, and the performance of the model for extracting the relation in the Chinese electronic medical record is improved.

Description

Technical Field

[0001] The present invention relates to the field of computer technology, and in particular to an electronic medical record relationship extraction technology, and specifically to a Chinese electronic medical record relationship joint extraction method based on a global context mechanism. Background Art

[0002] Entity relationship extraction is an important task in natural language processing. Based on the named entity recognition task, it further extracts the relationship between two entities from the text. As the core task of text mining and information extraction, entity relationship extraction mainly extracts the semantic relationship between entity pairs and extracts effective semantic knowledge by modeling text information.

[0003] In the field of medicine, with the rapid development of electronic medical records and clinical information, doctors need information-based means to quickly and accurately obtain patient-related information from a large number of electronic medical records (EMRs) to improve work efficiency. There are two main types of EMRs, namely outpatient medical records and hospitalization medical records. Outpatient medical records are usually short and have less information, so doctors can easily obtain the required information from them; hospitalization records contain a lot of information, such as hospitalization records, medical course records, medical orders, case data, etc. Among them, disease course records are a key part, focusing on the occurrence, evolution and treatment of existing diseases in patients, including a large number of medical entities, and are the key research content of EMRs.

[0004] Traditionally, ternary extraction of Chinese electronic medical records relationships has been studied as two independent tasks: entity extraction and relational prediction. Early work (Zelenko et al., 2002; Chan and Roth, 2011) after extracting all entities, the pipeline method was used to classify the entity pairs in relation. To establish the correlation between these two tasks, the joint model has received widespread attention. Previous feature-based joint models (Yu and Lam, 2010; Li and Ji, 2014; Miwa and Sasaki, 2014; Ren et al., 2017) require a complex feature engineering process and rely on various NLP tools, which are cumbersome to operate manually. Later, the joint extraction task was converted into sequence labeling task, but the overlapping problem was not solved.

[0005] In order to solve the above technical problems, this paper proposes a joint extraction method of Chinese electronic medical record relations based on the global context mechanism. Summary of the invention

[0006] The purpose of the present invention is to effectively solve the problem of overlapping relationships, and to address the limitations of existing Chinese electronic medical record technology, especially the phenomenon of error transmission and insufficient utilization of medical text information, it proposes a joint Chinese electronic medical record extraction technology based on global context mechanism; by introducing a global context mechanism, this technology can more accurately capture complex relationships in medical text, improve the accuracy and comprehensiveness of information extraction, and thus better support clinical decision-making and medical research.

[0007] In order to solve the above technical problems, the technical solution adopted by the present invention is:

[0008] A model for joint extraction of Chinese electronic medical records, which includes an encoding layer encoder, a bidirectional entity-to-extraction layer BiEPE, and a relational extraction layer RE; where the input of the encoding layer encoder is used for input statements, the output end of the encoding layer encoder is connected to the input end of the bidirectional entity-to-extraction layer BiEPE, the output end of the bidirectional entity-to-extraction layer BiEPE is connected to the input end of the relational extraction layer RE, and the output end of the relational extraction layer RE outputs a relation triple.

[0009] The internal structure of the encoder layer includes the pre-trained language model BERT, the bidirectional long short-term memory network BiLSTM, and the global context mechanism;

[0010] The input of BERT is used to input sentences, the output of BERT is connected to the input of BiLSTM, the output of BiLSTM is connected to the input of Global Context Mechanism, and the output of Global Context Mechanism outputs subject representation, object representation and relationship representation.

[0011] The internal structure of the bidirectional entity pair extraction layer BiEPE includes the subject tagging module Subject Tagger, the object tagging module Object Tagger, the subject-based object tagging module Subject-based Object Tagger, and the object-based subject tagging module Object-based Subject Tagger;

[0012] The input of the subject tagging module Subject Tagger is used for subject representation, the output of Subject Tagger is connected to the input of Subject-based Object Tagger, and the output of Subject-based Object Tagger outputs subject-object pairs;

[0013] The input of the object tagging module Object Tagger is used for object representation, the output end of Object Tagger is connected to the input end of Object-based Subject Tagger, and the output end of Object-based Subject Tagger outputs object-subject pairs.

[0014] The internal structure of the relation extraction layer RE includes a relation-based biaffine module Biaffine-based RE, where the input of the Biaffine-based RE is used for all possible subject-object pairs, and the output end of the Biaffine-based RE outputs relation triples.

[0015] When the model is working, the following steps are taken:

[0016] Step 1: Preprocess the data.

[0017] Step 2: Input the preprocessed data into BERT to obtain text representation;

[0018] Step 3: Send the text representation into BiLSTM to obtain forward LSTM and backward LSTM;

[0019] Step 4: Send the forward LSTM and backward LSTM to the global context mechanism to integrate the sentence information and obtain more complete context information;

[0020] Step 5: Use context information to obtain the subject and object in parallel;

[0021] Step 6: Use the subject and its part of speech to match the object, and similarly use the object and its part of speech to match the subject;

[0022] Step 7: Use Biaffine to identify the subject-object pair and obtain the final entity relationship triple;

[0023] Through the above steps, relations are extracted from Chinese electronic medical records.

[0024] Step 1 specifically includes the following steps:

[0025] Step 1-1: Preprocess the acquired data set to remove special characters in the text; use the word segmentation tool to segment the text data and obtain the part of speech corresponding to the words in the word segmentation results; remove words that are not verbs, nouns, adjectives, and adverbs in the data;

[0026] Step 2 specifically includes the following sub-steps:

[0027] Step 2-1: Input the preprocessed data into the pre-trained language model BERT to convert the text into text encoding;

[0028] Step 3 specifically includes the following sub-steps:

[0029] Step 3-1: Input the obtained text encoding into the BiLSTM layer to obtain the forward LSTM and backward LSTM. The calculation formula is as follows:

[0030]

[0031] in, refers to the forward LSTM sentence representation at the t-th position; Refers to forward LSTM; refers to the sentence representation of the t-1th forward LSTM; z t refers to the sentence representation at the t-th position; refers to the backward LSTM sentence representation at the t-th position; Refers to backward LSTM; refers to the forward LSTM sentence representation at the t+1th position.

[0032] Step 4 specifically includes the following sub-steps:

[0033] Step 4-1: Merge the sentence representations of the first and last cells of the obtained BiLSTM to obtain Represent the entire sentence with H t Merge to get O t =G||H t ;

[0034] Among them, G refers to the entire sentence representation; refers to the sentence representation of the first cell; refers to the sentence representation of the nth cell;

[0035] Step 4-2: Generate i using the gating mechanism t H and i t G , the specific calculation formula is as follows:

[0036] R H =W H O t +b H

[0037] R G =W G O t +b G

[0038] i t H =sigmoid(R t H )

[0039] i t G =sigmoid(R t G )

[0040] Where W H and W G ∈R 2d×d ; R t G , R t H Represents the global information G and the current sentence representation H respectively t , sigmoid is the activation function; i t H Refers to the assignment to H t The weight of i t G refers to the weight assigned to G, R H Refers to H t The sentence indicates that W H and W G refers to the weight, O t refers to the sentence representation of the tth position that incorporates global information, b H and b G refers to the bias vector;

[0041] Step 4-3: Through H t Fusion with G t H and i t G Generate global context information, the calculation formula is as follows:

[0042]

[0043] in represents global context information, ⊙ represents element-wise product;

[0044] Step 5 specifically includes the following sub-steps:

[0045] Step 5-1: Generate three different token representation sequences from the acquired global context information as the context representations of subject, object and relation, which are O i s , O i o , O ir , the calculation formula is as follows:

[0046] O i s =W s O i +b s

[0047] O i o =W o O i +b o

[0048] O i r =W r O i +b r

[0049] in is a trainable matrix, is the bias vector, d h It represents the dimension;

[0050] W s refers to the subject weight, O i refers to the sentence representation at the ith position, b s refers to subject bias; W o refers to the object weight, b o Refers to object bias; W r refers to the relationship weight, b r It refers to relational bias.

[0051] Step 6 specifically includes the following sub-steps:

[0052] Step 6-1: Subject Tagger is a binary tagging-based module that extracts all subjects from the input sentence. For each subject in the input sentence, two probabilities are assigned to represent the likelihood of the start and end markers. The probability calculation formula is as follows:

[0053] p i，start s =σ(W start s O i s +b start s )

[0054] p i，end s =σ(W end s O i s +bend s )

[0055] where p i，start s and p i，end s denote the probability of the i-th token being the start and end of the subject, respectively. and is a trainable matrix representing weights, O i s represents the subject sentence representation of the ith token, b start s ∈R 1 and b end s ∈R 1 is the bias vector; σ represents the sigmoid activation function;

[0056] Step 6-2: Subject-based Object Tagger is used to extract the corresponding object based on the extracted subject; the obtained part-of-speech information is used to assist the subject in matching the corresponding object, and two probabilities are also assigned to represent the possibility of the start and end marks of the corresponding object. The probability calculation formula is as follows:

[0057] p i，start o =σ(W start o (O i o *v s_k s )+b start o )

[0058] p i，end o =σ(W end o (O i o *v s_k s )+b end o )

[0059] where v s_k s represents the kth subject representation that incorporates the part of speech, p i，start o and p i，end o Respectively represent the probabilities of the start and end tags of the object associated with the kth subject; * represents the multiplication of the corresponding elements, and is a trainable matrix representing weights, b start o ∈R 1 and b end o ∈R 1 is a bias vector, O i o represents the object sentence representation of the i-th token, σ represents the sigmoid activation function;

[0060] Step 7 specifically includes the following sub-steps:

[0061] Step 7-1: The BiEPE module outputs many subject-object pairs. The Biaffine module is used to identify each subject-object pair to obtain the final relation triple. The calculation formula is as follows:

[0062] v s_k r =maxpool(O s_k_start r , ..., O s_k_end r )

[0063] v o_j r =maxpool(O o_j_start r , ..., O o_j_end r )

[0064]

[0065] where v s_k r and v o_j r Represents the representation vectors of the two entities, O s_k_start r and O s_k_end r The start and end vectors of the kth subject are represented by O o_j_start r and O o_j_end r represents the start and end vector representation of the jth object, p i r Indicates (s k , o j ) has the possibility of the i-th relationship, is the parameter matrix of the i-th relation, and maxpool is the maximum pooling operation.

[0066] Compared with the prior art, the present invention has the following technical effects:

[0067] 1) Compared with the traditional joint extraction model, the present invention provides the model with more text context information, including the association and logical coherence between sentences, by adding the global context mechanism, which strengthens the model's understanding of the electronic medical record text and improves the model's performance in extracting entity relationships in the electronic medical record;

[0068] 2) In the present invention, subject part of speech is added to help the subject match the object, and object part of speech is added to help match the subject, so that the model can better capture the relationship between entities, so that the model can more accurately understand the text content and improve the accuracy of relationship extraction. BRIEF DESCRIPTION OF THE DRAWINGS

[0069] The present invention will be further described below in conjunction with the accompanying drawings and embodiments:

[0070] Figure 1 It is a schematic diagram of the process of the present invention.

[0071] Figure 2 This is a model structure diagram of the present invention. DETAILED DESCRIPTION

[0072] like Figure 1 , Figure 2 As shown, a model for joint extraction of Chinese electronic medical records includes an encoding layer encoder1, a bidirectional entity pair extraction layer BiEPE2, and a relationship extraction layer RE3; wherein the input of the encoding layer encoder1 is used to input sentences, the output end of the encoding layer encoder1 is connected to the input end of the bidirectional entity pair extraction layer BiEPE2, the output end of the bidirectional entity pair extraction layer BiEPE2 is connected to the input end of the relationship extraction layer RE3, and the output end of the relationship extraction layer RE3 outputs relationship triples.

[0073] The internal structure of the encoding layer encoder1 includes the pre-trained language model BERT101, the bidirectional long short-term memory network BiLSTM102, and the global context mechanism Global Context Mechanism103;

[0074] The input of BERT101 is used to input sentences, the output of BERT101 is connected to the input of BiLSTM102, the output of BiLSTM102 is connected to the input of Global Context Mechanism103, and the output of GlobalContext Mechanism103 outputs subject representation, object representation and relationship representation.

[0075] The internal structure of the bidirectional entity pair extraction layer BiEPE2 includes a subject tagging module Subject Tagger201, an object tagging module Object Tagger202, a subject-based object tagging module Subject-based ObjectTagger203, and an object-based subject tagging module Object-based Subject Tagger204;

[0076] The input of the subject tagging module Subject Tagger201 is used for subject representation, the output end of Subject Tagger201 is connected to the input end of Subject-based Object Tagger203, and the output end of Subject-based Object Tagger203 outputs a subject-object pair;

[0077] The input of the object tag module Object Tagger 202 is used for object representation, the output end of Object Tagger 202 is connected to the input end of Object-based Subject Tagger 204, and the output end of Object-based Subject Tagger 204 outputs an object-subject pair.

[0078] The internal structure of the relation extraction layer RE3 includes a relation-based biaffine module Biaffine-based RE, where the input of the Biaffine-based RE is used for all possible subject-object pairs, and the output end of the Biaffine-based RE outputs relation triples.

[0079] When the model is working, the following steps are taken:

[0080] Step 1: Preprocess the data.

[0081] Step 2: Input the preprocessed data into BERT to obtain text representation;

[0082] Step 3: Send the text representation into BiLSTM to obtain forward LSTM and backward LSTM;

[0083] Step 4: Send the forward LSTM and backward LSTM to the global context mechanism to integrate the sentence information and obtain more complete context information;

[0084] Step 5: Use context information to obtain the subject and object in parallel;

[0085] Step 6: Use the subject and its part of speech to match the object, and similarly use the object and its part of speech to match the subject;

[0086] Step 7: Use Biaffine to identify the subject-object pair and obtain the final entity relationship triple;

[0087] Through the above steps, relations are extracted from Chinese electronic medical records.

[0088] Step 1 specifically includes the following steps:

[0089] Step 1-1: Preprocess the acquired data set to remove special characters in the text; use the word segmentation tool to segment the text data and obtain the part of speech corresponding to the words in the word segmentation results; remove words that are not verbs, nouns, adjectives, and adverbs in the data;

[0090] Step 2 specifically includes the following sub-steps:

[0091] Step 2-1: Input the preprocessed data into the pre-trained language model BERT to convert the text into text encoding;

[0092] Step 3 specifically includes the following sub-steps:

[0093] Step 3-1: Input the obtained text encoding into the BiLSTM layer to obtain the forward LSTM and backward LSTM. The calculation formula is as follows:

[0094]

[0095] in, refers to the forward LSTM sentence representation at the t-th position; Refers to forward LSTM; refers to the sentence representation of the t-1th forward LSTM; z t refers to the sentence representation at the t-th position; refers to the backward LSTM sentence representation at the t-th position; Refers to backward LSTM; refers to the forward LSTM sentence representation at the t+1th position.

[0096] Step 4 specifically includes the following sub-steps:

[0097] Step 4-1: Merge the sentence representations of the first and last cells of the obtained BiLSTM to obtain Represent the entire sentence with H t Merge to get O t =G||H t ;

[0098] Among them, G refers to the entire sentence representation; refers to the sentence representation of the first cell; refers to the sentence representation of the nth cell;

[0099] Step 4-2: Generate i using the gating mechanism t H and i t G , the specific calculation formula is as follows:

[0100] R H =W H O t +b H

[0101] R G =W G O t +b G

[0102] i t H =sigmoid(R t H )

[0103] i t G =sigmoid(R t G )

[0104] Where W H and W G ∈R 2d×d ; R t G , R t H Represents the global information G and the current sentence representation H respectively t , sigmoid is the activation function; i t H Refers to the assignment to H t The weight of i t G refers to the weight assigned to G, R H Refers to H t The sentence indicates that W H and W G refers to the weight, O t refers to the sentence representation of the tth position that incorporates global information, b H and b G refers to the bias vector;

[0105] Step 4-3: Through H t Fusion with Gt H and i t G Generate global context information, the calculation formula is as follows:

[0106]

[0107] in represents global context information, ⊙ represents element-wise product;

[0108] Step 5 specifically includes the following sub-steps:

[0109] Step 5-1: Generate three different token representation sequences from the acquired global context information as the context representations of subject, object and relation, which are O i s , O i o , O i r , the calculation formula is as follows:

[0110] O i s =W s O i +b s

[0111] O i o =W o O i +b o

[0112] O i r =W r O i +b r

[0113] in is a trainable matrix, is the bias vector, d h It represents the dimension;

[0114] W s refers to the subject weight, O i refers to the sentence representation at the ith position, b s refers to subject bias; W o refers to the object weight, b o Refers to object bias; W r refers to the relationship weight, b r It refers to relational bias.

[0115] Step 6 specifically includes the following sub-steps:

[0116] Step 6-1: Subject Tagger is a binary tagging-based module that extracts all subjects from the input sentence. For each subject in the input sentence, two probabilities are assigned to represent the likelihood of the start and end markers. The probability calculation formula is as follows:

[0117] p i，start s =σ(W start s O i s +b start s )

[0118] p i，end s =σ(W end s O i s +b end s )

[0119] where p i，start s and p i，end s denote the probability of the i-th token being the start and end of the subject, respectively. and is a trainable matrix representing weights, O i s represents the subject sentence representation of the ith token, b start s ∈R 1 and b end s ∈R 1 is the bias vector; σ represents the sigmoid activation function;

[0120] Step 6-2: Subject-based Object Tagger is used to extract the corresponding object based on the extracted subject; the obtained part-of-speech information is used to assist the subject in matching the corresponding object, and two probabilities are also assigned to represent the possibility of the start and end marks of the corresponding object. The probability calculation formula is as follows:

[0121] p i，start o =σ(W start o (O i o *v s_k s )+b start o )

[0122] p i，end o =σ(W end o (O i o *v s_k s )+b end o )

[0123] where v s_k s represents the kth subject representation that incorporates the part of speech, p i，start o and p i，end o Respectively represent the probabilities of the start and end tags of the object associated with the kth subject; * represents the multiplication of the corresponding elements, and is a trainable matrix representing weights, b start o ∈R 1 and b end o ∈R 1 is a bias vector, O i o represents the object sentence representation of the i-th token, σ represents the sigmoid activation function;

[0124] Step 7 specifically includes the following sub-steps:

[0125] Step 7-1: The BiEPE module outputs many subject-object pairs. The Biaffine module is used to identify each subject-object pair to obtain the final relation triple. The calculation formula is as follows:

[0126] v s_k r =maxpool(O s_k_start r , ..., O s_k_end r )

[0127] v o_j r =maxpool(O o_j_start r , ..., O o_j_end r )

[0128]

[0129] where v s_k r and v o_jr Represents the representation vectors of the two entities, O s_k_start r and O s_k_end r The start and end vectors of the kth subject are represented by O o_j_start r and O o_j_end r represents the start and end vector representation of the jth object, p i r Indicates (s k , o j ) has the possibility of the i-th relationship, is the parameter matrix of the i-th relation, and maxpool is the maximum pooling operation.

[0130] Example:

[0131] On the diabetes dataset, the performance of the proposed model method is compared with the following baseline models:

[0132] Model Precision(%) Recall(%) F1(％) PRGC 73.169 64.648 68.645 SPN 77.200 63.175 69.487 GRTE 81.687 64.975 72.084 <![CDATA[CASREL ERNIE ]]> 79.452 66.448 72.371 <![CDATA[AMFRel ERNIE ]]> 83.914 67.021 74.145 BiRTE 79.516 79.774 79.645 Our 83.487 80.583 82.009

[0133] The above table lists the results of the model of the present invention and other baseline models on all data sets. It can be observed that the present invention outperforms the comparison models in all evaluation indicators. It is worth noting that the model of the present invention has achieved a 2.364% improvement over BiRTE on the diabetes data set. The reason is that the new model proposed in the present invention integrates the Global context mechanism and adds part-of-speech information to assist matching, which enables the model to have stronger analysis capabilities and context information capture capabilities when processing complex text data.

[0134] Ablation experiment comparison:

[0135]

[0136] It can be seen from the above table:

[0137] (1) Removing the global context mechanism will reduce the model's performance in extracting triples. This is because the global context mechanism can provide more information about the text context, including the association and logical coherence between sentences. Removing this mechanism will cause the model's understanding of the text to become more limited, thereby reducing the model's performance in the triple extraction task.

[0138] (2) After removing the part-of-speech features, the global performance of the model deteriorates. This is because the entities and relations in the medical triples are basically nouns or verbs. Removing the part-of-speech information will cause the model to be unable to obtain deeper semantic features and lose certain semantic information when understanding the text, resulting in performance degradation.

Claims

1. A model for joint extraction of Chinese electronic medical records, characterized in that: The model includes an encoding layer encoder (1), a bidirectional entity pair extraction layer BiEPE (2), and a relation extraction layer RE (3); wherein the input of the encoding layer encoder (1) is used to input sentences, the output end of the encoding layer encoder (1) is connected to the input end of the bidirectional entity pair extraction layer BiEPE (2), the output end of the bidirectional entity pair extraction layer BiEPE (2) is connected to the input end of the relation extraction layer RE (3), and the output end of the relation extraction layer RE (3) outputs relation triples.

2. The model according to claim 1, characterized in that The internal structure of the encoding layer encoder (1) includes a pre-trained language model BERT (101), a bidirectional long short-term memory network BiLSTM (102), and a global context mechanism Global Context Mechanism (103); The input of BERT (101) is used to input a sentence, the output of BERT (101) is connected to the input of BiLSTM (102), the output of BiLSTM (102) is connected to the input of Global Context Mechanism (103), and the output of Global Context Mechanism (103) outputs a subject representation, an object representation, and a relation representation.

3. The model according to claim 1, characterized in that The internal structure of the bidirectional entity pair extraction layer BiEPE (2) includes a subject tagging module Subject Tagger (201), an object tagging module Object Tagger (202), a subject-based object tagging module Subject-based Object Tagger (203), and an object-based subject tagging module Object-basedSubject Tagger (204); The input of the subject tagging module Subject Tagger (201) is used for subject representation, the output end of the Subject Tagger (201) is connected to the input end of the Subject-based Object Tagger (203), and the output end of the Subject-based Object Tagger (203) outputs a subject-object pair; The input of the object tag module Object Tagger (202) is used for object representation, the output end of the Object Tagger (202) is connected to the input end of the Object-based Subject Tagger (204), and the output end of the Object-based Subject Tagger (204) outputs an object-subject pair.

4. The model according to claim 1, characterized in that The internal structure of the relation extraction layer RE (3) includes a relation-based biaffine module Biaffine-based RE, where the input of the Biaffine-based RE is used for all possible subject-object pairs, and the output of the Biaffine-based RE outputs relation triples.

5. The model according to claim 1, characterized in that When the model is working, the following steps are adopted: Step 1: Preprocess the data. Step 2: Input the preprocessed data into BERT to obtain text representation; Step 3: Send the text representation into BiLSTM to obtain forward LSTM and backward LSTM; Step 4: Send the forward LSTM and backward LSTM to the global context mechanism to integrate sentence information and obtain more complete context information; Step 5: Use context information to obtain the subject and object in parallel; Step 6: Use the subject and its part of speech to match the object, and similarly use the object and its part of speech to match the subject; Step 7: Use Biaffine to identify the subject-object pair and obtain the final entity relationship triple; Through the above steps, relations are extracted from Chinese electronic medical records.

6. The model according to claim 5, characterized in that Step 1 specifically includes the following steps: Step 1-1: Preprocess the acquired data set to remove special characters in the text; use the word segmentation tool to segment the text data and obtain the part of speech corresponding to the words in the word segmentation results; remove words that are not verbs, nouns, adjectives, and adverbs in the data; Step 2 specifically includes the following sub-steps: Step 2-1: Input the preprocessed data into the pre-trained language model BERT to convert the text into text encoding; Step 3 specifically includes the following sub-steps: Step 3-1: Input the obtained text encoding into the BiLSTM layer to obtain the forward LSTM and backward LSTM. The calculation formula is as follows: in, refers to the forward LSTM sentence representation at the t-th position; Refers to forward LSTM; refers to the sentence representation of the t-1th forward LSTM; z t refers to the sentence representation at the t-th position; refers to the backward LSTM sentence representation at the t-th position; Refers to backward LSTM; refers to the forward LSTM sentence representation at the t+1th position.

7. The model according to claim 6, characterized in that Step 4 specifically includes the following sub-steps: Step 4-1: Merge the sentence representations of the first and last cells of the obtained BiLSTM to obtain Represent the entire sentence with H t Merge to get O t =G||H t ; Among them, G refers to the entire sentence representation; refers to the sentence representation of the first cell; refers to the sentence representation of the nth cell; Step 4-2: Generate i using the gating mechanism t H and i t G , the specific calculation formula is as follows: R H =W H O t +b H R G =W G O t +b G i t H =sigmoid(R t H ) i t G =sigmoid(R t G ) Where W H and W G ∈R 2d×d ; R t G , R t H Represents the global information G and the current sentence representation H respectively t , sigmoid is the activation function; i t H Refers to the assignment to H t The weight of i t G refers to the weight assigned to G, R H Refers to H t The sentence indicates that W H and W G refers to the weight, O t refers to the sentence representation of the tth position that incorporates global information, b H and b G refers to the bias vector; Step 4-3: Through H t Fusion with G t H and i t G Generate global context information, the calculation formula is as follows: in represents global context information, ⊙ represents element-wise product; Step 5 specifically includes the following sub-steps: Step 5-1: Generate three different token representation sequences from the acquired global context information as the context representations of subject, object and relation, which are O i s , O i o , O i r , the calculation formula is as follows: O i s =W s O i +b s O i o =W o O i +b o O i r =W r O i +b r in is a trainable matrix, is the bias vector, d h It represents the dimension; W s refers to the subject weight, O i refers to the sentence representation at the ith position, b s refers to subject bias; W o refers to the object weight, b o Refers to object bias; W r refers to the relationship weight, b r It refers to relationship bias.

8. The model according to claim 6 or 7, characterized in that Step 6 specifically includes the following sub-steps: Step 6-1: Subject Tagger is a binary tagging-based module that extracts all subjects from the input sentence. For each subject in the input sentence, two probabilities are assigned to represent the likelihood of the start and end markers. The probability calculation formula is as follows: p i，start s =σ(W start s The i s +b starts ) p i，end s =σ(W end s The i s +b end s ) where p i，start s and p i，end s denote the probability of the i-th token being the start and end of the subject, respectively. and is a trainable matrix representing weights, O i s represents the subject sentence representation of the ith token, b start s ∈R 1 and b end s ∈R 1 is the bias vector; σ represents the sigmoid activation function; Step 6-2: Subject-based Object Tagger is used to extract the corresponding object based on the extracted subject; the obtained part-of-speech information is used to assist the subject in matching the corresponding object, and two probabilities are also assigned to represent the possibility of the start and end marks of the corresponding object. The probability calculation formula is as follows: p i，start 0=σ(W start o (The i o *v s_k s )+b start o ) p i，end o =σ(W end o (The i o *v s_k s )+b end o ) where v s_k s represents the kth subject representation that incorporates the part of speech, p i，start o and p i，end o Respectively represent the probabilities of the start and end tags of the object associated with the kth subject; * represents the multiplication of the corresponding elements, and is a trainable matrix representing weights, b start o ∈R 1 and b end o ∈R 1 is a bias vector, O i o represents the object sentence representation of the i-th token, σ represents the sigmoid activation function; Step 7 specifically includes the following sub-steps: Step 7-1: The BiEPE module outputs many subject-object pairs. The Biaffine module is used to identify each subject-object pair to obtain the final relation triple. The calculation formula is as follows: v s_k r =maxpool(O s_k_start r ,...,O s_k_end r ) v o_j r =maxpool(O o_j_start r ,...,O o_j_end r ) where v s_k r and v o_j r Represents the representation vectors of the two entities, O s_k_start r and O s_k_end r The start and end vectors of the kth subject are represented by O o_j_staart r and O o_j_end r represents the start and end vector representation of the jth object, p i r Indicates (s k , o j ) has the possibility of the i-th relationship, is the parameter matrix of the i-th relation, and maxpool is the maximum pooling operation.

Images