A relation extraction method fusing entity type representation and relation representation

Abstract

The application discloses a relation extraction method fusing entity type representation and relation representation, and belongs to the technical field of relation extraction. The application relates to a text-subject-object weakly related semantic representation mechanism, entity type information is introduced to replace entity word meaning information, and then the dependence of an extraction model on subject-object semantic association is reduced; on the basis of the above, the application further models abstract semantic information of entity relation, and is fused with context semantic representation containing subject-object type information to generate semantic mapping of the entity relation, so that more accurate prediction effect of the subject-relation-object triple is obtained.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of relation extraction, and in particular to a relation extraction method fusing entity type representation and relation representation. BACKGROUND

[0002] In the context of today's information explosion, information extraction technology extracts important information from massive unstructured text and reconstructs it into structured information that is easy to use for downstream tasks (such as: knowledge graph construction, search engine knowledge base construction, question and answer system knowledge base construction). Relation extraction is an important field of information extraction, aiming to extract structured relation triplets (subject, relation, object) from unstructured text to help depict the association between entities.

[0003] Most existing relation extraction methods use joint or pipeline methods based on named entity recognition to achieve relation extraction. In modeling, first, the subject and object of named entity are identified, and based on the identified results, the semantic information of the subject and object is strengthened in the feature information to extract the relation, while ignoring the global context information of the sentence, which leads to the model being degraded to a certain extent based on entity pair relation matching model, affecting the rationality and robustness of relation extraction. Therefore, the present application proposes a relation extraction method fusing entity type representation and relation representation, which increases the performance robustness of the model for unseen entities or sentences. SUMMARY

[0004] To solve the problems in the prior art, the present application provides a relation extraction method fusing entity type representation and relation representation, which starts from semantic representation, entity type representation and relation representation, uses a text-subject-object weakly related semantic representation mechanism and a relation feature fusion mechanism, and proposes a novel relation extraction model, which can effectively capture the context information of the sentence and realize the extraction of entity pairs and relations in unstructured text, solving the problems mentioned in the background.

[0005] To achieve the above purpose, the present application provides the following technical scheme: a relation extraction method fusing entity type representation and relation representation, the specific steps of which are as follows:

[0006] Step S10: For the natural language text input into the system, encode the semantic information, entity type information and relation information of the text based on the word-piece segmentation method, and output the Word-Piece semantic representation, entity type representation and relation representation;

[0007] Step S20: Based on the output word-piece semantic representation, further extract the subject and object in the text using BERT and binary labeling method;

[0008] Step S30: Replace the extracted semantic representations of the subject and object with the entity type representations of the output to weaken the semantic association information between the subject and the object, construct a weak semantic association mechanism between the subject and the object in the text, and generate a new text with weak semantic association between the subject and the object.

[0009] Step S40: Construct a relation encoder based on the BERT representation model to encode new text with weak semantic associations, extract high-level abstract semantic information from the text, and output a context semantic vector representation of the weak text-subject-object association by combining bidirectional context information.

[0010] Step S50: Construct a fusion mechanism for text-subject weakly correlated contextual semantic information and relational information. The fused representation vector will be used to capture subject-relation-object triples.

[0011] Preferably, the specific steps of step S10 are as follows:

[0012] Step S101: The natural language text input to the system is a sequence of words, s = {w1, ..., w...} l}, where w i Let i ∈ {1, 2, ..., l}, representing the i-th word in the sentence, and l be the number of words in the sentence to be extracted. A Word-Piece representation model based on BPE double-byte encoding is constructed to represent words in the vector space. Each word in the input sentence is segmented into fine-grained sub-words, and the sub-word representation sequence is output. Where t i , i∈{1,2,...,L}, represents the i-th sub-word in the sentence, and is the length of the sub-word in the sentence to be extracted after Word-Piece segmentation;

[0013] Step S102: Entity types and relation types are pre-input into the system for vector representation. ε is the set of entity types, and R is the set of relation types. For any entity type e∈ε and any relation type r∈R in the input system, entity type and relation representation models based on a multilayer perceptron are constructed respectively, transforming discrete entity type symbols and relation type symbols into continuous high-dimensional representation vectors. It outputs fine-grained semantic information about entity types and relation types.

[0014] Preferably, the specific steps of step S20 are as follows:

[0015] Step S201: Construct a named entity encoder based on the BERT neural network representation model, and process the subword sequence As the input of the system encoder, sequentially pass through N Transformer encoder blocks, encode the bidirectional context information of each word piece by fine-tuning parameters, and output a sequence of deep bidirectional language representation vectors

[0016] Wherein, Trans represents the Transformer encoder block, h α-1 represents the encoding result of the previous Transformer encoder block;

[0017] Step S202, establish a named entity subject decoder and object decoder based on a fully connected neural network, to extract candidate subjects and candidate objects in the subword sequence, and the output of the last block of the encoder As the input of the decoder, for each word piece i in the subword sequence, calculate the probability that the word piece is the start of the subject span, the end of the subject span, the start of the object span, and the end of the object span, respectively as follows:

[0018]

[0019] Wherein, represent the learnable weight parameters and bias parameters in the fully connected neural network, and σ is the sigmoid activation function;

[0020] Compare the calculated probability values type∈start_s,end_s,start_o,end_o whether the threshold value 0.5 is exceeded (the threshold value is a hyperparameter set artificially in combination with prior knowledge and hyperparameter experiments. The control system determines whether the word piece is the label corresponding to type according to whether the output probability value exceeds the threshold value), if so, the corresponding determination label type∈start_s,end_s,start_o,end_o is assigned to 1, otherwise the label is assigned to 0;

[0021] According to the results of the above determination label Output the corresponding sequence representation of the start of the subject span, the end of the subject span, the start of the object span, and the end of the object span,

[0022] Step S203, for a 1 label in the subject start determination sequence d start_s , find the nearest 1 label to the right in the subject end determination sequence d end_s , to form a potential subject span sub i ; the same operation is performed on the object determination sequence, and a potential object span obj i is output;

[0023] The above operation is performed on the 1 label in the subject and object start judgment sequence of all subjects and objects, respectively outputting potential subject span sequence H sub = (sub1,..., sub m ) and potential object span sequence H obj = (obj1,..., obj n ), and combining them two by two to form a potential subject-object span pair sequence H = (sub1, obj1),..., (sub m×n , obj m×n );

[0024] Wherein, m, n are the number of potential subjects and potential objects extracted from the subword sequence respectively.

[0025] Preferably, the specific steps of the step S30 are as follows:

[0026] Step S301, construct a text-subject-object weakly related semantic representation mechanism, input entity type information to weaken the subject-object semantic association information, for a given subject-object span pair (sub i , obj j ), i≠j, use the corresponding entity type representation vector Replace the representation vector of the corresponding span in the subword sequence to weaken the subject-object semantic association information, output a new text representation sequence L2 is the length of the replaced subword sequence, and the type representation vectors e(sub i ), e(obj j ) are output simultaneously in the position of the new sequence T, (s1,..., s m ) represents the subject replacement position sequence, and m is the subject replacement length, (o1,..., o n ) represents the object replacement position sequence, and n is the object replacement length.

[0027] Preferably, the specific steps of the step S40 are as follows:

[0028] Step S401, for a subject-object pair (sub i , obj j ), i≠j, construct a relationship encoder based on a BERT neural network representation model, input the new text representation sequence as the input of the system encoder, sequentially pass through N Transformer encoder blocks, and deeply encode the bidirectional context information of each word element by fine-tuning the parameters, output a deep bidirectional language representation vector sequence α∈[1, N], where Trans represents a Transformer encoder block, and hα-1 represents the encoding result of the last Transformer encoder block; the output of the relation encoder is the encoding result of the last Transformer encoder block, that is, the context semantic representation of the text-subject-object weak correlation wherein h i , i∈{1, 2,..., L2}, is the context encoding result of the word unit t i of the sub-word sequence.

[0029] Preferably, the specific steps of the step S50 are as follows:

[0030] Step S501, constructing a relation decoder based on a fully connected linear neural network, calculating the probability of the subject-object pair (sub i , obj j ) having a certain relation, when the system input natural language text sentence s = {w1,..., w l}, the formula is as follows:

[0031] H = H sub + H obj

[0032]

[0033] p i，j，k = σ(W(H; e(r k ))+b)

[0034] wherein, is the representation vector of the relation r , and and are the semantic representations of the encoder outputs at positions (s1,..., s i ), (o1,..., o j ), MaxPooling represents the maximum pooling layer operation, and outputs the subject representation H sub and the object representation H obj , which are added to form the overall entity representation H, W and b are the learnable weight parameters and bias parameters in the fully connected linear neural network, and σ is the sigmoid activation function.

[0035] If the calculated probability value p i，j，k exceeds the preset threshold value 0.6 (the threshold value is a hyperparameter set artificially in combination with prior knowledge and hyperparameter experiments. The control system determines whether the triple has a certain relation according to whether the output probability value exceeds the threshold value), it is considered that the subject-object pair (sub i , obj j ​), i≠j when natural language text sentence is s={w1,..., w l} exists relation For any entity pair and any m×n m×n ) in H=(sub1, obj1),..., (sub Calculate its occurrence probability, and finally output the result as all the probability of the triple exceeding the preset threshold to form the extraction result, that is, the relation extraction result of the natural language text sentence s={w1,..., w l} Reuslt=((sub1, r1, obj1),..., (sub n , r n , obj n )), n is the number of extracted triples.

[0036] The beneficial effects of the present application are:

[0037] 1) The present application takes natural language text as the research object, and provides a relation extraction method fusing entity type representation and relation representation, which starts from semantic representation, entity type representation and relation representation, uses a text-subject-object weakly related semantic representation mechanism and a relation feature fusion mechanism, and proposes a novel relation extraction model, which can effectively capture the context information of the sentence and realize the extraction of entity pairs and relations in unstructured text.

[0038] 2) In addition to outputting semantic representation, the present application also outputs entity type representation and relation representation, uses a neural network model based on BERT to extract the subject and the object and pair the extracted subject and object; according to the pairing result, the type information of the subject and the object is used to replace the encoding information of the subject and the object to obtain new semantic information; the obtained new semantic information is re-encoded using a neural network model based on BERT, and whether there is a relation is predicted using maximum pooling and a multilayer perception machine. The present application designs a text-subject-object weakly related semantic representation mechanism, which replaces the entity word meaning information by introducing entity type information, thereby reducing the dependence of the extraction model on the semantic association of the subject-object. BRIEF DESCRIPTION OF DRAWINGS

[0039] Figure 1 It is a flowchart of the method steps of the present application. DETAILED DESCRIPTION

[0040] ​The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0041] Please see Figure 1 This invention provides a technical solution: a relation extraction method that integrates weakly correlated semantic representation and relation representation, wherein the specific steps of the relation extraction method are as follows:

[0042] Step 1: For the natural language text input to the system, encode the semantic information, entity type information, and relational information of the text based on the word-piece segmentation method, and output the Word-Piece semantic representation, entity type representation, and relational representation; Step 1-1, the natural language text input to the system is a sequence of words, s = {w1, ..., w...} l}, where w i Let i ∈ {1, 2, ..., l}, representing the i-th word in the sentence, and l be the number of words in the sentence to be extracted. The system constructs a Word-Piece representation model based on BPE double-byte encoding to represent words in the vector space, segmenting each word in the input sentence into fine-grained sub-words, and outputting a sub-word representation sequence. Where t i , i∈{1,2,...,L}, represents the i-th sub-word in the sentence, and L is the length of the sub-words in the sentence to be extracted after Word-Piece segmentation;

[0043] Steps 1-2 involve pre-inputting entity types and relation types into the system for vector representation. ε is the set of entity types, and R is the set of relation types. For any entity type e∈ε and any relation type r∈R in the input system, construct entity type and relation representation models based on a multilayer perceptron, transforming discrete entity type symbols and relation type symbols into continuous high-dimensional representation vectors. It outputs fine-grained semantic information about entity types and relation types.

[0044] Step 2: Based on the output word-piece semantic representation, further extract the subject and object from the text using BERT and binary annotation.

[0045] Step 2-1: Construct a named entity encoder based on the BERT neural network representation model. This involves processing the word sequence... As the input of the system encoder, sequentially pass through N Transformer encoder blocks, encode the bidirectional context information of each token by fine-tuning parameters, and output a sequence of deep bidirectional language representation vectors α∈[1,N],where Trans represents a Transformer encoder block, h α-1 represents the encoding result of the previous Transformer encoder block.

[0046] Step 2-2, establish a named entity subject decoder and object decoder based on a fully connected neural network to extract candidate subjects and candidate objects in the subword sequence. Take the output of the last block of the encoder as the input of the decoder, for each token i in the subword sequence, calculate the probability that the token is the start of the subject span, the end of the subject span, the start of the object span, and the end of the object span, respectively as follows:

[0047]

[0048] where represent the learnable weight parameters and bias parameters in the fully connected neural network, and σ is the sigmoid activation function.

[0049] Compare the calculated probability values type∈start_s,end_s,start_o,end_o whether the threshold value 0.5 exceeds the preset threshold value (the threshold value is a hyperparameter set by combining prior knowledge and hyperparameter experiments. The control system determines whether the token is the label corresponding to type according to whether the output probability value exceeds the threshold value), if so, the corresponding determination label type∈start_s,end_s,start_o,end_o is assigned to 1, otherwise the label is assigned to 0.

[0050] The system outputs the corresponding subject span start, subject span end, object span start, and object span end sequence representation according to the results of the above determination labels

[0051] Step 2-3, the system finds the nearest 1 label in the subject start determination sequence d start_s , and finds the nearest 1 label to the right in the subject end determination sequence d end_s to form a potential subject span sub i ; the same operation is performed on the object determination sequence, and a potential object span obj i. The above operation is performed on the 1 label in all subject-object start judgment sequences, and the potential subject span sequence H sub =(sub1,...,sub m ) and the potential object span sequence H obj =(obj1,...,obj n ) are output respectively. Two-by-two combination forms a potential subject-object span pair sequence H=(sub1, obj1),..., (sub m×n , obj m×n ), m and n are the number of potential subjects and potential objects extracted from the subword sequence respectively;

[0052] Step 3: Replace the extracted subject and object word sense representation with the output entity type representation to weaken the subject-object semantic association information, construct a weakly related semantic representation mechanism of the subject and object in the text, and generate a new text of weak semantic association between the subject and the object;

[0053] Step 3-1, construct a text-subject-object weakly related semantic representation mechanism, input additional entity type information to weaken the subject-object semantic association information. For a given subject-object span pair (sub i , obj j ), i≠j, the system uses the corresponding entity type representation vector Replace the representation vector of the corresponding span in the subword sequence to weaken the subject-object semantic association information, and output a new text representation sequence L2 is the length of the replaced subword sequence. At the same time, output the type representation vector e(sub i ), e(obj j ) in the new sequence T, (s1,...,s m ) represents the subject replacement position sequence, m is the subject replacement length, (o1,...,o n ) represents the object replacement position sequence, and n is the object replacement length;

[0054] Step 4: Construct a relationship encoder based on the BERT representation model to encode the weakly related semantic text, extract high-level abstract semantic information in the text, and output a weakly related context semantic vector representation of the text-subject-object based on the bidirectional context information;

[0055] Step 4-1, for a subject-object pair (sub i , obj j ), i≠j, construct a relationship encoder based on the BERT neural network representation model. Replace the new text representation sequence As the input of the system encoder, sequentially pass through N Transformer encoder blocks, encode the bidirectional context information of each word piece by fine-tuning parameters, output the deep bidirectional language representation vector sequence α∈[1,N],where Trans denotes the Transformer encoder block, h α-1 represents the encoding result of the previous Transformer encoder block. The output of the relation encoder is the encoding result of the last Transformer encoder block, which is the context semantic representation of the text-subject-object weakly related where h i , i∈{1,2,...,L2} is the context encoding result of the word piece t i of the sub-word sequence;

[0056] Step 5: Construct the fusion mechanism of the text-subject-object weakly related context semantic information and the relationship information, and the fused representation vector will be used to capture the subject-relation-object triplets;

[0057] Step 5-1, construct a relation decoder based on a fully connected linear neural network, calculate the probability of the relation i , obj j ), i≠j, when the natural language text sentence in the system input is s={w1,...,w l}, output the probability of the relation

[0058] H=H sub +H obj

[0059]

[0060] p i，j，k =σ(W(H;e(r k ))+b)

[0061] where, is the representation vector of the relation , and are the semantic representations of the encoder output at positions (s1,...,s i ), (o1,...,o j ), MaxPooling represents the max-pooling layer operation, output the subject representation H sub and the object representation H obj , and the sum of the two forms the overall entity representation H, W and b are the learnable weight parameters and bias parameters in the fully connected linear neural network, and σ is the sigmoid activation function;​

[0062] If the calculated probability value p i，j，k If the probability exceeds a preset threshold of 0.6 (this threshold is a hyperparameter artificially set by combining prior knowledge and hyperparameter experiments. The control system determines whether the triplet has a certain relationship based on whether the output probability value exceeds the threshold), then it is considered a subject-object pair (sub... i obj j In natural language text, i ≠ j and s = {w1, ..., w} is a statement where i ≠ j. l When a relationship exists... For H = (sub1, obj1), ..., (sub m×n obj m×n Any pair of entities and any The probability of occurrence is calculated, and the final output of the system is the extraction result consisting of all triples with a probability exceeding the preset threshold of 0.6, i.e., the natural language text statement s = {w1, ..., w...}. l The relation extraction result is Reuslt = ((sub1, r1, obj1), ..., (sub n r n obj n )), where n is the number of triples extracted.

[0063] This invention takes natural language text as the research object and designs a relation extraction method that integrates entity type representation and relation representation. Starting from semantic representation, entity type representation, and relation representation, it proposes a novel relation extraction model by utilizing the text-subject-object weak correlation semantic representation mechanism and relation feature fusion mechanism. This model can effectively capture the contextual information of sentences and realize the extraction of entity pairs and relations in unstructured text.

[0064] Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art can still modify the technical solutions described in the foregoing embodiments or make equivalent substitutions for some of the technical features. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A relation extraction method of fusing entity type representation and relation representation, characterized in that, The relationship extraction method comprises the following specific steps: Step S10: For the natural language text input into the system, the semantic information, entity type information and relationship information of the text are encoded based on the word-piece segmentation method, and the word-piece semantic representation, entity type representation and relationship representation are output; Step S20: Based on the output word-piece semantic representation, the subject and object in the text are further extracted by using BERT and binary labeling method; Step S30: The semantic representation of the extracted subject and object is replaced by the output entity type representation to weaken the semantic association information of the subject-object, a weakly related semantic representation mechanism of the subject and object in the text is constructed, and a new text of weak semantic association between the subject and object is generated; The specific steps are as follows: Constructing a text-subject-object weakly related semantic representation mechanism, inputting entity type information to weaken the subject-object semantic association information, for a given subject-object span pair (sub i , obj j ), i≠j, using the corresponding entity type representation vector Replace the representation vector of the corresponding span in the sub-word representation sequence L is the sub-word length of the sentence to be extracted after Word-Piece division, to weaken the subject-object semantic association information, output the new text representation sequence L2 is the length of the sub-word sequence after replacement, and the positions of e(sub i ), e(obj j ) in the new text representation sequence T2 are output simultaneously, (s1,..., s m ) represents the subject replacement position sequence, and m is the subject replacement length, (o1,..., o n ) represents the object replacement position sequence, and n is the object replacement length; Step S40: A relationship encoder based on the BERT representation model is constructed to encode the weakly semantic associated new text, extract high-level abstract semantic information in the text, and output the context semantic vector representation of the weakly related text-subject-object based on the bidirectional context information; the specific steps are as follows: For the sub i , obj j ) pair, i≠j, a relationship encoder based on the BERT neural network representation model is constructed, and a new text representation sequence is taken as the input of the system encoder, sequentially passing through N Transformer encoder blocks, and deep encoding the bidirectional context information of each word piece by fine-tuning the parameters, to output a deep bidirectional language representation vector sequence where Trans represents the Transformer encoder block, h α-1 represents the encoding result of the previous Transformer encoder block; the output of the relationship encoder is the encoding result of the last Transformer encoder block, that is, the text-subject-object weakly related context semantic representation where h i is the context encoding result of the word piece of the sub-word representation sequence, i∈{1,2,...,L2}. Step S50: A fusion mechanism of the context semantic information and relationship information of the weakly related text-subject-object is constructed, and the fused representation vector is used to capture the subject-relation-object triple; the specific steps are as follows: A relation decoder based on a fully connected linear neural network is constructed, and the probability of a subject-object pair (sub i , obj j ) is calculated, i≠j, when the natural language text sentence input by the system is s={w1,..., w l}, and the output relation r k ∈R′ is a set of relation types, and the formula is as follows: H = H sub + H obj p i,j,k = σ(W(H; e(r k ))+ b) wherein, is the representation vector of the relation r k ∈ R′, and (h o1 ,..., h oj ) are the semantic representations of the encoder outputs at positions (s1,..., s i ), (o1,..., o j ), MaxPooling denotes the max pooling layer operation, and the output subject representation H sub and the object representation H obj , which are added together to form the overall entity representation H, W and b are the learnable weight parameters and bias parameters in the fully connected linear neural network, respectively, and σ is the sigmoid activation function. If the calculated probability value p i，j，k If the threshold is exceeded, the subject-object pair is considered to be... i obj j In natural language text, i ≠ j and s = {w1, ..., w} is a statement where i ≠ j. l When}, does the relation r exist? k ∈R′, for H=(sub1,obj1),...,(sub m×n obj m×n For any pair of entities and any relation type r∈R′ in ), calculate the probability of their occurrence; The final output is the extraction result formed by all triples whose probabilities exceed the preset threshold, that is, the natural language text statement s = {w1, ..., w...} l The relation extraction result Reuslt = ((sub1, r1, obj1), ..., (sub n r n obj n )), where n is the number of triples extracted.

2. The fusion entity type representation and relation representation relationship extraction method according to claim 1, characterized in that: The specific steps of step S10 are as follows: Step S101, the natural language text input by the system is a word sequence, s∈{w1, w2,..., w l}, wherein w i represents the i-th word in the sentence, i∈{1, 2,..., l}, and l is the number of words contained in the sentence to be extracted. A Word-Piece representation model based on a BPE double-byte encoding mode is constructed to represent a word in a vector space, each word in an input sentence is divided into fine-grained subwords, and a subword representation sequence is output where t i represents the representation of the i-th subword in the sentence, i∈{1, 2,..., L}. In step S102, the entity types and the relation types are pre-input into the system for vector representation, ε is a set of entity types, R' is a set of relation types, for any entity type e of the input system ε and any relation type r of R', an entity type and a relation representation model based on a multi-layer perception are constructed respectively, and discrete entity type symbols and relation type symbols are converted into continuous high-dimensional representation vectors The fine-grained semantic information of the output entity types and the relation types is output.

3. The fusion entity type representation and relation representation relationship extraction method according to claim 1, characterized in that: The specific steps of step S20 are as follows: Step S201, construct a named entity encoder based on a BERT neural network representation model, encode the subword representation sequence As the input of the system encoder, sequentially pass through N Transformer encoder blocks, encode the bidirectional context information of each word piece by fine-tuning parameters, and output a deep bidirectional language representation vector sequence wherein Trans represents a Transformer encoder block, h α-1 represents the encoding result of the previous Transformer encoder block; Step S202, a subject decoder and an object decoder based on a full connection neural network are established to extract candidate subjects and candidate objects in the sub-word representation sequence, and the output of the last block of the encoder is taken as the input of the decoder For the input of the decoder, the probability of each word in the sub-word representation sequence being the start point of the subject span, the end point of the subject span, the start point of the object span, and the end point of the object span is calculated, and the formulas are as follows: wherein, are the learnable weight and bias parameters in the fully connected neural network, respectively, and σ is the sigmoid activation function. Comparing the calculated probability values whether a preset threshold is exceeded, type e start s, end s, start o, end o, if yes, the corresponding decision label is assigned the value 1, otherwise the label is assigned the value 0; According to the result of the determination tag outputs a corresponding subject span start point, subject span end point, object span start point, and object span end point sequence representation, Step S203, for the subject start determination sequence d slart_s , find the nearest 1 tag to the right in the subject end determination sequence d end_s to form a potential subject span sub i ; for the object determination sequence, the same operation is performed, and a potential object span obj j is output; The above operation is performed on the 1 tags in the subject and object start determination sequence for all subjects and objects, respectively outputting potential subject span sequence H sub =(sub1,...,sub m ) and potential object span sequence H obj =(obj1,...,obj n ), and combining them two by two to form a potential subject-object span pair sequence, H=(sub1, obj1),..., (sub m×n , obj m×n ). Wherein, m and n are the number of potential subjects and potential objects extracted from the sub-word representation sequence respectively.

Images