Ancient Chinese entity relationship extraction method based on multi-step information fusion and boundary alignment

Abstract

The present invention discloses an ancient Chinese entity relationship extraction method based on multi-step information fusion and boundary alignment. The method completes the ancient Chinese entity relationship extraction through multi-step fusion of relationship semantic information and entity boundary alignment; the relationship features in the text are enriched by multi-step fusion of relationship semantic information, which alleviates the problem of low relationship extraction recall rate in ancient Chinese entity relationship extraction. At the same time, the entity relationship extraction task is modeled into two subtasks of entity extraction and initial triple extraction and verified by entity boundary alignment, which improves the accuracy of entity boundary recognition and entity relationship triple extraction in complex text scenarios and reduces the redundancy of information extraction; the present invention adopts a method of jointly extracting entities and their relationships, which avoids the error propagation problem existing in the process from entity recognition to relationship extraction.

Description

Technical Field

[0001] The present invention relates to the technical field of ancient Chinese entity relation extraction, and specifically to an ancient Chinese entity relation extraction method based on information multi-step fusion and boundary alignment. Background Art

[0002] Entity relation extraction is an important task in the construction of knowledge graphs for natural language processing, aiming to identify the entities and the relationships between entities in unstructured texts, usually represented in the form of entity relation triples: <subject entity, relation, object entity>. With the rapid development of deep learning and the rise of research related to information extraction, researchers have continuously delved into entity relation extraction and related issues, achieving many good research results. Currently, entity relation extraction work mainly focuses on the English and modern Chinese fields, with less research in the ancient Chinese field. An important reason is that the current dataset for the ancient Chinese entity relation extraction task is relatively small. In recent years, with the rise of the cultural industry, researchers have placed more emphasis on the research of ancient knowledge graph construction tasks. The entities and the relationships between entities in ancient Chinese, as the core of ancient Chinese cultural knowledge representation, the accuracy of extracting entity relationships plays an important role in the construction of knowledge graphs related to ancient Chinese culture; it can directly affect the quality of the knowledge graph.

[0003] Ancient Chinese has the special linguistic feature of mostly expressing meanings with single characters and having a large number of omissions, and the writing characteristics of ancient Chinese texts in different periods are usually inconsistent. Compared with modern Chinese, it has significant differences in language structure, vocabulary, etc. Ancient Chinese with the same number of characters contains more semantic information than modern Chinese, and the number of entities and relationships is usually several times that of modern Chinese. Among them, entity relation information is often contained in a single relation trigger word. For example, in the sentence "Pang Tongshan and Gao Kan were still in Xincheng. Quan Nanjian sent troops to attack their camp, and Xue Rengui, the left Wuwei General, defeated them.", this sentence contains 6 entities and three types of entity relationships, up to 6 entity relation triples. Among them, the semantic information conducive to the model to identify the type of the belonging relationship is contained in relation trigger words such as "in" and "attack", and there are also many cases of nested entities. Therefore, how to automatically, accurately, and jointly extract entities and their relationships from ancient Chinese texts is a difficult problem.

[0004] Existing traditional entity relationship extraction methods, such as pipeline extraction methods, usually divide entity relationship triple extraction into two steps: entity extraction and relationship recognition, which has the problem of error propagation; that is, when the entity recognition is wrong, the relationship between entities will also be extracted incorrectly or considered non-existent, which ultimately leads to incorrect or missing entity relationship triple extraction. In recent years, most entity relationship extraction methods use joint extraction, that is, unified modeling of entity recognition and relationship extraction subtasks, which effectively solves the error propagation problem. Compared with modern Chinese, ancient Chinese texts contain less semantic and structural information, and there are a large number of long-span entities, which makes it difficult to identify entity boundaries, extract relationships, and match entity pairs and relationships in ancient Chinese entity relationship extraction.

[0005] In the field of ancient Chinese, some existing entity relationship extraction methods have limited understanding of the semantics of ancient Chinese knowledge, which makes it difficult to extract potential relationships in sentences. There is little work on mining deep entity relationship information in sentences by introducing external knowledge, and there is almost no work on using cross-fusion relationship semantic information to alleviate the problem of low accuracy in entity relationship extraction. Although some existing named entity work considers introducing modern English translated by large language models as external knowledge into proprietary models, and can help improve performance to a certain extent, the current large model's translation effect on ancient Chinese cannot completely reach the level of manual translation, and still cannot better solve the problem of lack of relationship semantic information in ancient Chinese entity relationship extraction. Summary of the invention

[0006] In view of the shortcomings of the prior art, the present invention provides an ancient Chinese entity relationship extraction method based on multi-step information fusion and boundary alignment. Its purpose is to unify ancient Chinese entity recognition and relationship extraction into an entity relationship joint extraction task from a new perspective, avoid the error propagation problem caused by the use of a pipeline method, and achieve one-time completion of named entity recognition and extraction of relationships between entity pairs. At the same time, it alleviates the problems of insufficient semantic information of ancient Chinese relationships and difficulty in matching entities and relationships.

[0007] To achieve the above object, the present invention provides the following technical solution: an ancient Chinese entity relationship extraction method based on multi-step information fusion and boundary alignment, comprising the following steps:

[0008] Step S1: construct an ancient Chinese entity relationship dataset, which includes a number of ancient Chinese texts;

[0009] Step S2: Define the ancient Chinese entity relationship dataset and set The ancient Chinese entity relationship dataset of ancient Chinese texts is , for Middle Ancient Chinese text; Ancient Chinese text is , for The words; the predefined set of relations is ,Include relationship categories, for Middle Class relations; extract entities and relations from ancient Chinese text and output them in the format of triples. Entities include main entities and object entities. The triples are expressed as , express The span of the main entity, represents the span of the guest entity in Y, express The relationship between the subject entity and the object entity;

[0010] Step S3: constructing an ancient Chinese entity relationship extraction model, the ancient Chinese entity relationship extraction model consists of a feature encoder, a fuser, an entity extractor, an entity relationship triple extractor, and an entity boundary alignment module; the feature encoder includes a text feature encoder and a relationship feature encoder; the fuser consists of an interactive text relationship information fuser and a relationship text information fuser, and the text relationship information fuser and the relationship text information fuser are both composed of an information interaction module, a residual module, and an information splicing module; the entity extractor includes an entity boundary link label prediction module and a label decoding module; the entity relationship triple extractor includes an entity relationship link label prediction module and a label decoding module;

[0011] Step S4: taking the ancient Chinese text and the relation sentence formed by concatenating relations as inputs of the text feature encoder and the relation feature encoder respectively, and obtaining the word embedding feature vector of the ancient Chinese text and the word embedding feature vector of the relation sentence; the text feature encoder and the relation feature encoder are both composed of a pre-trained BERT model;

[0012] Step S5: inputting the word embedding feature vector of the ancient Chinese text as the main information and the word embedding feature vector of the relational sentence as the auxiliary information into the text relational information fuser to obtain the text feature representation; inputting the word embedding feature vector of the relational sentence as the main information and the word embedding feature vector of the ancient Chinese text as the auxiliary information into the relational text information fuser to obtain the relational feature representation;

[0013] Step S6: Concatenate the different word embedding feature vectors in the text feature representation to obtain the text feature concatenation representation , concatenate the text feature representation and the relation feature representation to obtain the relation feature concatenation representation ;

[0014] Step S7: Input the text feature concatenation representation into the entity extractor to obtain the predicted entity boundary link label, and decode the text feature concatenation representation based on the predicted entity boundary link label to obtain the position of the entity head word and the tail word, concatenate the entity head word and the tail word to form an ancient Chinese entity set, and then calculate the entity boundary loss through the cross entropy loss function , optimize and update the model parameters of the feature encoder and entity extractor;

[0015] Step S8: Input the concatenated representation of the relational features into the entity relation triple extractor to obtain the initial triple, and then calculate the initial triple loss through the cross entropy loss function ,Optimize and update the model parameters of feature encoder and entity relationship triple extractor;

[0016] Step S9: input the predicted entity boundary link label and the initial triple into the entity boundary alignment module, align the predicted entity boundary link label and the initial triple based on the ancient Chinese entity set, and select the final entity relationship triple.

[0017] Furthermore, the specific process of step S5 is as follows:

[0018] Step S5.1: The word embedding feature vector of the word and relational sentences Word embedding feature vector of class relation Input into the information interaction module, , , using multi-head attention to interactively fuse the word embedding feature vectors of ancient Chinese text and the word embedding feature vectors of related sentences; specifically:

[0019] Will As the query vector in the attention mechanism ,Will As the key vector in the attention mechanism Sum value vector , to perform cross attention calculation; As the query vector in the attention mechanism ,Will As the key vector in the attention mechanism Sum value vector Perform cross attention calculation; the cross attention calculation process is expressed as:

[0020] ;

[0021] ;

[0022] ;

[0023] ;

[0024] In the formula, , , Respectively represent the query vector when cross attention is calculated , key vector Sum value vector The projection parameter matrix of , , Respectively represent the query vector when cross attention is calculated , key vector Sum value vector The projection parameter matrix of for The size of the second dimension of for The size of the second dimension of for The transpose of for The transpose of express function; represents the attention mechanism;

[0025] Step S5.2: The output of the cross attention calculation is used as the input of the residual module. After residual calculation, the text feature representation is obtained. and relational feature representation , expressed as:

[0026] ;

[0027] ;

[0028] In the formula, Represents a normalization operation.

[0029] Furthermore, the specific process of step S7 is as follows:

[0030] Step S7.1: Input the text feature concatenation representation into the entity boundary link label prediction module, and record the ancient Chinese text by setting an entity boundary link matrix The link information between the head word and the tail word of the entity in the prediction entity boundary link label ;

[0031] Step S7.2: Link the predicted entity boundaries to labels Input the label decoding module, decode it in the label decoding module according to the first preset rule to obtain the entity's head word and tail word bigram; compare the entity's head word and tail word bigram with the ancient Chinese text Matching is performed, and according to the matching results, all words in the corresponding span of the entity's head word and tail word are concatenated to obtain a complete entity, and all complete entities constitute an ancient Chinese entity set.

[0032] Furthermore, in step S7.1, an entity boundary link matrix is ​​set to record the ancient Chinese text The link information between the head word and the tail word of the entity in the prediction entity boundary link label The specific process is:

[0033] Unify the entity boundary link matrix size to , is the number of characters in the ancient Chinese text, and the first prediction label is filled in the entity boundary link matrix through the score of the entity boundary link scoring function, and the filled first prediction label is used to indicate whether the corresponding word is the head word or the tail word of the entity; when the first prediction label is "1", it indicates that the corresponding word is the head word or the tail word of the entity, and when the first prediction label is "0", it indicates that the corresponding word is not the head word or the tail word of the entity; wherein, the row element of the entity boundary link matrix is ​​the head word of the entity, and the column element of the entity boundary link matrix is ​​the tail word of the entity;

[0034] Among them, the entity boundary link scoring function is expressed as:

[0035] ;

[0036] In the formula, , , , is the trainable parameter matrix; is the activation function; Indicates the probability of the head word and tail word of a certain type of entity in ancient Chinese text; Indicates the first word; Indicates the first word; Represents the ordinal number of the entity type in the entity type collection;

[0037] Through the parameter matrix , , is the embedding dimension, Representing real number domain, adaptive learning , and finally through the activation function Calculate the classification of the first predicted label :

[0038] ;

[0039] In the formula, is the activation function, is the first predicted label for each filled position in the entity boundary link matrix; when When it is greater than the set threshold, the i-th word is considered to be the entity head word and the j-th word is the entity tail word.

[0040] Furthermore, the specific process of step S8 is as follows:

[0041] Step S8.1: The concatenated representation of the relational features is input into the entity relation link label prediction module in the entity relation triple extractor, and an entity relation link matrix is ​​set up to record the link information of entities and relations in the ancient Chinese text, that is, to predict the entity relation link label. ;

[0042] Step S8.2: Link the predicted entity relationship labels Input the label decoding module, and decode it in the label decoding module according to the second preset rule to obtain the main entity head word and the relationship bigram, the object entity head word and the relationship bigram, and the main entity head word and the object entity head word bigram;

[0043] Step S8.3: Obtain an initial triple based on the main entity head word and relation bigram, the guest entity head word and relation bigram, and the main entity head word and guest entity head word bigram.

[0044] Furthermore, in step S8.1, an entity relationship link matrix is ​​set to record the link information of entities and relationships in the ancient Chinese text, that is, to predict the entity relationship link label The specific process is:

[0045] Assume that the dimension size of the entity relationship link matrix is ; The entity relationship link matrix includes three parts: the main entity and the guest entity link, the main entity and the relationship link, and the guest entity and the relationship link; the entity relationship link matrix is ​​filled with the second prediction label through the score of the entity relationship link scoring function, and the second prediction label includes two types: "1" and "0";

[0046] In the link between the main entity and the guest entity, when the second prediction label is "1", it means that the row element and column element corresponding to the filling position in the entity relationship link matrix are the main entity head word and the guest entity head word bigrams, and when the second prediction label is "0", it means that the corresponding row element and column element are not the main entity head word and the guest entity head word bigrams, where both the row element and the column element are ancient Chinese texts;

[0047] In the main entity and relationship link part, when the second prediction label is "1", it means that the row element and column element corresponding to the filling position in the entity relationship link matrix are the main entity head word and the relationship bigram. When the second prediction label is "0", it means that the corresponding row element and column element are not the main entity head word and the relationship bigram, where the row element is the ancient Chinese text and the column element is the predefined relationship.

[0048] In the object entity and relationship link part, when the second prediction label is "1", it means that the row element and column element corresponding to the filling position in the entity relationship link matrix are the object entity header word and the relationship bigram, and when the second prediction label is "0", it means that the corresponding row element and column element are not the object entity header word and the relationship bigram, where the row element is the predefined relationship and the column element is the ancient Chinese text;

[0049] Wherein, the entity relationship link scoring function is expressed as:

[0050] ;

[0051] In the formula, , , , is a trainable parameter; is the activation function; Represents the first row in the entity relationship link matrix elements, Represents the column in the entity relationship link matrix elements;

[0052] Through the parameter matrix Adaptive Learning ; Finally, through the activation function Calculate the classification of the second predicted label :

[0053] ;

[0054] In the formula, The second predicted label filled in the entity relationship link matrix;

[0055] when When it is greater than the set threshold, the corresponding second prediction label is "1", otherwise it is "0".

[0056] Furthermore, the specific process of obtaining the initial triples based on the main entity head word and the relationship bigram, the object entity head word and the relationship bigram, and the main entity head word and the object entity head word bigram is as follows: based on the classification results, the combinations with the same relationship in the main entity head word and the relationship bigram and the object entity head word and the relationship bigram are merged into triples, namely, the main entity head word, the relationship, and the object entity head word, and the triples that are not in the main entity head word and the object entity head word bigram are excluded from all the merged triples to obtain the initial triples;

[0057] Excluding triples that are not in the main entity head word and the guest entity head word bigram among all triples obtained by merging means verifying the triples using the main entity head word and the guest entity head word bigram, comparing whether the main entity head word and the guest entity head word in the triples are consistent with the main entity head word and the guest entity head word bigram, and excluding the corresponding triples if they are inconsistent.

[0058] Furthermore, the first preset rule is: obtaining row elements and column elements corresponding to all first prediction labels that are "1" in the entity boundary link matrix to form a head word and tail word tuple of the entity;

[0059] The second preset rule is:

[0060] Obtain the row and column element combinations corresponding to all second prediction labels that are "1" in the entity relationship link matrix, then count the subscripts of the positions of the row and column element combinations in the entity relationship link matrix, and classify the obtained row and column element combinations according to the set standard;

[0061] The criteria are set as:

[0062] When the row element subscript and column element subscript of the entity relationship link matrix are both less than or equal to Z, it is classified as a binary of the main entity head word and the object entity head word;

[0063] When the row element subscript of the entity relationship link matrix is ​​less than or equal to Z, and the column element subscript is greater than Z, it is classified as the main entity head word and the relationship bigram;

[0064] When the row element subscript of the entity relationship link matrix is ​​greater than Z and the column element subscript is less than or equal to Z, it is classified as the object entity head word and relationship bigram.

[0065] Further, according to the classification of the first predicted label, the predicted entity boundary link label and real entity boundary link labels Input to the cross entropy loss function, and use the cross entropy loss function to optimize the parameters of the entity extractor, expressed as:

[0066] ;

[0067] In the formula, represents entity boundary loss; express and equal probability;

[0068] According to the second predicted label classification probability, the predicted entity relationship link label and real entity relationship link labels Input to the cross entropy loss function, and use the cross entropy loss function to update and optimize the parameters of the entity relationship triple extractor, which is expressed as:

[0069] ;

[0070] In the formula, represents the initial triplet loss; express and equal probability;

[0071] Among them, the real entity boundary link label and real entity relationship link labels All of them are obtained from the ancient Chinese entity relationship dataset.

[0072] Furthermore, the specific process of step S9 is: use the head word of the entity in the predicted entity boundary link label to match the head word of the entity in the initial triple, when the head word of the entity in the predicted entity boundary link label is the same as the head word of the entity in the initial triple, filter out the initial triple with the same head word of the entity, and fill the corresponding complete entity in the Middle Chinese entity set in step S7.2 and the predicted entity boundary link label into the corresponding position in the filtered initial triple, to obtain the final entity relationship triple;

[0073] When the head word of the entity in the predicted entity boundary link label is different from the head word of the entity in the initial triplet, the corresponding initial triplet is removed.

[0074] Compared with the existing technology, the present invention has the following beneficial effects:

[0075] (1) Aiming at the common characteristics of ancient Chinese text content, which is that words are written in abbreviated form and are expressed in single words, the present invention designs an entity extractor and conducts joint training with the entity relationship triple extraction task. This not only integrates relational semantic information into text embedding features, but also alleviates some adverse effects brought by the relational information itself, thereby improving the accuracy of triple extraction and ensuring a certain improvement in the accuracy of entity span boundary recognition.

[0076] (2) The present invention adopts the method of "splicing relations and ancient Chinese text" by first splicing candidate relations with ancient Chinese text to form a new text, and then extracting the semantic information of the text through an encoder. Compared with the prior art method of treating relations as categories and obtaining relations only through a classifier, the method of "splicing relations and ancient Chinese text" adopted by the present invention is to add ancient Chinese text and relations together to the model for training, so that the model can better understand the semantics of the relations from a semantic level, thereby enhancing the model's ability to predict relations.

[0077] (3) This paper introduces relational semantic information by adopting a multi-head cross-attention approach, so that the model strengthens the interaction between input text information and relational semantic information during training. At the same time, a matrix-based entity extractor is designed, and the model's ability to extract long-span entities is enhanced through entity boundary matching verification. BRIEF DESCRIPTION OF THE DRAWINGS

[0078] Figure 1 It is a flow chart of the overall model structure of the present invention. DETAILED DESCRIPTION

[0079] like Figure 1 As shown, the present invention provides a technical solution: an ancient Chinese entity relationship extraction method based on multi-step information fusion and boundary alignment, comprising the following steps:

[0080] Step S1: construct an ancient Chinese entity relationship dataset, which includes several ancient Chinese texts.

[0081] Step S2: Define the ancient Chinese entity relationship dataset and set The ancient Chinese entity relationship dataset of ancient Chinese texts is , for Middle Ancient Chinese text; Ancient Chinese text is , for The characters; the set of predefined entity types is ,Include entity types, for Middle entity types; the predefined set of relationships is ,Include relationship categories, for Middle Class relations; extract entities (main entities and object entities) and relations from ancient Chinese texts and output them in triple format. The triples are represented as , express The span of the main entity, represents the span of the guest entity in Y, express The relationship between the subject entity and the object entity.

[0082] Step S3: construct an ancient Chinese entity relationship extraction model, which consists of a feature encoder, a fuser, an entity extractor, an entity relationship triple extractor, and an entity boundary alignment module.

[0083] The feature encoder includes a text feature encoder and a relation feature encoder.

[0084] The fuser is composed of a text relation information fuser and a relation text information fuser which interact with each other. Both the text relation information fuser and the relation text information fuser are composed of an information interaction module, a residual module and an information splicing module.

[0085] The entity extractor includes an entity boundary link label prediction module and a label decoding module.

[0086] The entity relationship triple extractor includes an entity relationship link label prediction module and a label decoding module.

[0087] Step S4: The ancient Chinese text and the relational sentences formed by splicing relations are used as inputs of the text feature encoder and the relational feature encoder respectively, to obtain the word embedding feature vector of the ancient Chinese text and the word embedding feature vector of the relational sentences.

[0088] Among them, the text feature encoder and the relationship feature encoder are both composed of the pre-trained BERT model.

[0089] The specific process of step S4 is:

[0090] Ancient Chinese text Input the pre-trained BERT model to obtain the word embedding feature vector of ancient Chinese text, expressed as:

[0091] (1);

[0092] In the formula, Indicates the first The word embedding feature vector of the word; Represents the pre-trained BERT model.

[0093] The relations in the predefined relation set are stitched together to form relation sentences. Input the pre-trained BERT model to obtain the word embedding feature vector of the relational sentence, expressed as:

[0094] (2);

[0095] In the formula, In a sentence expressing a relationship Class word embedding feature vector.

[0096] Step S5: Input the word embedding feature vector of the ancient Chinese text as the main information and the word embedding feature vector of the relational sentence as the auxiliary information into the text relational information fuser to obtain the text feature representation; input the word embedding feature vector of the relational sentence as the main information and the word embedding feature vector of the ancient Chinese text as the auxiliary information into the relational text information fuser to obtain the relational feature representation.

[0097] The specific process of step S5 is:

[0098] Step S5.1: The word embedding feature vector of the word and relational sentences Word embedding feature vector of class relation Input into the information interaction module, , , using multi-head attention to interactively fuse the word embedding feature vectors of ancient Chinese text and the word embedding feature vectors of related sentences; specifically:

[0099] Will As the query vector in the attention mechanism ,Will As the key vector in the attention mechanism Sum value vector , to perform cross attention calculation; As the query vector in the attention mechanism ,Will As the key vector in the attention mechanism Sum value vector Perform cross attention calculation; the cross attention calculation process is expressed as:

[0100] (3);

[0101] (4);

[0102] (5);

[0103] (6);

[0104] In the formula, , , Respectively represent the query vector when cross attention is calculated , key vector Sum value vector The projection parameter matrix of , , Respectively represent the query vector when cross attention is calculated , key vector Sum value vector The projection parameter matrix of for The size of the second dimension of for The size of the second dimension of for The transpose of for The transpose of express function; Represents the attention mechanism.

[0105] Step S5.2: The output of the cross attention calculation is used as the input of the residual module. After residual calculation, the text feature representation is obtained. and relational feature representation , expressed as:

[0106] (7);

[0107] (8);

[0108] In the formula, Represents a normalization operation.

[0109] Step S6: Concatenate the different word embedding feature vectors in the text feature representation to obtain the text feature concatenation representation , concatenate the text feature representation and the relation feature representation to obtain the relation feature concatenation representation ; expressed as:

[0110] (9);

[0111] (10);

[0112] In the formula, Represents vector concatenation; Represents the word embedding feature vector of the jth word in the text feature representation.

[0113] Step S7: Input the text feature concatenation representation into the entity extractor to obtain the predicted entity boundary link label, and decode the text feature concatenation representation based on the predicted entity boundary link label to obtain the position of the entity head word and the tail word, concatenate the entity head word and the tail word to form an ancient Chinese entity set, and then calculate the entity boundary loss through the cross entropy loss function , optimize and update the model parameters of the feature encoder and entity extractor.

[0114] The specific process of step S7 is:

[0115] Step S7.1: Input the text feature concatenation representation into the entity boundary link label prediction module, and record the ancient Chinese text by setting an entity boundary link matrix The link information between the head word and the tail word of the entity in the prediction entity boundary link label Specific:

[0116] Unify the entity boundary link matrix size to , is the number of characters in the ancient Chinese text, and the first prediction label is filled in the entity boundary link matrix according to the score of the entity boundary link scoring function. The filled first prediction label is used to indicate whether the corresponding word is the head word or the tail word of the entity; when the first prediction label is "1", it indicates that the corresponding word is the head word or the tail word of the entity, and when the first prediction label is "0", it indicates that the corresponding word is not the head word or the tail word of the entity; wherein, the row elements of the entity boundary link matrix are the head words of the entity, and the column elements of the entity boundary link matrix are the tail words of the entity.

[0117] Among them, the entity boundary link scoring function is expressed as:

[0118] (11);

[0119] In the formula, , , , is the trainable parameter matrix; is the activation function; Indicates the probability of the head word and tail word of a certain type of entity in ancient Chinese text; Indicates the first word; Indicates the first word; Represents the ordinal number of an entity type in a collection of entity types.

[0120] Through the parameter matrix in formula (11) ( , is the embedding dimension, Representing the real number domain) adaptive learning , and finally through the activation function Calculate the classification of the first predicted label :

[0121] (12);

[0122] In the formula, is the activation function, is the first predicted label for each filled position in the entity boundary link matrix; when When it is greater than the set threshold, the i-th word is considered to be the entity head word and the j-th word is the entity tail word.

[0123] Step S7.2: Link the predicted entity boundaries to labels Input the label decoding module, decode it in the label decoding module according to the first preset rule to obtain the entity's head word and tail word bigram; compare the entity's head word and tail word bigram with the ancient Chinese text Matching is performed, and according to the matching results, all words in the corresponding span of the entity's head word and tail word are concatenated to obtain a complete entity, and all complete entities constitute an ancient Chinese entity set.

[0124] The first default rule is:

[0125] Get the row elements and column elements corresponding to all the first predicted labels that are "1" in the entity boundary link matrix to form the entity's head word and tail word bigrams.

[0126] According to the classification of the first predicted label, the predicted entity boundary link label and real entity boundary link labels (obtained from the ancient Chinese entity relationship dataset) is input into the cross entropy loss function, and the cross entropy loss function is used to optimize the parameters of the entity extractor, which is expressed as:

[0127] (13);

[0128] In the formula, represents entity boundary loss; express and Equal probability.

[0129] Step S8: Input the concatenated representation of the relational features into the entity relation triple extractor to obtain the initial triple: (main entity header, relation, guest entity header), and then calculate the initial triple loss through the cross entropy loss function ,Optimize and update the model parameters of the feature encoder and entity relationship triplet extractor.

[0130] The specific process of step S8 is:

[0131] Step S8.1: The concatenated representation of the relational features is input into the entity relation link label prediction module in the entity relation triple extractor, and an entity relation link matrix is ​​set up to record the link information of entities and relations in the ancient Chinese text, that is, to predict the entity relation link label. To ensure that the initial triples can be extracted; specifically:

[0132] Assume that the dimension size of the entity relationship link matrix is ; The entity relationship link matrix includes three parts: the main entity and the guest entity link, the main entity and the relationship link, and the guest entity and the relationship link; among them, the main entity and the guest entity link part is used to record the connection between the main entity head word and the guest entity head word in the initial triple, the main entity and the relationship link part is used to record the connection between the main entity head word and the relationship in the initial triple, and the guest entity and the relationship link part is used to record the connection between the guest entity head word and the relationship in the initial triple; the second prediction label is filled in the entity relationship link matrix through the score of the entity relationship link scoring function, and the second prediction label includes "1" and "0"; in different link parts, the second prediction label represents different meanings.

[0133] In the link part between the main entity and the guest entity, when the second prediction label is "1", it means that the row elements and column elements corresponding to the filling position in the entity relationship link matrix are the main entity head word and the guest entity head word bigrams. When the second prediction label is "0", it means that the corresponding row elements and column elements are not the main entity head word and the guest entity head word bigrams, where the row elements and column elements are both ancient Chinese texts.

[0134] In the main entity and relationship link part, when the second prediction label is "1", it means that the row elements and column elements corresponding to the filling position in the entity relationship link matrix are the main entity head word and relationship bigram. When the second prediction label is "0", it means that the corresponding row elements and column elements are not the main entity head word and relationship bigram, where the row elements are ancient Chinese text and the column elements are predefined relationships.

[0135] In the object entity and relationship link part, when the second prediction label is "1", it means that the row elements and column elements corresponding to the filling position in the entity relationship link matrix are the object entity header word and relationship bigram. When the second prediction label is "0", it means that the corresponding row elements and column elements are not the object entity header word and relationship bigram, where the row elements are predefined relationships and the column elements are ancient Chinese text.

[0136] Wherein, the entity relationship link scoring function is expressed as:

[0137] (14);

[0138] In the formula, , , , is a trainable parameter; is the activation function; Represents the first row in the entity relationship link matrix elements, Represents the column in the entity relationship link matrix elements.

[0139] Through the parameter matrix in formula (14) Adaptive Learning ; Finally, through the activation function Calculate the classification of the second predicted label :

[0140] (15);

[0141] In the formula, The second predicted label filled in the entity relationship link matrix.

[0142] when When it is greater than the set threshold, the corresponding second prediction label is "1", otherwise it is "0".

[0143] The entity relationship link matrix is ​​filled with the second predicted labels of "1" and "0" obtained by classification. The filled content indicates whether the row elements and column elements belong to a link between the main entity header and the guest entity header, a link between the main entity header and the relationship, and a link between the guest entity header and the relationship.

[0144] Step S8.2: Link the predicted entity relationship labels The label decoding module is input and decoded according to the second preset rule in the label decoding module to obtain the main entity head word and relationship bigram, the guest entity head word and relationship bigram, and the main entity head word and guest entity head word bigram.

[0145] Step S8.3: Based on the main entity head word and relation bigram, the object entity head word and relation bigram, and the main entity head word and object entity head word bigram, an initial triple is obtained: (main entity head word, relation, object entity head word).

[0146] Among them, the second preset rule is:

[0147] Get the row and column element combinations corresponding to all the second prediction labels that are "1" in the entity relationship link matrix, then count the subscripts of the positions of the row and column element combinations in the entity relationship link matrix, and classify the obtained row and column element combinations according to the following set standards.

[0148] Setting the Standard:

[0149] (1) When the row element subscript and column element subscript of the entity relationship link matrix are both less than or equal to Z, they are classified as the main entity head word and the guest entity head word bigram.

[0150] (2) When the row element subscript of the entity relationship link matrix is ​​less than or equal to Z and the column element subscript is greater than Z, it is classified as the main entity head word and relationship bigram.

[0151] (3) When the row element subscript of the entity relationship link matrix is ​​greater than Z and the column element subscript is less than or equal to Z, it is classified as a guest entity head word and relationship tuple.

[0152] Among them, the specific process of obtaining the initial triples based on the main entity head word and relationship bigrams, the object entity head word and relationship bigrams, and the main entity head word and object entity head word bigrams is: based on the classification results, the combinations with the same relationship in the main entity head word and relationship bigrams and the object entity head word and relationship bigrams are merged into triplets, namely, the main entity head word, the relationship, and the object entity head word, and the triplets that are not in the main entity head word and the object entity head word bigrams in all the merged triplets are excluded to obtain the initial triples.

[0153] Excluding triples that are not in the main entity head word and the guest entity head word bigram among all triples obtained by merging means verifying the triples using the main entity head word and the guest entity head word bigram, comparing whether the main entity head word and the guest entity head word in the triples are consistent with the main entity head word and the guest entity head word bigram, and excluding the corresponding triples if they are inconsistent.

[0154] According to the second predicted label classification probability, the predicted entity relationship link label and real entity relationship link labels (obtained from the ancient Chinese entity relationship dataset) is input into the cross entropy loss function, and the cross entropy loss function is used to update and optimize the parameters of the entity relationship triple extractor, which is expressed as:

[0155] (16);

[0156] In the formula, represents the initial triplet loss; express and Equal probability.

[0157] Step S9: input the predicted entity boundary link label and the initial triple into the entity boundary alignment module, align the predicted entity boundary link label and the initial triple based on the ancient Chinese entity set, and select the final entity relationship triple.

[0158] The specific process of step S9 is: use the head word of the entity in the predicted entity boundary link label to match the head word of the entity in the initial triple. When the head word of the entity in the predicted entity boundary link label is the same as the head word of the entity in the initial triple, filter out the initial triples with the same head word of the entity, and fill the corresponding complete entities in the Middle Chinese entity set in step S7.2 and in the predicted entity boundary link label into the corresponding positions in the filtered initial triples to obtain the final entity relationship triples; when the head word of the entity in the predicted entity boundary link label is not the same as the head word of the entity in the initial triple, remove the corresponding initial triple.

[0159] The entity boundary loss and the initial triplet loss Combined, we get the total loss function of the process of obtaining the final entity relationship triple , expressed as:

[0160] (17).

[0161] Although embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions and variations may be made to the embodiments without departing from the principles and spirit of the present invention, and that the scope of the present invention is defined by the appended claims and their equivalents.

Claims

1. An ancient Chinese entity relationship extraction method based on multi-step information fusion and boundary alignment, characterized in that: The steps include: Step S1: construct an ancient Chinese entity relationship dataset; Step S2: Define the ancient Chinese entity relationship dataset and set The ancient Chinese entity relationship dataset of ancient Chinese texts is , for Middle Ancient Chinese text; Ancient Chinese text is , for The words; the predefined relationship set is ,Include relationship categories, for Middle Class relations; extract entities and relations from ancient Chinese text and output them in the format of triples. Entities include main entities and object entities. The triples are expressed as , express The span of the main entity, represents the span of the guest entity in Y, express The relationship between the subject entity and the object entity; Step S3: constructing an ancient Chinese entity relationship extraction model, wherein the ancient Chinese entity relationship extraction model consists of a feature encoder, a fuser, an entity extractor, an entity relationship triple extractor, and an entity boundary alignment module; Step S4: taking the ancient Chinese text and the relation sentence formed by concatenating relations as inputs of the text feature encoder and the relation feature encoder respectively, and obtaining the word embedding feature vector of the ancient Chinese text and the word embedding feature vector of the relation sentence; the text feature encoder and the relation feature encoder are both composed of a pre-trained BERT model; Step S5: inputting the word embedding feature vector of the ancient Chinese text as the main information and the word embedding feature vector of the relational sentence as the auxiliary information into the text relational information fuser to obtain the text feature representation; inputting the word embedding feature vector of the relational sentence as the main information and the word embedding feature vector of the ancient Chinese text as the auxiliary information into the relational text information fuser to obtain the relational feature representation; Step S6: Concatenate the different word embedding feature vectors in the text feature representation to obtain the text feature concatenation representation , concatenate the text feature representation and the relation feature representation to obtain the relation feature concatenation representation ; Step S7: Input the text feature concatenation representation into the entity extractor to obtain the predicted entity boundary link label, and decode the text feature concatenation representation based on the predicted entity boundary link label to obtain the position of the entity head word and the tail word, concatenate the entity head word and the tail word to form an ancient Chinese entity set, and then calculate the entity boundary loss through the cross entropy loss function , optimize and update the model parameters of the feature encoder and entity extractor; Step S8: Input the concatenated representation of the relational features into the entity relation triple extractor to obtain the initial triple, and then calculate the initial triple loss through the cross entropy loss function ,Optimize and update the model parameters of feature encoder and entity relationship triple extractor; Step S9: input the predicted entity boundary link label and the initial triple into the entity boundary alignment module, align the predicted entity boundary link label and the initial triple based on the ancient Chinese entity set, and select the final entity relationship triple.

2. The ancient Chinese entity relationship extraction method based on multi-step information fusion and boundary alignment according to claim 1 is characterized by: The specific process of step S5 is: Step S5.1: The word embedding feature vector of the word and relational sentences Word embedding feature vector of class relation Input into the information interaction module, , , using multi-head attention to interactively fuse the word embedding feature vectors of ancient Chinese text and the word embedding feature vectors of related sentences; specifically: Will As the query vector in the attention mechanism ,Will As the key vector in the attention mechanism Sum value vector , to perform cross attention calculation; As the query vector in the attention mechanism ,Will As the key vector in the attention mechanism Sum value vector Perform cross attention calculation; the cross attention calculation process is expressed as: ； ； ； ； In the formula, , , Respectively represent the query vector when cross attention is calculated , key vector Sum value vector The projection parameter matrix of , , Respectively represent the query vector when cross attention is calculated , key vector Sum value vector The projection parameter matrix of for The size of the second dimension of for The size of the second dimension of for The transpose of for The transpose of express function; represents the attention mechanism; Step S5.2: The output of the cross attention calculation is used as the input of the residual module. After residual calculation, the text feature representation is obtained. and relational feature representation , expressed as: ； ； In the formula, Represents a normalization operation.

3. The ancient Chinese entity relationship extraction method based on multi-step information fusion and boundary alignment according to claim 2 is characterized by: The specific process of step S7 is: Step S7.1: Input the text feature concatenation representation into the entity boundary link label prediction module, and record the ancient Chinese text by setting an entity boundary link matrix The link information between the head word and the tail word of the entity in the prediction entity boundary link label ; Step S7.2: Link the predicted entity boundaries to labels Input the label decoding module, decode it in the label decoding module according to the first preset rule to obtain the entity's head word and tail word bigram; compare the entity's head word and tail word bigram with the ancient Chinese text Matching is performed, and according to the matching results, all words in the corresponding span of the entity's head word and tail word are concatenated to obtain a complete entity, and all complete entities constitute an ancient Chinese entity set.

4. The ancient Chinese entity relationship extraction method based on multi-step information fusion and boundary alignment according to claim 3 is characterized by: In step S7.1, an entity boundary link matrix is ​​set to record the ancient Chinese text The link information between the head word and the tail word of the entity in the prediction entity boundary link label The specific process is: Unify the entity boundary link matrix size to , is the number of characters in the ancient Chinese text, and the first prediction label is filled in the entity boundary link matrix through the score of the entity boundary link scoring function. The filled first prediction label is used to indicate whether the corresponding word is the head word or the tail word of the entity; when the first prediction label is "1", it indicates that the corresponding word is the head word or the tail word of the entity, and when the first prediction label is "0", it indicates that the corresponding word is not the head word or the tail word of the entity; wherein, the row element of the entity boundary link matrix is ​​the head word of the entity, and the column element of the entity boundary link matrix is ​​the tail word of the entity; Among them, the entity boundary link scoring function is expressed as: ； In the formula, , , , is the trainable parameter matrix; is the activation function; Indicates the probability of the head word and tail word of a certain type of entity in ancient Chinese text; Indicates the first word; Indicates the first word; Represents the ordinal number of the entity type in the entity type collection; Through the parameter matrix , , is the embedding dimension, Representing real number domain, adaptive learning , and finally through the activation function Calculate the classification of the first predicted label : ； In the formula, is the activation function, is the first predicted label for each filled position in the entity boundary link matrix; when When it is greater than the set threshold, the i-th word is considered to be the entity head word and the j-th word is the entity tail word.

5. The ancient Chinese entity relationship extraction method based on multi-step information fusion and boundary alignment according to claim 4 is characterized by: The specific process of step S8 is: Step S8.1: The concatenated representation of the relational features is input into the entity relation link label prediction module in the entity relation triple extractor, and an entity relation link matrix is ​​set up to record the link information of entities and relations in the ancient Chinese text, that is, to predict the entity relation link label. ; Step S8.2: Link the predicted entity relationship labels Input the label decoding module, and decode it in the label decoding module according to the second preset rule to obtain the main entity head word and the relationship bigram, the object entity head word and the relationship bigram, and the main entity head word and the object entity head word bigram; Step S8.3: Obtain an initial triple based on the main entity head word and relation bigram, the guest entity head word and relation bigram, and the main entity head word and guest entity head word bigram.

6. The ancient Chinese entity relationship extraction method based on multi-step information fusion and boundary alignment according to claim 5 is characterized by: In step S8.1, an entity relationship link matrix is ​​set up to record the link information of entities and relationships in the ancient Chinese text, that is, to predict the entity relationship link label The specific process is: Assume that the dimension size of the entity relationship link matrix is ; The entity relationship link matrix includes three parts: the main entity and the object entity link, the main entity and the relationship link, and the object entity and the relationship link; the entity relationship link matrix is ​​filled with the second prediction label through the score of the entity relationship link scoring function, and the second prediction label includes two types: "1" and "0"; In the link between the main entity and the guest entity, when the second prediction label is "1", it means that the row element and column element corresponding to the filling position in the entity relationship link matrix are the main entity head word and the guest entity head word bigrams, and when the second prediction label is "0", it means that the corresponding row element and column element are not the main entity head word and the guest entity head word bigrams, where both the row element and the column element are ancient Chinese texts; In the main entity and relationship link part, when the second prediction label is "1", it means that the row element and column element corresponding to the filling position in the entity relationship link matrix are the main entity head word and the relationship bigram. When the second prediction label is "0", it means that the corresponding row element and column element are not the main entity head word and the relationship bigram, where the row element is the ancient Chinese text and the column element is the predefined relationship. In the object entity and relationship link part, when the second prediction label is "1", it means that the row element and column element corresponding to the filling position in the entity relationship link matrix are the object entity header word and the relationship bigram, and when the second prediction label is "0", it means that the corresponding row element and column element are not the object entity header word and the relationship bigram, where the row element is the predefined relationship and the column element is the ancient Chinese text; Wherein, the entity relationship link scoring function is expressed as: ； In the formula, , , , is a trainable parameter; is the activation function; Represents the first row in the entity relationship link matrix elements, Represents the column in the entity relationship link matrix elements; Through the parameter matrix Adaptive Learning ; Finally, through the activation function Calculate the classification of the second predicted label : ； In the formula, The second predicted label filled in the entity relationship link matrix; when When it is greater than the set threshold, the corresponding second prediction label is "1", otherwise it is "0".

7. The ancient Chinese entity relationship extraction method based on multi-step information fusion and boundary alignment according to claim 6 is characterized by: The specific process of obtaining the initial triples based on the main entity head word and the relationship bigram, the object entity head word and the relationship bigram, and the main entity head word and the object entity head word bigram is as follows: based on the classification results, the combinations with the same relationship in the main entity head word and the relationship bigram and the object entity head word and the relationship bigram are merged into triples, namely, the main entity head word, the relationship, and the object entity head word, and the triples that are not in the main entity head word and the object entity head word bigram are excluded from all the merged triples to obtain the initial triples; Excluding triples that are not in the main entity head word and the guest entity head word bigram among all triples obtained by merging means verifying the triples using the main entity head word and the guest entity head word bigram, comparing whether the main entity head word and the guest entity head word in the triples are consistent with the main entity head word and the guest entity head word bigram, and excluding the corresponding triples if they are inconsistent.

8. The ancient Chinese entity relationship extraction method based on multi-step information fusion and boundary alignment according to claim 7 is characterized by: The first preset rule is: obtain the row elements and column elements corresponding to all the first predicted labels that are "1" in the entity boundary link matrix to form the entity's head word and tail word bigrams; The second preset rule is: Obtain the row and column element combinations corresponding to all second prediction labels that are "1" in the entity relationship link matrix, then count the subscripts of the positions of the row and column element combinations in the entity relationship link matrix, and classify the obtained row and column element combinations according to the set standard; The criteria are set as: When the row element subscript and column element subscript of the entity relationship link matrix are both less than or equal to Z, it is classified as a binary of the main entity head word and the object entity head word; When the row element subscript of the entity relationship link matrix is ​​less than or equal to Z, and the column element subscript is greater than Z, it is classified as the main entity head word and the relationship bigram; When the row element subscript of the entity relationship link matrix is ​​greater than Z and the column element subscript is less than or equal to Z, it is classified as the object entity head word and relationship bigram.

9. The ancient Chinese entity relationship extraction method based on multi-step information fusion and boundary alignment according to claim 8 is characterized by: According to the classification of the first predicted label, the predicted entity boundary link label and real entity boundary link labels Input to the cross entropy loss function, and use the cross entropy loss function to optimize the parameters of the entity extractor, expressed as: ； In the formula, represents entity boundary loss; express and equal probability; According to the second predicted label classification probability, the predicted entity relationship link label and real entity relationship link labels Input to the cross entropy loss function, and use the cross entropy loss function to update and optimize the parameters of the entity relationship triple extractor, which is expressed as: ； In the formula, represents the initial triplet loss; express and equal probability; Among them, the real entity boundary link label and real entity relationship link labels All of them are obtained from the ancient Chinese entity relationship dataset.

10. The ancient Chinese entity relationship extraction method based on multi-step information fusion and boundary alignment according to claim 9 is characterized in that: The specific process of step S9 is: use the head word of the entity in the predicted entity boundary link label to match the head word of the entity in the initial triple. When the head word of the entity in the predicted entity boundary link label is the same as the head word of the entity in the initial triple, filter out the initial triple with the same head word of the entity, and fill the corresponding complete entity in the Middle Chinese entity set in step S7.2 and the predicted entity boundary link label into the corresponding position in the filtered initial triple to obtain the final entity relationship triple. When the head word of the entity in the predicted entity boundary link label is different from the head word of the entity in the initial triplet, the corresponding initial triplet is removed.

Images