Relationship extraction method, device and equipment integrated with prior relation knowledge

Abstract

The invention discloses a relation extraction method, device and equipment integrated with prior relation knowledge, and belongs to the technical field of relation extraction. A representation fusion model based on an attention mechanism is designed to help to extract a relationship. Firstly, a vector coded by a prior relation label is used as relation information, and a relation embedding matrix is replaced with less relation information. In order to obtain vectors with rich semantic information for relation extraction, an attention mechanism is used to selectively fuse important semantic features between sentences and relation tags. The relation labels are effectively utilized to enrich relation information and improve semantic interaction between sentences and relations. The fusion layer designed by the invention plays a crucial role in promoting the process and improving the overall performance of simple structure relation extraction, so that a more accurate prediction effect is obtained.

Description

Technical Field

[0001] The present invention relates to the technical field of natural language processing, and in particular to a method, device and equipment for extracting a relationship by incorporating priori relationship knowledge. Background Art

[0002] Knowledge is the result of human beings' objective understanding of the world in practice. In the information age, knowledge graph is an important branch of knowledge engineering, which describes the concepts in the physical world and their relationships in a structured symbolic form. How to obtain structured information to build a knowledge graph has become a major challenge. Information extraction, as a technology that can automatically extract structured information, has gradually attracted widespread attention.

[0003] In the field of natural language processing, relation extraction is one of the basic tasks of information extraction, and its purpose is to correctly identify the relationship between entities in the text. The significance of relation extraction is to provide support for downstream tasks such as knowledge graph construction, information retrieval and question-answering systems. Since the development of relation extraction, there are mainly three relation extraction methods: fully supervised relation extraction, semi-supervised relation extraction and remotely supervised relation extraction. The semi-supervised relation extraction method and the remotely supervised relation extraction method focus on weakly labeled samples with a lot of noise; the fully supervised relation extraction method is divided into two categories: pipeline-based method and joint extraction method. The joint extraction method extracts entities and relations at the same time. The pipeline-based method regards named entity recognition and relationship extraction as two independent tasks, that is, first build a model for entity recognition, and then perform relationship extraction on the basis of entity recognition. Compared with the joint extraction model, the pipeline-based method can be used as a plug-and-play plug-in, which is compatible with a variety of excellent named entity recognition methods.

[0004] With the development of pre-trained language models based on the Transformers architecture, thanks to large-scale training corpora, pre-trained language models have shown excellent performance in representing texts, covering a variety of natural language processing tasks. Especially in the field of relationship extraction, the use of pre-trained language models to encode target texts has been widely used in deep learning-based methods. Existing deep learning-based methods divide the relationship extraction task into three main steps: feature information extraction, entity recognition, and relationship classification. Algorithms represented by the Casrel and TPlinker series automatically perform feature extraction and classification through pre-trained models. This end-to-end approach can make full use of large-scale data sets for training, thereby learning feature representations with more generalization capabilities.

[0005] In recent years, mainstream relation extraction models have made significant progress in dealing with extraction tasks under complex relations. For example, Chinese patent CN116975284A proposes an entity relation extraction method, device and storage medium based on prior knowledge. By using prior knowledge and pre-trained models to fuse to build an entity relation extraction model capable of relation filtering, the training speed and reasoning speed of entity relation extraction are accelerated, and the accuracy of the model is significantly improved. However, in addition to studying new models to solve the problem of relationship overlap, it is equally important to obtain more appropriate representation vectors for joint entity and relationship extraction. Relation information plays a very important role in improving the performance of relation extraction models. Most current studies use relation embedding matrices to represent relation information, while ignoring the prior relation information contained in the data set: relation labels. In addition, most studies do not fully explore the potential of relation information or use complex model structures to utilize relation information. Summary of the invention

[0006] In order to solve the problems existing in the prior art, the present invention proposes a relationship extraction method, device and equipment that incorporates prior relationship knowledge, which are used for relationship extraction to make full use of relationship information. The relationship information is encoded by the relationship label through the preprocessing model, and the encoding layer is shared with the sentence, which is more conducive to the subsequent interaction between the sentence and the relationship. In order to enhance the semantic interaction between sentences and relationships, the present invention uses an attention mechanism to fuse the additional semantic information of sentences and relationships. Specifically, first, by fine-tuning the pre-trained language model, it can automatically learn and recognize complex semantic features. Secondly, the vector encoded by the prior relationship label is used as the relationship information, and the relationship embedding matrix is ​​replaced with less relationship information. Subsequently, the attention mechanism is used to selectively fuse the sentence representation and the relationship label, so as to obtain vector features rich in semantic information, realize the extraction of entity pairs and relationships in unstructured text, and solve the problems mentioned in the above background technology.

[0007] To achieve the above object, the present invention provides the following technical solutions:

[0008] In a first aspect of the present invention, the present invention proposes a relationship extraction method incorporating prior relationship knowledge, wherein the relationship extraction method comprises the following specific steps:

[0009] Obtain part-of-speech sequences corresponding to multiple training texts and multiple relationship label texts respectively;

[0010] Inputting the part-of-speech sequence corresponding to each training text and each relationship label text into the pre-trained sentence encoder to generate a sentence vector corresponding to each training text and a relationship vector corresponding to multiple relationship label texts;

[0011] Inputting the sentence vector corresponding to each training text and the relationship vectors corresponding to the multiple relationship label texts into the specific relationship attention mechanism module to generate an updated sentence vector corresponding to each training text and an updated relationship vector corresponding to the multiple relationship label texts;

[0012] Input the updated sentence vector corresponding to each training text into the subject tagger to generate the possible position of the subject in the sentence vector;

[0013] The word vector sequence of the possible position of the subject in the updated sentence vector is processed by average pooling to generate the feature vector of the candidate subject;

[0014] The feature vector of the candidate subject, the updated sentence vector, and the updated relationship vector are merged to generate the feature vector corresponding to each training text;

[0015] The feature vector corresponding to each training text is input into a relation-specific object tagger to generate the possible position of the object in the sentence vector.

[0016] In a second aspect of the present invention, the present invention further provides a relationship extraction device incorporating prior relationship knowledge, the device comprising:

[0017] A data acquisition module, used to acquire part-of-speech sequences corresponding to a plurality of training texts and a plurality of relationship label texts;

[0018] A data encoding module, used for inputting the part-of-speech sequence corresponding to each training text and each relationship label text into a pre-trained sentence encoder to generate a sentence vector corresponding to each training text and a relationship vector corresponding to a plurality of relationship label texts;

[0019] A data processing module, used for inputting the sentence vector corresponding to each training text and the relationship vectors corresponding to the multiple relationship label texts into the specific relationship attention mechanism module, and generating an updated sentence vector corresponding to each training text and an updated relationship vector corresponding to the multiple relationship label texts;

[0020] The data decoding module is used to input the updated sentence vector corresponding to each training text into the subject tagger to generate the possible position of the subject in the sentence vector;

[0021] And the word vector sequence of the possible position of the subject in the updated sentence vector is processed by average pooling to generate the feature vector of the candidate subject;

[0022] and fusing the feature vector of the candidate subject, the updated sentence vector, and the updated relationship vector to generate a feature vector corresponding to each training text;

[0023] And the feature vector corresponding to each training text is input into the relation-specific object tagger to generate the possible position of the object in the sentence vector.

[0024] In a third aspect of the present invention, the present invention further provides an electronic device, the electronic device comprising:

[0025] at least one processor; and,

[0026] a memory communicatively connected to the at least one processor; wherein,

[0027] The memory stores a computer program executable by the at least one processor, and the computer program is executed by the at least one processor so that the at least one processor can execute the relationship extraction method incorporating prior relationship knowledge as described in the first aspect of the present invention.

[0028] The beneficial effects of the present invention are as follows: the present invention proposes a method, device and apparatus for relation extraction that incorporates prior relation knowledge. Compared with the prior art, the present invention uses a relation vector encoded by a preprocessing model to replace the relation embedding matrix method. The model learns more relation information to enhance the information interaction between sentences and relations. In addition, in order to obtain a vector with rich semantic information for relation extraction, the present invention uses a specific relation attention mechanism to selectively fuse important semantic features between sentences and relation labels. This makes the fusion layer of the present invention more comprehensive and simpler in structure. BRIEF DESCRIPTION OF THE DRAWINGS

[0029] Figure 1 A schematic diagram of the process flow of a method for extracting a relationship by incorporating prior relationship knowledge according to an embodiment of the present invention;

[0030] Figure 2 A schematic diagram of a network structure of an embodiment of the present invention;

[0031] Figure 3 A structural diagram of a relationship extraction device incorporating prior relationship knowledge according to an embodiment of the present invention;

[0032] Figure 4 A structural diagram of a relation extraction device incorporating prior relation knowledge according to a preferred embodiment of the present invention;

[0033] Figure 5 Schematic diagram of the structure of an electronic device according to an embodiment of the present invention. DETAILED DESCRIPTION

[0034] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0035] A relation embedding matrix usually refers to a two-dimensional matrix in which each row (or column) represents a specific relation type. Each relation type is represented as a fixed-length vector that makes up the entire matrix. In a knowledge graph, a relation embedding matrix is ​​used to represent multiple relation types between different entities; these embeddings can be used to predict unknown relations or complete knowledge graph completion tasks. Suppose we have a knowledge graph that contains three relation types: "friends", "colleagues", and "family". We can use a 3-row N-column matrix to represent these three relations, with each row corresponding to an embedding vector for a relation type.

[0036] A relation information vector is a single vector that represents a specific relation instance. Unlike a row vector in a relation embedding matrix, a relation information vector may contain more information about a specific relation instance, such as the strength and directionality of the relation. If we consider a specific relation instance in a knowledge graph, such as "Zhang San - Friend - Li Si", then this relation instance can be encoded into a relation information vector that may contain specific attributes of the friendship between Zhang San and Li Si. The present invention replaces the relation embedding matrix with a relation information vector, thereby learning more relation information to enhance the information complementarity between sentences and relations.

[0037] The embodiment of the present invention provides a method for extracting a relationship that incorporates prior relationship knowledge. The execution subject of the method for extracting a relationship that incorporates prior relationship knowledge includes but is not limited to at least one of the electronic devices such as a server and a terminal that can be configured to execute the method provided by the embodiment of the present application. In other words, the method for extracting a relationship can be executed by software or hardware installed on a terminal device or a server device, and the software can be a blockchain platform. The server includes but is not limited to: a single server, a server cluster, a cloud server or a cloud server cluster, etc.

[0038] Reference Figure 1 FIG. 1 is a flow chart of a method for extracting a relationship by incorporating prior relationship knowledge according to an embodiment of the present invention. In the embodiment of the present invention, the method for extracting a relationship by incorporating prior relationship knowledge comprises the following steps S1-S6:

[0039] 101. Obtain part-of-speech sequences corresponding to multiple training texts and multiple relationship label texts respectively; in some embodiments of the present invention, the training text is a text for entity relationship extraction, and the specific content of the text can be determined based on the actual business scenario. The relationship label text is a text that annotates entities and the relationships between them; this embodiment uses the vector encoded by the prior relationship label as the relationship information, and replaces the relationship embedding matrix with less relationship information to reduce the model training complexity during relationship extraction and improve the accuracy of relationship extraction.

[0040] In an embodiment of the present invention, the public data sets TACRED, TACREV and RE-TACRED can be used to obtain training texts and relationship sets, wherein the TACREV and RE-TACRED data sets are improved data sets of the TACRED data set, and some data that were misclassified in TACRED have been re-annotated. The training set of the TACRED data set and the TACRV data set includes 68,124 statements, the validation set includes 22,631 statements, and the test set includes 15,509 statements. The entire data set covers 42 different relationships. The training set of the RE-TACRED data set includes 58,464 statements, the validation set includes 19,584 statements, and the test set includes 13,418 statements, covering 40 relationships.

[0041] In the embodiment of the present invention, the part-of-speech sequence corresponding to each training text includes each word segmentation, classification mark and separation mark of each training text, and the part-of-speech sequence corresponding to each relationship tag text includes each word segmentation, classification mark and separation mark of each relationship tag. Therefore, this embodiment needs to preprocess the training text and the built-in relationship set. First, remove the special characters in the text; then use the pre-trained model to segment the text, and finally add start marks and end marks to the beginning and end of the sentences in the text to obtain the pre-processed text. That is, sentence W is processed as:

[0042] W=[CLS,w 1 ,…,w n ,SEP]

[0043] Among them, the CLS tag is at the beginning of the text sequence, the SEP tag is used to separate different text sequences, and the w 1 ,…,w n Represents n tokens of a text sequence.

[0044] 102. Input the part-of-speech sequence corresponding to each training text and each relationship label text into the pre-trained sentence encoder to generate a sentence vector corresponding to each training text and a relationship vector corresponding to multiple relationship label texts;

[0045] In the embodiment of the present invention, the pre-processed training text and relationship information are input into a shared sentence encoder for processing to obtain a corresponding vector representation.

[0046] In some preferred embodiments, the sentence encoder may use a bert-base-uncaused or Roberta-large model. The pre-trained parameters are fine-tuned, and the initial learning rates assigned to the above two pre-trained language models are 0.00003 and 0.000005 respectively.

[0047] The preprocessed text is input into the sentence encoder for processing. The BERT model composed of N layers of Transformers encodes the sentence into a vectorized expression. The Transformer module is constructed by the attention mechanism. The last hidden layer of BERT is used as the sentence vector.

[0048] The relation information is encoded by BERT on the relation label and the BERT encoder is shared with the sentence embedding to encode the relation label into a vectorized representation. To obtain the embedding vector for each label, the output of the label text is summed after BERT processing. Each encoded embedding vector is concatenated to obtain the relation embedding.

[0049] Furthermore, the formula for calculating the sentence vector is:

[0050] H S =BERT(S)

[0051] Among them, H S Represents the sentence vector corresponding to each training text, H S ∈R n×d , n is the number of words in the sentence, and d is the vector dimension of the sentence embedding encoded by BERT. BERT(*) denotes the BERT model processing.

[0052] Furthermore, the formula for calculating the relationship vector is:

[0053]

[0054] in, m is the relation label text r i The number of tags to include. is the BERT-encoded relation vector.

[0055] Furthermore, the formula for the connection relationship vector is:

[0056]

[0057] Among them, H R Represents the relationship vector corresponding to each training text, H R∈R |R|×d is the number of relation labels in the relation set.

[0058] 103. Input the sentence vector corresponding to each training text and the relationship vectors corresponding to the multiple relationship label texts into a specific relationship attention mechanism module to generate an updated sentence vector corresponding to each training text and an updated relationship vector corresponding to the multiple relationship label texts;

[0059] In an embodiment of the present invention, the present invention utilizes an attention mechanism to dynamically assign a significance weight to each relationship tag according to its relevance to a specific word in a sentence, thereby enriching the information of all words in a sentence vector.

[0060] In order to enhance the representation ability of the network and prevent gradient vanishing, the obtained sentence vector is concatenated with the original sentence vector using residual connection to obtain the final sentence vector for relation extraction.

[0061] As in the above steps, the attention mechanism and residual connection are used to update the relation vector.

[0062] First, the correlation between the word vector and the relation vector is calculated, and then the obtained relation vector is connected with the original relation vector to finally get the final relation vector.

[0063] In an embodiment of the present invention, the method for generating the updated sentence vector corresponding to each training text includes:

[0064] The first Q vector is calculated based on the sentence vector corresponding to each training text and the corresponding trainable weight;

[0065] According to the relationship vector corresponding to each training text and the two corresponding trainable weights, a first K vector and a first V vector are calculated;

[0066] According to the first Q vector, the first K vector, and the first V vector, the first sentence vector is calculated using the self-attention mechanism;

[0067] According to the sentence vector corresponding to each training text and the first sentence vector, an updated sentence vector corresponding to each training text is calculated.

[0068] Specifically, the formula for calculating the updated sentence vector corresponding to each training text is:

[0069] Q S =H S W Q ,K R =H R W K ,V R =H R W V

[0070]

[0071] Among them, Q S represents the first Q vector, K R represents the first K vector, V R represents the first V vector, H SA represents the first sentence vector, d k =d / N, where N is the number of transformer layers in BERT. Q , W K , W V are the corresponding trainable weight vectors respectively.

[0072] In an embodiment of the present invention, the generation method of the relationship vector of the updated sentence vector corresponding to each training text includes:

[0073] According to the relationship vector corresponding to each training text and the corresponding trainable weight, a second Q vector is calculated;

[0074] According to the updated sentence vector corresponding to each training text and the two corresponding trainable weights, a second K vector and a second V vector are calculated;

[0075] According to the second Q vector, the second K vector, and the second V vector, a first relationship vector is calculated using a self-attention mechanism;

[0076] According to the relationship vector corresponding to each training text and the first relationship vector, an updated relationship vector corresponding to each training text is calculated.

[0077] Specifically, the formula for calculating the updated relationship vector corresponding to each training text is:

[0078] Q R =H R W Q ,

[0079]

[0080]

[0081] Among them, Q R represents the second Q vector, K S represents the second K vector, V S represents the second V vector, H RA represents the first relationship vector, W Q , W K , W Vare the corresponding trainable weight vectors respectively.

[0082] 104. Input the updated sentence vector corresponding to each training text into the subject tagger to generate the possible position of the subject in the sentence vector;

[0083] In an embodiment of the present invention, the present invention uses a subject tagger to predict the possible position of the subject in the final representation of the sentence vector. The process of calculating the possible position of a subject includes:

[0084] Two binary classifiers are used to determine the likelihood of the start and end positions of the subject, so that the subject can be accurately and effectively identified in text data. The calculation formula is:

[0085]

[0086] in, and It represents the probability of the token being at the start or end position of the subject. The output of the classifier is the probability distribution of each position being marked as the start or end. Finally, the point with the highest probability is selected as the start and end index to confirm the start and end position of the subject. Is to update the sentence vector The final representation of the i-th token in W start_s ,W end_s ,b start_s ,b end_s is a trainable parameter.

[0087] 105. The word vector sequence of the possible position of the subject in the updated sentence vector is processed by average pooling to generate the feature vector of the candidate subject;

[0088] In this embodiment of the present invention, in order to better obtain the possible position of the subject in the updated sentence vector, this embodiment performs average pooling processing on it to reduce the dimension of the input data, thereby reducing the number of parameters that need to be processed during model training, which is expressed as:

[0089]

[0090] Among them, s k Represents the vector of the kth candidate subject, meanpool(*) represents the average pooling operation, and the subscripts s_k_start and s_k_end represent s k exist The start and end index in .

[0091] 106. Fusing the feature vector of the candidate subject, the updated sentence vector, and the updated relationship vector to generate a feature vector corresponding to each training text;

[0092] In the embodiment of the present invention, the embodiment splices and predicts the candidate subject vector, sentence vector and relationship label vector prepared for object extraction. The process of calculating feature fusion includes:

[0093] Concatenate predictions to prepare candidate subject vectors, sentence vectors, and relation label vectors for object extraction.

[0094] Furthermore, the formula for calculating feature fusion is:

[0095]

[0096] Among them, h i ' jk is the final representation of the i-th token, which combines sentence information, relation label information, and candidate subject information. is the updated relation vector The final representation of the j-th relation label in , Is to update the sentence vector The final representation of the i-th token in W h ,b h are training parameters.

[0097] 107. Input the feature vector corresponding to each training text into a relation-specific object tagger to generate the possible position of the object in the sentence vector. In an embodiment of the present invention, a relation-specific object tagger is used to predict the possible position of the object in the final representation of the sentence vector.

[0098] The process of calculating the possible location of an object involves:

[0099] Two binary classifiers are used to determine the probability of the start and end positions of an object, so that objects can be accurately and effectively identified in text data. The calculation formula is:

[0100]

[0101] in, and are the probabilities of the start and end positions of the object, respectively, and W start_o , W end_o , b start_o , b end_o is a trainable parameter.

[0102] Figure 2 is a network model involved in the relation extraction scheme incorporating prior relation knowledge in the embodiment of the present invention, such as Figure 2As shown in the figure, it includes three core parts: embedding layer, fusion layer and decoding layer, each of which undertakes different tasks, thereby achieving efficient multi-relation extraction. Embedding layer: First, the sentence and relationship labels are encoded into vectorized representations through the BERT model. In this process, the sentence and relationship labels share the same encoding method, so that their semantic representations can be more naturally fused. The vector representation of the relationship label not only carries rich semantic information, but also helps the subsequent semantic interaction. Fusion layer: In the fusion layer, the attention mechanism is introduced to realize the interaction of semantic information between the sentence and the relationship label. Specifically, through the attention weight calculation, the model can give higher weights to the parts that are more related to the target relationship or entity, so as to focus more on important information during feature fusion. In addition, the residual connection is used to avoid the problem of gradient disappearance and ensure that the semantic information is not lost during the transmission process. The fused representation vector can more accurately reflect the semantics related to a specific relationship in the sentence, thereby enhancing the overall feature expression. Decoding layer: The decoding layer includes a subject tagger and a relationship-specific object tagger, which are used to locate the subject and object in the sentence, respectively. The subject tagger predicts the position of the subject through a binary classifier, while the relation-specific object tagger combines the subject vector, sentence vector, and relation label vector to accurately extract the relations and objects associated with the subject. Through this multi-level tagging mechanism, the model is able to achieve accurate relation extraction in complex multi-relation scenarios. The network structure brings significant improvement. First, through the introduction of the attention mechanism, the model can dynamically adjust the contribution of different words to relations and entities, thereby enhancing the ability to capture key semantics. Second, by leveraging the prior knowledge of relation labels, the model can interact with sentence information more naturally, reducing the problem of semantic information loss caused by traditional embedding methods. In addition, the model shows higher robustness and accuracy in handling noisy data and complex sentences. Overall, the attention mechanism in the fusion layer effectively strengthens the semantic interaction between sentences and relations, making it more accurate and adaptable in multi-relation extraction tasks.

[0103] In a preferred embodiment of the present invention, the present invention calculates the binary cross entropy loss and adjusts the model parameters according to the loss to improve the accuracy of relation extraction. The present invention applies binary cross entropy as a training target, which enables us to effectively train the model and improve its performance on the relation extraction task. The formula for calculating binary cross entropy is:

[0104]

[0105] in represents the cost of the task, t is the label 0 / 1, when t is 0, it means the classification label is false; when t is 1, it means the classification label is true, and p is the probability that the tag is the start and end position of the entity. By substituting the probability of the decoding layer part into the equation and summing the loss, a joint loss function is obtained to update the model parameters.

[0106] In some embodiments of the present invention, Figure 3 is a structural diagram of a relation extraction device incorporating prior relation knowledge according to an embodiment of the present invention. Figure 3 As shown, the device 200 includes:

[0107] A data acquisition module 201 is used to acquire part-of-speech sequences corresponding to a plurality of training texts and a plurality of relationship tag texts;

[0108] A data encoding module 202, used for inputting the part-of-speech sequence corresponding to each training text and each relationship label text into a pre-trained sentence encoder to generate a sentence vector corresponding to each training text and a relationship vector corresponding to a plurality of relationship label texts;

[0109] A data processing module 203 is used to input the sentence vector corresponding to each training text and the relationship vectors corresponding to the multiple relationship label texts into a specific relationship attention mechanism module to generate an updated sentence vector corresponding to each training text and an updated relationship vector corresponding to the multiple relationship label texts;

[0110] A data decoding module 204 is used to input the updated sentence vector corresponding to each training text into the subject tagger to generate the possible position of the subject in the sentence vector;

[0111] And the word vector sequence of the possible position of the subject in the updated sentence vector is processed by average pooling to generate the feature vector of the candidate subject;

[0112] and fusing the feature vector of the candidate subject, the updated sentence vector, and the updated relationship vector to generate a feature vector corresponding to each training text;

[0113] And the feature vector corresponding to each training text is input into the relation-specific object tagger to generate the possible position of the object in the sentence vector.

[0114] In some preferred embodiments of the present invention, Figure 4 FIG. 1 is a structural diagram of a relation extraction device incorporating prior relation knowledge according to a preferred embodiment of the present invention. Figure 4 As shown, the data decoding module 204 further includes: a subject prediction unit 401, which is used to input the updated sentence vector corresponding to each training text into the subject tagger to generate the possible position of the subject in the sentence vector;

[0115] A data pooling unit 402 is used to perform an average pooling process on the word vector sequence of the possible position of the subject in the updated sentence vector to generate a feature vector of the candidate subject;

[0116] A data fusion unit 403 is used to fuse the feature vector of the candidate subject, the updated sentence vector, and the updated relationship vector to generate a feature vector corresponding to each training text;

[0117] The object prediction unit 404 is used to input the feature vector corresponding to each training text into the relation-specific object tagger to generate the possible position of the object in the sentence vector.

[0118] In an embodiment of the present invention, Figure 5 Schematic diagram of the structure of an electronic device according to an embodiment of the present invention. Figure 5 As shown, the electronic device may include a processor 10, a memory 11, a communication bus 12 and a communication interface 13, and may also include a computer program stored in the memory 11 and executable on the processor 10, such as an entity relationship extraction program.

[0119] In some embodiments, the memory 11 can be used as a built-in storage component of an electronic device, such as a built-in mobile hard disk of the device. In other implementation scenarios, the memory 11 can be converted into an external storage component of an electronic device, including but not limited to a pluggable mobile hard disk, a smart memory card (Smart Media Card, SMC), a secure digital card (Secure Digital, SD card), and a flash card (Flash Card), etc. on the electronic device. Furthermore, the configuration of the memory 11 can also take into account the built-in storage and external storage of the electronic device to form a comprehensive storage component. This memory 11 not only bears the heavy responsibility of storing applications and various types of data installed on the electronic device (for example, the code of the relationship extraction method program that incorporates prior relationship knowledge), but also plays the role of temporarily storing the output or to-be-output data.

[0120] In some embodiments, the processor 10 is constructed on the basis of an integrated circuit, which can be a single packaged integrated circuit or a combination of multiple integrated circuits with the same or different functions. These integrated circuits can include one or more central processing units (CPUs), microprocessors, digital signal processing chips, graphics processors, and a collection of various control chips. As the control center of the electronic device, the processor 10 closely connects the various components of the entire electronic device through various interfaces and lines. It realizes the diversified functions of the electronic device and processes related data by running or executing programs or modules stored in the memory 11 (for example, the code of the relationship extraction method program that incorporates prior relationship knowledge) and calling the data in the memory 11.

[0121] The communication bus 12 can be a bidirectional data bus (DB) for transmitting data information, a unidirectional address bus (AB) for transmitting address information issued by the CPU, or a control bus (CB) for transmitting control signals, timing signals, and status information, etc. These are based on the Peripheral Component Interconnect (PCI) bus or the Extended Industry Standard Architecture (EISA) bus, etc. The communication bus 12 is configured to realize the connection and communication between the memory 11 and at least one processor 10, etc. For ease of representation, only one thick line is used in the figure, but it does not mean that there is only one bus or one type of bus.

[0122] The present invention takes natural language text as the processing object, proposes a relationship extraction method, device and equipment that incorporates prior relationship knowledge, and helps extract relationships based on a representation fusion model of the attention mechanism. First, the vector encoded by the prior relationship label is used as the relationship information, and the relationship embedding matrix is ​​replaced with less relationship information. In order to obtain a vector with rich semantic information for relationship extraction, the attention mechanism is used to selectively fuse the important semantic features between sentences and relationship labels. The relationship labels are effectively used to enrich the relationship information and improve the semantic interaction between sentences and relationships, so as to realize the extraction of entity pairs and relationships in unstructured text.

[0123] Although the present invention has been described in detail with reference to the aforementioned embodiments, it is still possible for those skilled in the art to modify the technical solutions described in the aforementioned embodiments, or to make equivalent substitutions for some of the technical features therein. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the protection scope of the present invention.

[0124] A person skilled in the art can understand that all or part of the steps in the various methods of the above embodiments can be completed by instructing the relevant hardware through a program, and the program can be stored in a computer-readable storage medium, which can include: ROM, RAM, disk or CD, etc.

[0125] Although embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions and variations may be made to the embodiments without departing from the principles and spirit of the present invention, and that the scope of the present invention is defined by the appended claims and their equivalents.

Claims

1. A relation extraction method incorporating prior relation knowledge, characterized in that: The specific steps of the relationship extraction method are as follows: Obtain part-of-speech sequences corresponding to multiple training texts and multiple relationship label texts respectively; Inputting the part-of-speech sequence corresponding to each training text and each relationship label text into the pre-trained sentence encoder to generate a sentence vector corresponding to each training text and a relationship vector corresponding to multiple relationship label texts; Inputting the sentence vector corresponding to each training text and the relationship vectors corresponding to the multiple relationship label texts into the specific relationship attention mechanism module to generate an updated sentence vector corresponding to each training text and an updated relationship vector corresponding to the multiple relationship label texts; Input the updated sentence vector corresponding to each training text into the subject tagger to generate the possible position of the subject in the sentence vector; The word vector sequence of the possible position of the subject in the updated sentence vector is processed by average pooling to generate the feature vector of the candidate subject; The feature vector of the candidate subject, the updated sentence vector, and the updated relationship vector are merged to generate the feature vector corresponding to each training text; The feature vector corresponding to each training text is input into a relation-specific object tagger to generate the possible position of the object in the sentence vector.

2. The method for extracting a relation by incorporating prior relation knowledge according to claim 1, characterized in that: The part-of-speech sequence corresponding to each training text includes each word segmentation, classification mark and separation mark of each training text, and the part-of-speech sequence corresponding to each relationship tag text includes each word segmentation, classification mark and separation mark of each relationship tag.

3. The method for extracting a relation by incorporating prior relation knowledge according to claim 1, characterized in that: The method for generating the updated sentence vector corresponding to each training text includes: The first Q vector is calculated based on the sentence vector corresponding to each training text and the corresponding trainable weight; According to the relationship vector corresponding to each training text and the two corresponding trainable weights, a first K vector and a first V vector are calculated; According to the first Q vector, the first K vector, and the first V vector, the first sentence vector is calculated using the self-attention mechanism; According to the sentence vector corresponding to each training text and the first sentence vector, an updated sentence vector corresponding to each training text is calculated.

4. The method for extracting a relation incorporating prior relation knowledge according to claim 1 or 3, characterized in that: The method for generating the updated relationship vector corresponding to each training text includes: According to the relationship vector corresponding to each training text and the corresponding trainable weight, a second Q vector is calculated; According to the updated sentence vector corresponding to each training text and the two corresponding trainable weights, a second K vector and a second V vector are calculated; According to the second Q vector, the second K vector, and the second V vector, a first relationship vector is calculated using a self-attention mechanism; According to the relationship vector corresponding to each training text and the first relationship vector, an updated relationship vector corresponding to each training text is calculated.

5. The method for extracting a relation by incorporating prior relation knowledge according to claim 1, characterized in that: The subject tagger and the object tagger are both two binary classifiers; the two binary classifiers of the subject tagger respectively determine the probabilities of the position where the subject starts and the position where the subject ends; the two binary classifiers of the object tagger respectively determine the probabilities of the position where the object starts and the position where the object ends.

6. The method for extracting a relation incorporating prior relation knowledge according to claim 1 or 5, characterized in that: The subject tagger and the object tagger are trained using binary cross entropy.

7. The method for extracting a relation by incorporating prior relation knowledge according to claim 6, characterized in that: The binary cross entropy is expressed as: in, represents binary cross entropy, t is the label 0 / 1, when t is 0, it means the classification label is false; when t is 1, it means the classification label is true, and p is the probability that the tag is the start and end position of the entity.

8. A relation extraction device incorporating prior relation knowledge, characterized in that: The device comprises: A data acquisition module, used to acquire part-of-speech sequences corresponding to a plurality of training texts and a plurality of relationship label texts; A data encoding module, used for inputting the part-of-speech sequence corresponding to each training text and each relationship label text into a pre-trained sentence encoder to generate a sentence vector corresponding to each training text and a relationship vector corresponding to a plurality of relationship label texts; A data processing module, used for inputting the sentence vector corresponding to each training text and the relationship vectors corresponding to the multiple relationship label texts into the specific relationship attention mechanism module, and generating an updated sentence vector corresponding to each training text and an updated relationship vector corresponding to the multiple relationship label texts; The data decoding module is used to input the updated sentence vector corresponding to each training text into the subject tagger to generate the possible position of the subject in the sentence vector; And the word vector sequence of the possible position of the subject in the updated sentence vector is processed by average pooling to generate the feature vector of the candidate subject; and fusing the feature vector of the candidate subject, the updated sentence vector, and the updated relationship vector to generate a feature vector corresponding to each training text; And the feature vector corresponding to each training text is input into the relation-specific object tagger to generate the possible position of the object in the sentence vector.

9. The relationship extraction device incorporating prior relationship knowledge according to claim 8, characterized in that: The data decoding module comprises: A subject prediction unit, which is used to input the updated sentence vector corresponding to each training text into the subject tagger to generate the possible position of the subject in the sentence vector; A data pooling unit is used to average pool the word vector sequence of the possible position of the subject in the updated sentence vector to generate the feature vector of the candidate subject; A data fusion unit, used to fuse the feature vector of the candidate subject, the updated sentence vector, and the updated relationship vector to generate a feature vector corresponding to each training text; The object prediction unit is used to input the feature vector corresponding to each training text into the relation-specific object tagger to generate the possible position of the object in the sentence vector.

10. An electronic device, characterized in that: The electronic device comprises: at least one processor; and, a memory communicatively connected to the at least one processor; wherein, The memory stores a computer program executable by the at least one processor, and the computer program is executed by the at least one processor so that the at least one processor can perform the relationship extraction method incorporating prior relationship knowledge as described in any one of claims 1 to 8.

Images