Entity relationship joint extraction method and device

Abstract

Embodiments of this application provide a method and apparatus for joint entity relationship extraction. The method includes: first, identifying all possible subjects in a text; then, setting a threshold K to extract the K most likely relations in the text; then, applying a relation-specific tagger to each subject, simultaneously identifying the objects corresponding to these K relations, and finally constructing a complete triple. The method and apparatus of embodiments of this application can improve the accuracy and robustness of the model.

Description

Technical Field

[0001] The embodiments of the present application relate to the field of natural language processing, and in particular to a method and device for joint extraction of entity relationships. Background Art

[0002] With the development of information technology, massive amounts of unstructured data are accumulating across various fields. Extracting useful information from this data has become a crucial research topic. Joint entity-relationship extraction, a key task in information extraction, aims to simultaneously identify entities and their relationships within text. This task plays a vital role in applications such as knowledge graph construction, information retrieval, and natural language understanding.

[0003] However, current joint entity-relationship extraction still faces numerous challenges. For example, the accuracy and robustness of the model still need to be improved when handling complex scenarios with multiple entities and multiple relationships. Furthermore, designing efficient joint entity-relationship extraction methods for low-resource languages ​​and domain-specific texts also requires urgent research and solutions. Summary of the Invention

[0004] In view of this, an embodiment of the present application provides a method and device for joint extraction of entity relationships, which can improve the accuracy and robustness of the model.

[0005] In a first aspect, a method for joint extraction of entity relations is provided, which is characterized by comprising: performing word embedding processing on an initial text to be subjected to triple extraction to obtain a text vector of a target text in the initial text; inputting the text vector into a fully connected neural network and a Sigmod function, calculating the probability that each position in the target text can be used as the starting position of a head entity and the probability that the head entity can end, and determining the head entity based on the probability that each position in the target text can be used as the starting position of a head entity and the probability that the head entity can end; fusing the text vector and the head entity vector to generate a fusion vector, inputting the fusion vector into a convolutional neural network to obtain a convolution feature vector of the target text, and inputting the convolution feature vector into a Softmax function to calculate the existence probability of each relationship in the target text, wherein the head entity vector is a vector corresponding to the head entity obtained from the text vector; selecting K relationships with the highest existence probability, and filtering them from a global relationship parameter matrix M oh and the second global relationship parameter matrix M ot The parameter vectors of K relations are selected respectively, two sub-parameter matrices are generated, the fusion vector is passed into the two sub-parameter matrices, and the probability that each position in the target text can be used as the starting position of the tail entity and the probability of the ending position of the tail entity are calculated by the Sigmod function, thereby constructing a complete triplet, where K is a positive integer.

[0006] In one possible implementation, word embedding processing is performed on the initial text to be subjected to triple extraction to obtain a text vector of the target text in the initial text, including: preprocessing the initial text and inputting it into a Bert pre-trained model for word embedding processing to obtain a text vector of the target text.

[0007] In one possible implementation, the initial text is preprocessed and input into a Bert pre-training model for word embedding processing to obtain a text vector of the target text, including: preprocessing the initial text and inputting into a Bert pre-training model for word embedding processing to obtain a word vector for each word in the initial text; combining the word vectors of n words obtained successively to obtain a text vector of the target text with a length of n, where n is a positive integer greater than 1.

[0008] In one possible implementation, a text vector is input into a fully connected neural network and a Sigmod function, and the probability that each position in the target text can be used as the starting position of a head entity and the probability that the head entity can be used as the ending position of the head entity are calculated, including: inputting the text vector into a fully connected layer for predicting the starting position of a head entity and a Sigmod function, and calculating the probability that each position in the target text can be used as the starting position of the head entity; inputting the text vector into a fully connected layer for predicting the ending position of a head entity and a Sigmod function in sequence, and calculating the probability that each position in the target text can be used as the ending position of the head entity.

[0009] In one possible implementation, a head entity is determined based on the probability that each position in the target text can be the starting position of the head entity and the probability that the head entity can be the ending position, including: combining words at positions above a first threshold in the probability that each position in the target text can be the starting position of the head entity to words at positions above a second threshold in the probability that each position in the target text can be the ending position of the head entity to obtain the head entity.

[0010] In a second aspect, a device for joint extraction of entity relations is provided, comprising: a word embedding processing module for performing word embedding processing on an initial text to be subjected to triple extraction, and obtaining a text vector of a target text in the initial text; a head entity tagging module for inputting the text vector into a fully connected neural network and a Sigmod function, and calculating the probability that each position in the target text can be used as the starting position of a head entity and the probability that the head entity can end, and determining the head entity based on the probability that each position in the target text can be used as the starting position of a head entity and the probability that the head entity can end; a relationship recognition module for fusing the text vector and the head entity vector to generate a fusion vector, inputting the fusion vector into a convolutional neural network to obtain a convolutional feature vector of the target text, and inputting the convolutional feature vector into a Softmax function to calculate the existence probability of each relationship in the target text, wherein the head entity vector is a vector corresponding to the head entity obtained from the text vector; and a tail entity tagging module for selecting K relationships with the highest existence probability, and extracting the relationship from the global relationship parameter matrix M. oh and the second global relationship parameter matrix M ot The parameter vectors of K relations are selected respectively, two sub-parameter matrices are generated, the fusion vector is passed into the two sub-parameter matrices, and the probability that each position in the target text can be used as the starting position of the tail entity and the probability of the ending position of the tail entity are calculated by the Sigmod function, thereby constructing a complete triplet, where K is a positive integer.

[0011] In one possible implementation, the word embedding processing module is specifically used to: preprocess the initial text, and input it into the Bert pre-training model for word embedding processing to obtain the text vector of the target text.

[0012] In one possible implementation, the word embedding processing module is specifically used to: preprocess the initial text and input it into the Bert pre-training model for word embedding processing to obtain a word vector for each word in the initial text; combine the word vectors of n words obtained successively to obtain a text vector of the target text with a length of n, where n is a positive integer greater than 1.

[0013] In one possible implementation, the head entity tagging module is specifically used to: input the text vector into the head entity starting position prediction fully connected layer and Sigmod function, and calculate the probability that each position in the target text can be used as the starting position of the head entity; input the text vector into the head entity ending position prediction fully connected layer and Sigmod function in sequence, and calculate the probability that each position in the target text can be used as the ending position of the head entity.

[0014] In one possible implementation, the head entity tagging module is specifically used to combine the words at positions above a first threshold in the probability that each position in the target text can be the starting position of the head entity to the words at positions above a second threshold in the probability that each position in the target text can be the ending position of the head entity to obtain the head entity.

[0015] In the third aspect, a device for joint extraction of entity relationships is provided, comprising: a processor, and a memory communicatively connected to the processor, wherein the memory stores program instructions executable by the processor, and the processor calls the program instructions to execute the entity relationship joint extraction method of the first aspect and any possible implementation method of the first aspect.

[0016] In a fourth aspect, a computer-readable storage medium is provided, which includes a stored program, wherein when the program is run, the device where the computer-readable storage medium is located is controlled to execute the entity relationship joint extraction method such as the first aspect and any possible implementation method of the first aspect.

[0017] Based on the technical solution of the embodiment of the present application, by predicting the probabilities of various relationships and extracting some relationships to predict the tail entity, rather than using all relationships to predict the tail entity, the extraction of unnecessary relationships is reduced, reducing the amount of computation. In addition, using relationships to perceive entities increases the degree of interaction between entity information and relationship information. Finally, the entity-relationship joint extraction method of the embodiment of the present application obtains triples with high precision, recall rate, and F1 value. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the following is a brief introduction to the drawings required for use in the embodiments of the present application. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on the drawings without creative work.

[0019] Figure 1 A schematic block diagram of the entity relationship joint extraction method according to an embodiment of the present application is shown.

[0020] Figure 2 A schematic flowchart of the entity relationship joint extraction method according to an embodiment of the present application is shown.

[0021] Figure 3 A schematic block diagram of an entity relationship joint extraction device according to an embodiment of the present application is shown. DETAILED DESCRIPTION

[0022] To make the purpose, technical solutions, and advantages of the embodiments of this application more clear, the technical solutions in the embodiments of this application will be clearly described below in conjunction with the drawings in the embodiments of this application. Obviously, the described embodiments are part of the embodiments of this application, not all of the embodiments. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of this application.

[0023] With the development of information technology, massive amounts of unstructured data are accumulating across various fields. Extracting useful information from this data has become a crucial research topic. Entity relationship extraction, a key task in information extraction, aims to simultaneously identify entities and their relationships within text. This technology plays a role in tasks such as knowledge graph construction, information extraction and retrieval, question-answering systems, public opinion analysis, and natural language understanding. It helps machines understand text content and infer connections between entities, thereby improving text comprehension and semantic reasoning capabilities, bringing deeper intelligent processing and applications to the field of artificial intelligence.

[0024] Traditional entity relationship extraction methods typically treat entity recognition and relationship extraction as two separate steps. The goal of entity recognition is to identify words or phrases with specific semantics from text, such as names of people, places, and organizations. These methods primarily include rule-based, statistical learning, and deep learning. Rule-based methods rely on predefined entity dictionaries or regular expressions for matching; statistical learning methods, such as hidden Markov models (HMMs) and conditional random fields (CRFs), use training data to build models to predict the labels of each word in the text; deep learning methods, on the other hand, utilize advanced techniques such as recurrent neural networks (RNNs), long-short-term memory (LSTMs), and pre-trained models such as BERT to capture contextual information in the text. The goal of relationship extraction is to identify relationships between identified entities. This is typically accomplished through classification and dependency analysis. Classification methods utilize feature engineering to extract features from text and use classification algorithms to determine the type of relationship, while dependency analysis analyzes text based on syntactic structure to extract relationships between entities. Deep learning methods, such as bidirectional LSTM and convolutional neural networks (CNNs), are also widely used in relation extraction. These methods can better capture the relationship information between entities. Although step-by-step processing simplifies model design and allows each task to be optimized independently, this approach also has certain limitations, such as information loss, error accumulation, and lack of global optimization. Therefore, future research may develop towards joint entity and relation extraction, by simultaneously performing these two tasks in the same model to improve overall performance and reduce the impact of error propagation.

[0025] The joint entity-relationship extraction method aims to perform entity recognition and relationship extraction simultaneously. The core idea of ​​this method is to combine the two tasks within a unified framework, fully leveraging the complementary information between entities and relationships to improve the accuracy and efficiency of information extraction. The advantages of joint extraction are obvious. First, because entity recognition and relationship extraction are performed in the same model, the model is able to share information, reduce redundant computations, and thus improve efficiency. Second, the problem of relationship extraction performance degradation caused by entity recognition errors is reduced, and overall accuracy is improved. In addition, the joint extraction method can fully utilize contextual information, enhance the model's understanding of complex relationships, and optimize the parameters of the entire model through end-to-end learning.

[0026] Research has shown that joint extraction methods based on pre-trained language models, such as Bidirectional Encoder Representations from Transformers (BERT) and the Robustly Optimized BERT Pretraining Approach (RoBERTa), have made significant progress. These models, pre-trained on large-scale corpora, possess powerful language understanding capabilities, demonstrating excellent performance in entity and relation recognition tasks. Furthermore, some studies have proposed combining graph neural networks (GNNs) with Transformers to further enhance the ability to capture complex structured information.

[0027] However, current joint entity-relationship extraction still faces numerous challenges. For example, the accuracy and robustness of the model still need to be improved when handling complex scenarios with multiple entities and multiple relationships. Furthermore, designing efficient joint entity-relationship extraction methods for low-resource languages ​​and domain-specific texts also requires urgent research and solutions.

[0028] Therefore, the embodiments of the present application provide a new entity relationship joint extraction method and device, which can improve the accuracy and robustness of the model.

[0029] Figure 1 The schematic block diagram of an entity relationship joint extraction method 100 provided in an embodiment of the present application is shown. Optionally, the entity relationship joint extraction method 100 can be executed by a computer processor, for example, by a central processing unit (CPU). For another example, it can also be executed by a CPU and a graphics processing unit (GPU). The embodiment of the present application does not limit the execution subject. Figure 1 As shown, the entity relationship joint extraction method 100 includes part or all of the following contents.

[0030] S110 , performing word embedding processing on the initial text to be subjected to triple extraction, to obtain a text vector of the target text in the initial text.

[0031] S120, input the text vector into a fully connected neural network and a Sigmod function, calculate the probability that each position in the target text can be used as the starting position of the head entity and the probability that the head entity can be used as the ending position of the head entity, and determine the head entity based on the probability that each position in the target text can be used as the starting position of the head entity and the probability that the head entity can be used as the ending position of the head entity.

[0032] S130, fusing the text vector and the head entity vector to generate a fusion vector, inputting the fusion vector into a convolutional neural network to obtain a convolution feature vector of the target text, and inputting the convolution feature vector into a Softmax function to calculate the existence probability of each relationship in the target text, where the head entity vector is a vector corresponding to the head entity obtained from the text vector.

[0033] S140, select the K relationships with the highest probability of existence, and oh and the second global relationship parameter matrix M ot The parameter vectors of the K relationships are selected respectively to generate two sub-parameter matrices, the fusion vector is passed into the two sub-parameter matrices, and the probability that each position in the target text can be used as the starting position of the tail entity and the probability of the ending position of the tail entity are calculated by the Sigmod function, thereby constructing a complete triple, where K is a positive integer.

[0034] First of all, it should be noted that the embodiments of the present application are mainly based on relation triple extraction. The so-called triple is a data structure, which consists of three parts, usually expressed as (entity 1, relationship, entity 2), where entity 1 can also be called the head entity, and entity 2 can also be called the tail entity. It is widely used in many fields such as knowledge representation and information processing to describe the relationship between two entities concisely and clearly. Relation triple extraction is a key task in the field of information extraction. It mainly extracts triple information presented in the form of (entity 1, relationship, entity 2) from unstructured data such as text.

[0035] In S110, first, the initial text to be subjected to triple extraction needs to be obtained. For example, it can be obtained from network resources, collected documents, and manual input. The obtained initial text can then be subjected to word embedding processing. For example, the obtained initial text can first be subjected to vocabulary statistics to determine all the words that appear and to construct a vocabulary list. Secondly, a word embedding model can be selected and trained, such as the Word2Vec or GloVe model trained on large-scale corpus (such as Wikipedia, news articles, etc.). These models have learned rich semantic information, directly download the corresponding word vector file, and then map the words in the initial text to these pre-trained vectors. After obtaining the word vector, for each word in the initial text, extract the vector of part or all of the text to obtain the text vector of the target text.

[0036] In triple extraction, the vector processed by word embedding can better help the model understand the semantics of the text. At the same time, word embedding can effectively deal with problems such as polysemy, reflect different semantics in the vector space through contextual information, and improve the accuracy of triple extraction.

[0037] Furthermore, S110 may also include: preprocessing the initial text and inputting it into a Bert pre-training model for word embedding processing to obtain a text vector of the target text.

[0038] Typically, the initial text often contains various noises and complex formats. For example, the text may contain HTML tags, special characters, spelling errors, or non-standard abbreviations. Preprocessing can clean up these irrelevant information, making the data input into the model cleaner and more standardized, thereby improving the performance and accuracy of the model. The Bert pre-training model is unsupervised pre-trained on a large-scale text corpus. The pre-training process enables the model to learn rich language knowledge, including the semantic representation of vocabulary, grammatical structure, common language patterns, etc. For example, it knows how words of different parts of speech are usually combined, the relationship between different components in a sentence, etc. When used for the downstream task of triple extraction, this pre-trained knowledge can be effectively transferred, reducing the dependence on large-scale labeled data, and can achieve good performance after fine-tuning on a relatively small task-related dataset.

[0039] It should be understood that when the initial text itself is relatively clean and standardized, it can also be directly input into the BERT pre-training model. In other words, preprocessing of the initial text is not a necessary step.

[0040] It should also be understood that in addition to the BERT pre-trained model, other models can also be used to perform word embedding processing on the initial text. For example, the Generative Pretrained Transformer (GPT) model, the Enhanced Representation through kNowledge IntEgration (ERNIE) model, etc. The embodiments of this application do not limit the model used for word embedding processing.

[0041] As mentioned above, the target text can be part or all of the initial text. When the target text is part of the initial text, the text vector of the target text can be obtained by preprocessing the initial text and inputting it into the BERT pre-trained model for word embedding to obtain a word vector for each word in the initial text. The word vectors of n consecutive words are then combined to obtain a text vector of the target text of length n, where n is a positive integer greater than 1.

[0042] For example, the initial text is "Xiao Ming likes to read all kinds of books. He often goes to the library to borrow books. There are many kinds of books in the library, which makes him very happy." After preprocessing the initial text and inputting it into the Bert pre-training model, the vector corresponding to each word in the initial text can be obtained. The vectors corresponding to 10 consecutive words, that is, the vectors corresponding to each word in the first sentence of the initial text "Xiao Ming likes to read all kinds of books", are combined to obtain the text vector of the target text "Xiao Ming likes to read all kinds of books".

[0043] In S120, after obtaining the text vector of the target text, the text vector can be further input into a fully connected neural network and a Sigmod function to calculate the probability that each position in the target text can be used as the starting position of the head entity and the probability that each position in the target text can be used as the ending position of the head entity.

[0044] Specifically, S120 may further include: inputting the text vector into the head entity starting position prediction fully connected layer and Sigmod function, and calculating the probability that each position in the target text can be used as the head entity starting position; inputting the text vector into the head entity ending position prediction fully connected layer and Sigmod function in sequence, and calculating the probability that each position in the target text can be used as the head entity ending position.

[0045] That is to say, in the neural network architecture for relation triple extraction, two fully connected layers are usually required to predict the probability related to the head entity position. Each neuron in the fully connected layer is connected to all neurons in the input layer. It performs a linear transformation through the weight matrix and bias vector. Assuming that the text vector of the target text is x, the weight matrix is ​​W, and the bias vector is b, then the output after the linear transformation is y = Wx + b. Afterwards, in order to enhance the nonlinear expression ability, an activation function such as the ReLU function can be added in the middle, that is, z = ReLu(y). After a series of calculations, the final output is a vector with the same dimension as the length of the text vector of the target text. Each element of this vector represents the probability estimate of the corresponding position in the text as the target position of the head entity (for example, the starting position of the head entity or the ending position of the head entity). The probability value is usually obtained by passing the output vector after passing through the fully connected layer through the Sigmoid function.

[0046] After obtaining the probability that each position in the target text can be the starting position of the head entity and the probability that each position in the target text can be the ending position of the head entity, the starting position and the ending position of the head entity can be determined based on a preset threshold, and then the head entity can be determined.

[0047] Specifically, determining the head entity based on the probability that each position in the target text can be the starting position of the head entity and the probability that each position in the target text can be the ending position of the head entity may include: combining the characters from the positions in the probability that each position in the target text can be the starting position of the head entity that are higher than the first threshold to the characters from the positions in the probability that each position in the target text can be the ending position of the head entity that are higher than the second threshold to obtain the head entity. Wherein, the first threshold and the second threshold may be the same or different, and the embodiments of the present application do not limit this.

[0048] For example, for the target text "Beijing is the capital of China", the probabilities of "North", "Beijing", "is", "China", "of", "the", "capital", and "city" as the starting positions of the head entity are respectively (0.8, 0.5, 0.4, 0.3, 0.25, 0.20, 0.15, 0.1). Assuming the first threshold is 0.6, then it can be determined that the starting position of the head entity is the first position of the target text, and the corresponding character is "North". Similarly, the probabilities of these 8 characters as the ending positions of the head entity can be obtained as (0.5, 0.7, 0.25, 0.3, 0.35, 0.4, 0.5, 0.1). Similarly, assuming the second threshold is also 0.6, then it can be determined that the ending position of the head entity is the second position of the target text, and the corresponding character is "Beijing". Then the head entity in the target text can be determined as "Beijing".

[0049] It should be understood that the first threshold and the second threshold are empirical values and can be preset in advance. The embodiments of the present application do not limit the specific values of the first threshold and the second threshold.

[0050] After obtaining the head entity in the target text, the head entity vector can be obtained from the text vector of the target text. It should be noted that the text vector of the target text is actually a combination of the vectors corresponding to each character in the target text. Then after obtaining the head entity, the head entity vector can be directly extracted from the text vector of the target text according to the positional relationship of each character in the head entity.

[0051] In S130, after obtaining the head entity vector, the text vector of the target text and the head entity vector can be fused to generate a fusion vector. For example, the fusion can be performed by splicing. This method is simple and easy to implement and can completely retain the information of the text vector and the head entity vector. For another example, the fusion can be performed by addition. This method can fuse the information of the two vectors to a certain extent, reflecting the mutual correlation between the target text and the head entity. For another example, the fusion can be performed by multiplication or dot product. The multiplication method can emphasize the common part of the text vector and the head entity vector, which has a certain effect on highlighting the correlation between the two. The dot product method is mainly used to measure the similarity of the two vectors, and the obtained scalar value can be used as a fused feature to represent the degree of correlation between the two. In other embodiments, the fusion can also be performed based on a neural network. This method can automatically learn the most effective fusion method between the text vector and the head entity vector, and can adapt to different target texts and head entities through training.

[0052] Furthermore, the resulting fused vector can be input into a convolutional neural network to obtain a convolutional feature vector for the target text. Although the fused vector already contains information from the text vector and the head entity vector, this information is raw or preliminarily fused. Convolutional neural networks can automatically learn semantic features in text. For example, for the task of extracting relation triples, the convolutional neural network can automatically learn semantic patterns representing entity relationships from the fused vector. It slides the convolution kernel over the fused vector to detect local semantic information at different locations. This automatic learning capability avoids complex manual feature engineering and enables the model to adaptively mine the latent semantics in the text. In addition, because convolutional neural networks have multiple convolutional and pooling layers, they can perform multi-level semantic mining. In the initial convolutional layers, simple word combination features or features related to local grammatical structure may be extracted. As the layers deepen, subsequent convolutional layers can construct more complex semantic features based on previously extracted features, such as entity information within a sentence or complex semantic associations between entities.

[0053] After inputting the fused vector into the convolutional neural network to obtain the convolutional feature vector of the target text, the convolutional feature vector can be further input into the Softmax function to calculate the existence probability of each relationship in the target text. Similar to predicting the probability of a head entity starting position or ending position in the target text, if the dimension of the convolutional feature vector does not match the number of relationship categories, it is necessary to convert the dimension of the convolutional feature vector to the number of relationship categories through a fully connected layer before inputting it into the Softmax function. The output after the fully connected layer is then input into the Softmax function to obtain the existence probability of each relationship in the target text.

[0054] For example, taking the target text "Beijing is the capital of China" as an example, the relationship categories include "capital" and "provincial capital". After the convolutional feature vector of the target text is input into the fully connected layer and the Softmax function in turn, the probabilities of the two relationships are obtained (0.8, 0.5) respectively.

[0055] After obtaining the existence probability of each relationship in the target text, at least one relationship can be further screened out from all relationships. In one embodiment, the existence probabilities of all relationships can be arranged in order from high to low based on a preset K value, and the K relationships with the largest existence probabilities are taken. Assuming that in the above-mentioned example of "Beijing is the capital of China", K is 1, then the relationship "capital" with the highest probability can be selected from all the relationships "capital" and "provincial capital". In another embodiment, the existence probability of each relationship can be directly compared with the preset threshold based on a preset threshold, and the relationships greater than the preset threshold are retained. Assuming that in the above-mentioned example of "Beijing is the capital of China", the preset threshold is 0.4, then according to the probabilities (0.8, 0.5) corresponding to the two relationships "capital" and "provincial capital", it can be judged that both "capital" and "provincial capital" meet the requirements, and both relationships need to be subsequently processed.

[0056] After selecting K relations that meet the requirements, we can further obtain the global relation parameter matrix M oh and M ot Select the parameter vectors of the K relationships and generate two sub-parameter matrices. The global relationship parameter matrix M oh and M ot It is an important parameter structure in processing relation triples extraction. oh Mainly used to represent the relationship between the head entity, M ot Mainly used for the relationship representation related to the tail entity. Each of their rows (or columns, depending on the specific definition and application) can be regarded as a vector representation of a relationship. These vectors contain the features learned by the model about a specific relationship in the entire text semantic space. In other words, the global relationship parameter matrix M oh and M ot It is a combination of the parameter vectors of all relations, from which the parameter vectors of the K relations that meet the requirements can be extracted to generate the corresponding two sub-parameter matrices.

[0057] Furthermore, after obtaining the two sub-parameter matrices corresponding to K relationships, the fusion vector can be passed into the two sub-parameter matrices respectively, and the Sigmod function can be used to obtain the probability that each position in the target text can be used as the starting position of the tail entity and the probability that each position in the target text can be used as the ending position of the tail entity, thereby constructing a complete triple.

[0058] It should be understood that the calculation method of the probability that each position in the target text can be the starting position and the ending position of the tail entity is similar to the probability that each position in the target text can be the starting position and the ending position of the head entity. For the sake of brevity, it will not be repeated here.

[0059] Furthermore, after obtaining the probability that each position in the target text can be the starting position and the ending position of the tail entity, the starting position and the ending position of the tail entity can still be determined based on a preset threshold, thereby determining the tail entity. After obtaining the head entity, the relationship, and the tail entity in sequence, a complete triple can be formed, thus completing the extraction of the relationship triple.

[0060] The following will be combined Figure 2 A detailed embodiment of the entity relationship joint extraction method of the embodiment of the present application is described in detail.

[0061] like Figure 2 As shown, assuming that the initial text in the database is [word 1, word 2, word 3, ..., word m], the initial text is input into the Bert pre-training model, and the word vector of each word is obtained sequentially. Let x i ∈R d is the d-dimensional word vector x corresponding to the i-th word in the initial text i , combining the word vectors of consecutive n words can get a text vector of length n as [x1,x2,......,x n ], use x 1:n Represents. The text vector x 1:n The vectors are respectively passed into the fully connected layer for predicting the starting position of the head entity and the fully connected layer for predicting the ending position of the head entity, and linear transformation is performed to obtain the vectors and Then and The Sigmoid function is used to calculate the probability of each of the n positions in the target text being the start and end positions of the head entity. If the probability value of a position is not less than the preset threshold, it is set to 1, indicating that the position is the start or end position of the head entity. Other positions are set to 0.

[0062] Among them, the text vector x 1:n for: is the concatenation operator.

[0063] Furthermore, the head entity position is obtained according to the training set, and the text vector x 1:n Extract the head entity vector from . Specifically, the vector of the head entity position is the same as x 1:nThe same as in . Then add the text vector and the head entity vector to get the fusion vector v~; input the fusion vector v~ into the convolutional neural network, where the convolutional neural network includes the filter w∈R hd , h is the window size of filter w, filter w uses h words as a window to generate convolution feature vectors, and fills the window part beyond the target text boundary with 0 vector, where x i:i+h-1 After passing through the filter w, the convolution feature vector c is generated i ; Filter w scans all characters {x 1:h , x 2:h+1 ,...,x x-h+1:n} and generate the corresponding convolution feature map c; perform the maximum pooling operation on the convolution feature map c and use the maximum value max{c} as the convolution feature vector of the target text Then the convolution feature vector The probabilities of different relationship categories are calculated through the Softmax function.

[0064] Among them, c i =f(w·x i:i+h-1 ),c={c1,c2,...,c n-h+1}, f is a nonlinear function; c∈R n-h+1 .

[0065] Furthermore, a predetermined value of K relations with the highest occurrence probability in the target text are selected to form a relation set S r , from the global relationship parameter matrix M oh ∈R p×d and M ot ∈R p×d , select S r The labels of the relations appearing in the matrix form the sub-relation parameter matrix m oh ∈R k×d and m ot ∈R k×d ; Fusion vector Pass in two sub-relationship parameter matrices and Sigmod function respectively to get the matrix M poh ∈R n×d and M pot ∈R n×d , for the matrix M poh ∈R n×d , each column of which corresponds to a relation r i ∈S r , which means for the relationship r i The probability of extracting the starting position of the tail entity; for the matrix M pot ∈R n×d , each column of which corresponds to a relation r i ∈S r, which means for the relationship r i The probability of extracting the end position of the tail entity; for each target text, this method can be used to extract the head entity e through the above method h , for each relation r i ∈S r , extract e h The corresponding tail entity e t .

[0066] The entity relationship joint extraction method provided in the embodiment of the present application predicts the probability of various relationships and extracts some relationships to predict the tail entity, rather than using all relationships to predict the tail entity. This reduces the extraction of unnecessary relationships and reduces the amount of computation. In addition, using relationships to perceive entities increases the degree of interaction between entity information and relationship information. Finally, the entity relationship joint extraction method of the embodiment of the present application has high precision, recall rate, and F1 value for the triples obtained.

[0067] Table 1 shows a comparison of the precision, recall, and F1 values ​​obtained by some traditional entity-relationship joint extraction models and the entity-relationship joint extraction model provided by the embodiment of the present application. From Table 1, it can be seen that the precision, recall, and F1 value of the entity-relationship joint extraction model provided by the embodiment of the present application are all higher.

[0068] Table 1

[0069] Model Accuracy Recall F1 Novel-Tagging 75.8 36.7 49.4 GraphRel 52.2 25.8 34.5 CMAN 77.3 68.1 72.4 TPLinker 78.4 70.5 74.2 This application model 79.3 68.3 73.8

[0070] The embodiment of the present application also provides a device for extracting entity relationships. Figure 3 As shown, the entity relationship joint extraction device 200 includes part or all of the following contents.

[0071] The word embedding processing module 210 is used to perform word embedding processing on the initial text to be subjected to triple extraction, and obtain a text vector of the target text in the initial text.

[0072] The head entity tagging module 220 is used to calculate the probability that each position in the target text can be used as the starting position of the head entity and the probability that each position in the target text can be used as the ending position of the head entity through the text vector using a fully connected neural network and a Sigmund function, and determine the head entity based on the probability that each position in the target text can be used as the starting position of the head entity and the probability that each position in the target text can be used as the ending position of the head entity.

[0073] The relationship identification module 230 is used to fuse the text vector and the head entity vector to generate a fusion vector, input the fusion vector into a convolutional neural network to obtain a convolution feature vector of the target text, and calculate the convolution feature vector through a Softmax function to obtain the existence probability of each relationship in the target text. The head entity vector is a vector corresponding to the head entity obtained from the text vector.

[0074] The tail entity tagging module 240 is used to select the K relationships with the highest probability of existence and oh and M ot The parameter vectors of the K relationships are selected respectively to generate two sub-parameter matrices, the fusion vector is passed into the two sub-parameter matrices, and the probability that each position in the target text can be used as the starting position of the tail entity and the probability that each position in the target text can be used as the ending position of the tail entity are calculated by the Sigmod function, thereby constructing a complete triple, where K is a positive integer.

[0075] Optionally, in an embodiment of the present application, the word embedding processing module 210 is specifically used to: preprocess the initial text, and input it into the Bert pre-training model for word embedding processing to obtain the text vector of the target text.

[0076] Optionally, in an embodiment of the present application, the word embedding processing module 210 is specifically used to: preprocess the initial text and input it into the Bert pre-training model for word embedding processing to obtain a word vector for each word in the initial text; combine the word vectors of n words obtained successively to obtain a text vector of the target text with a length of n, where n is a positive integer greater than 1.

[0077] Optionally, in an embodiment of the present application, the head entity labeling module 220 is specifically used to: input the text vector into the head entity starting position prediction fully connected layer and Sigmod function, and calculate the probability that each position in the target text can be used as the starting position of the head entity; input the text vector into the head entity ending position prediction fully connected layer and Sigmod function in sequence, and calculate the probability that each position in the target text can be used as the ending position of the head entity.

[0078] Optionally, in an embodiment of the present application, the head entity marking module 220 is specifically used to: combine the words at positions above a first threshold in the probability that each position in the target text can be the starting position of the head entity to the words at positions above a second threshold in the probability that each position in the target text can be the ending position of the head entity to obtain the head entity.

[0079] Based on the same concept, embodiments of the present application also provide another entity relationship joint extraction device. The entity relationship joint extraction device includes a memory for storing computer program instructions and a processor for executing the program instructions. When the program instructions are executed by the processor, the entity relationship joint extraction device performs the various method embodiments described above.

[0080] It should be noted that the details of the device side embodiment can be found in the method side embodiment, and for the sake of brevity, they will not be repeated here.

[0081] Based on the same idea, an embodiment of the present application also provides a computer-readable storage medium, which includes a stored program, wherein when the program is running, the device where the computer-readable storage medium is located is controlled to execute the above-mentioned various method embodiments.

[0082] Although the present application has been described with reference to preferred embodiments, various modifications may be made thereto and components may be substituted with equivalents without departing from the scope of the present application. In particular, the various technical features described in the various embodiments may be combined in any manner as long as there are no structural conflicts. The present application is not limited to the specific embodiments disclosed herein, but encompasses all technical solutions within the scope of the claims.

Claims

1. A method for joint extraction of entity relationships, characterized in that: include: Performing word embedding processing on the initial text to be subjected to triple extraction to obtain a text vector of the target text in the initial text; Inputting the text vector into a fully connected neural network and a Sigmund function, calculating the probability that each position in the target text can be the starting position of the head entity and the probability that the head entity ends, and determining the head entity based on the probability that each position in the target text can be the starting position of the head entity and the probability that the head entity ends; Fusing the text vector and the head entity vector to generate a fused vector, inputting the fused vector into a convolutional neural network to obtain a convolution feature vector of the target text, and inputting the convolution feature vector into a Softmax function to calculate the existence probability of each relationship in the target text, where the head entity vector is a vector corresponding to the head entity obtained from the text vector; Select the K relationships with the highest probability of existence and select them from the global relationship parameter matrix M oh and M ot The parameter vectors of the K relationships are selected respectively to generate two sub-parameter matrices, the fusion vector is passed into the two sub-parameter matrices, and the probability that each position in the target text can be used as the starting position of the tail entity and the probability of the ending position of the tail entity are calculated by the Sigmod function, thereby constructing a complete triple, where K is a positive integer.

2. The entity relationship joint extraction method according to claim 1, characterized in that: The performing word embedding processing on the initial text to be subjected to triple extraction to obtain a text vector of the target text in the initial text includes: The initial text is preprocessed and input into the Bert pre-training model for word embedding processing to obtain the text vector of the target text.

3. The entity relationship joint extraction method according to claim 2, characterized in that: The initial text is preprocessed and input into the Bert pre-training model for word embedding processing to obtain the text vector of the target text, including: Preprocessing the initial text and inputting it into the Bert pre-training model for word embedding processing to obtain a word vector for each word in the initial text; The word vectors of n words obtained successively are combined to obtain a text vector of the target text having a length of n, where n is a positive integer greater than 1.

4. The entity relationship joint extraction method according to claim 1, characterized in that: Inputting the text vector into a fully connected neural network and a Sigmund function, and calculating the probability that each position in the target text can be the starting position of the head entity and the probability that the head entity can be the ending position, includes: Input the text vector into the head entity starting position prediction fully connected layer and Sigmod function, and calculate the probability that each position in the target text can be the head entity starting position; The text vector is sequentially input into the head entity end position prediction fully connected layer and the Sigmod function to calculate the probability that each position in the target text can be the head entity end position.

5. The entity relationship joint extraction method according to claim 1, characterized in that: The determining of the head entity based on the probability that each position in the target text can serve as the starting position of the head entity and the probability that the head entity can serve as the ending position of the head entity includes: The head entity is obtained by combining the characters at positions above a first threshold value in the probability that each position in the target text can be the starting position of the head entity to the characters at positions above a second threshold value in the probability that each position in the target text can be the ending position of the head entity.

6. An entity relationship joint extraction device, characterized in that: include: A word embedding processing module is used to perform word embedding processing on the initial text to be subjected to triple extraction, so as to obtain a text vector of the target text in the initial text; a head entity tagging module, configured to input the text vector into a fully connected neural network and a Sigmund function, calculate the probability that each position in the target text can be the starting position of the head entity and the probability that the head entity ends, and determine the head entity based on the probability that each position in the target text can be the starting position of the head entity and the probability that the head entity ends; a relationship recognition module, configured to fuse the text vector and the head entity vector to generate a fused vector, input the fused vector into a convolutional neural network to obtain a convolution feature vector of the target text, and input the convolution feature vector into a Softmax function to calculate the existence probability of each relationship in the target text, where the head entity vector is a vector corresponding to the head entity obtained from the text vector; The tail entity tagging module is used to select the K relationships with the highest probability of existence and oh and M ot The parameter vectors of the K relationships are selected respectively to generate two sub-parameter matrices, the fusion vector is passed into the two sub-parameter matrices, and the probability that each position in the target text can be used as the starting position of the tail entity and the probability of the ending position of the tail entity are calculated by the Sigmod function, thereby constructing a complete triple, where K is a positive integer.

7. The entity relationship joint extraction device according to claim 6, characterized in that: The word embedding processing module is specifically used for: The initial text is preprocessed and input into the Bert pre-training model for word embedding processing to obtain the text vector of the target text.

8. The entity relationship joint extraction device according to claim 7, characterized in that: The word embedding processing module is specifically used for: Preprocessing the initial text and inputting it into the Bert pre-training model for word embedding processing to obtain a word vector for each word in the initial text; The word vectors of n words obtained successively are combined to obtain a text vector of the target text having a length of n, where n is a positive integer greater than 1.

9. The entity relationship joint extraction device according to claim 6, characterized in that: The header entity marking module is specifically used to: Input the text vector into the head entity starting position prediction fully connected layer and Sigmod function, and calculate the probability that each position in the target text can be the head entity starting position; The text vector is sequentially input into the head entity end position prediction fully connected layer and the Sigmod function to calculate the probability that each position in the target text can be the head entity end position.

10. The entity relationship joint extraction device according to claim 6, characterized in that: The header entity marking module is specifically used to: The head entity is obtained by combining the characters at positions above a first threshold value in the probability that each position in the target text can be the starting position of the head entity to the characters at positions above a second threshold value in the probability that each position in the target text can be the ending position of the head entity.

Images