Entity relation joint extraction method and device

Abstract

The embodiment of the invention provides an entity relation joint extraction method and device. The method comprises the following steps: firstly, identifying all possible subjects in a text; then setting a threshold value K, and extracting K relationships which are most likely to appear in the text; and then applying a relation specific marker to each subject, identifying objects corresponding to the K relations at the same time, and finally constructing a complete triple. According to the method and the device provided by the embodiment of the invention, the accuracy and the robustness of the model can be improved.

Description

Technical Field

[0001] The embodiments of the present application relate to the field of natural language processing, and in particular, to a method and device for joint extraction of entity relationships. Background Art

[0002] With the development of information technology, massive amounts of unstructured data are constantly accumulating in various fields. How to extract useful information from this data has become an important research topic. Entity-relationship joint extraction, as a key task in the field of information extraction, aims to simultaneously identify entities in text and their relationships. This task plays an important role in applications such as knowledge graph construction, information retrieval, and natural language understanding.

[0003] However, the current entity-relationship joint extraction still faces many challenges. For example, when dealing with complex scenarios with multiple entities and multiple relationships, the accuracy and robustness of the model still need to be improved. In addition, for low-resource languages ​​and domain-specific texts, how to design efficient entity-relationship joint extraction methods also needs to be studied and solved. Summary of the invention

[0004] In view of this, an embodiment of the present application provides a method and device for joint extraction of entity relationships, which can improve the accuracy and robustness of the model.

[0005] In a first aspect, a method for joint extraction of entity relations is provided, characterized in that it includes: performing word embedding processing on an initial text to be subjected to triple extraction to obtain a text vector of a target text in the initial text; inputting the text vector into a fully connected neural network and a Sigmod function, calculating the probability that each position in the target text can be used as the starting position of a head entity and the probability that the head entity can end, and determining the head entity based on the probability that each position in the target text can be used as the starting position of the head entity and the probability that the head entity can end; fusing the text vector and the head entity vector to generate a fused vector, inputting the fused vector into a convolutional neural network to obtain a convolutional feature vector of the target text, and inputting the convolutional feature vector into a Softmax function, calculating the existence probability of each relationship in the target text, the head entity vector being a vector corresponding to the head entity obtained from the text vector; selecting K relationships with the highest existence probability, and selecting the K relationships from the global relationship parameter matrix M. oh and the second global relationship parameter matrix M ot The parameter vectors of K relations are selected respectively, two sub-parameter matrices are generated, the fusion vector is passed into the two sub-parameter matrices, and the probability that each position in the target text can be used as the starting position of the tail entity and the probability of the ending position of the tail entity are calculated by the Sigmod function, thereby constructing a complete triple, K is a positive integer.

[0006] In one possible implementation, the initial text to be subjected to triple extraction is subjected to word embedding processing to obtain a text vector of a target text in the initial text, including: preprocessing the initial text and inputting it into a Bert pre-trained model for word embedding processing to obtain a text vector of the target text.

[0007] In a possible implementation, the initial text is preprocessed and input into the Bert pre-trained model for word embedding processing to obtain a text vector of the target text, including: preprocessing the initial text and inputting into the Bert pre-trained model for word embedding processing to obtain a word vector for each word in the initial text; combining the word vectors of n words obtained successively to obtain a text vector of the target text with a length of n, where n is a positive integer greater than 1.

[0008] In one possible implementation, a text vector is input into a fully connected neural network and a Sigmod function, and the probability that each position in the target text can be used as the starting position of a head entity and the probability that the head entity can be used as the ending position are calculated, including: inputting the text vector into a fully connected layer for predicting the starting position of a head entity and a Sigmod function, and calculating the probability that each position in the target text can be used as the starting position of the head entity; inputting the text vector into a fully connected layer for predicting the ending position of a head entity and a Sigmod function in sequence, and calculating the probability that each position in the target text can be used as the ending position of the head entity.

[0009] In one possible implementation, a head entity is determined based on the probability that each position in the target text can be the starting position of the head entity and the probability that the head entity ends, including: combining characters at positions above a first threshold in the probability that each position in the target text can be the starting position of the head entity to characters at positions above a second threshold in the probability that each position in the target text can be the ending position of the head entity, to obtain the head entity.

[0010] In a second aspect, a device for joint extraction of entity relations is provided, including: a word embedding processing module, used to perform word embedding processing on an initial text to be subjected to triple extraction, and obtain a text vector of a target text in the initial text; a head entity tagging module, used to input the text vector into a fully connected neural network and a Sigmod function, calculate the probability that each position in the target text can be used as the starting position of the head entity and the probability that the head entity can be used as the ending position, and determine the head entity based on the probability that each position in the target text can be used as the starting position of the head entity and the probability that the head entity can be used as the ending position; a relationship recognition module, used to fuse the text vector and the head entity vector to generate a fused vector, input the fused vector into a convolutional neural network to obtain a convolutional feature vector of the target text, and input the convolutional feature vector into a Softmax function to calculate the existence probability of each relationship in the target text, wherein the head entity vector is a vector corresponding to the head entity obtained from the text vector; and a tail entity tagging module, used to select K relationships with the highest existence probability, and select the K relationships from the global relationship parameter matrix M. oh and the second global relationship parameter matrix M ot The parameter vectors of K relations are selected respectively, two sub-parameter matrices are generated, the fusion vector is passed into the two sub-parameter matrices, and the probability that each position in the target text can be used as the starting position of the tail entity and the probability of the ending position of the tail entity are calculated by the Sigmod function, thereby constructing a complete triple, K is a positive integer.

[0011] In a possible implementation, the word embedding processing module is specifically used to: pre-process the initial text, and input it into the Bert pre-training model for word embedding processing to obtain the text vector of the target text.

[0012] In one possible implementation, the word embedding processing module is specifically used to: preprocess the initial text and input it into the Bert pre-trained model for word embedding processing to obtain a word vector for each word in the initial text; combine the word vectors of n words obtained successively to obtain a text vector of the target text with a length of n, where n is a positive integer greater than 1.

[0013] In one possible implementation, the head entity tagging module is specifically used to: input the text vector into the head entity starting position prediction fully connected layer and the Sigmod function, and calculate the probability that each position in the target text can be used as the starting position of the head entity; input the text vector into the head entity ending position prediction fully connected layer and the Sigmod function in sequence, and calculate the probability that each position in the target text can be used as the ending position of the head entity.

[0014] In one possible implementation, the head entity marking module is specifically used to combine the characters at positions above a first threshold in the probability that each position in the target text can be the starting position of the head entity to the characters at positions above a second threshold in the probability that each position in the target text can be the ending position of the head entity to obtain a head entity.

[0015] In a third aspect, a device for joint extraction of entity relationships is provided, comprising: a processor, and a memory communicatively connected to the processor, wherein the memory stores program instructions executable by the processor, and the processor calls the program instructions to execute the method for joint extraction of entity relationships of the first aspect and any possible implementation method of the first aspect.

[0016] In a fourth aspect, a computer-readable storage medium is provided, which includes a stored program, wherein when the program is run, the device where the computer-readable storage medium is located is controlled to execute a method for joint extraction of entity relationships such as the first aspect and any possible implementation method of the first aspect.

[0017] Based on the technical solution of the embodiment of the present application, by predicting the probability of various relationships and extracting some relationships to predict the tail entity, rather than using all relationships to predict the tail entity, the extraction of unnecessary relationships is reduced and the amount of calculation is reduced. In addition, the relationship is used to perceive entities, and the degree of interaction between entity information and relationship information is increased. Finally, the accuracy, recall rate, and F1 value of the triples obtained by the entity relationship joint extraction method of the embodiment of the present application are relatively high. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required for use in the embodiments of the present application will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on the drawings without paying creative work.

[0019] Figure 1 A schematic block diagram of an entity relationship joint extraction method according to an embodiment of the present application is shown.

[0020] Figure 2 A schematic flowchart of a method for joint extraction of entity relationships according to an embodiment of the present application is shown.

[0021] Figure 3 A schematic block diagram of an entity relationship joint extraction device according to an embodiment of the present application is shown. DETAILED DESCRIPTION

[0022] In order to make the purpose, technical solution and advantages of the embodiments of the present application clearer, the technical solution in the embodiments of the present application will be clearly described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0023] With the development of information technology, massive amounts of unstructured data are constantly accumulating in various fields. How to extract useful information from this data has become an important research topic. Entity relationship extraction, as a key task in the field of information extraction, aims to simultaneously identify entities in text and the relationships between them. This technology plays a role in tasks such as knowledge graph construction, information extraction or retrieval, question-answering systems, public opinion analysis, and natural language understanding, helping machines understand text content and infer the connections between entities, thereby improving the ability of text understanding and semantic reasoning, and bringing deeper intelligent processing and applications to the field of artificial intelligence applications.

[0024] Traditional entity relationship extraction methods usually regard entity recognition and relationship extraction as two independent steps. The goal of entity recognition is to identify words or phrases with specific semantics from text, such as names of people, places, and organizations, and mainly include rule-based methods, statistical learning methods, and deep learning methods. Rule-based methods rely on predefined entity dictionaries or regular expressions for matching; statistical learning methods such as Hidden Markov Model (HMM) and Conditional Random Field (CRF) use training data to build models to predict the label of each word in the text; while deep learning methods use advanced technologies such as Recurrent Neural Network (RNN), Long-Short Term Memory (LSTM), and pre-trained models (such as BERT) to capture contextual information in the text. The goal of relationship extraction is to identify the relationship between identified entities, which is usually carried out through classification methods and dependency analysis. Classification methods use feature engineering to extract features from text and use classification algorithms to determine the type of relationship, while dependency analysis analyzes text through syntactic structure to extract the relationship between entities. Deep learning methods are also widely used in relation extraction, such as bidirectional LSTM and convolutional neural network (CNN), which can better capture the relational information between entities. Although step-by-step processing simplifies the model design and allows each task to be optimized independently, this method also has certain limitations, such as information loss, error accumulation, and lack of global optimization. Therefore, future research may be directed towards joint entity and relation extraction, by performing these two tasks simultaneously in the same model to improve the overall performance and reduce the impact of error propagation.

[0025] The entity-relationship joint extraction method aims to perform entity recognition and relationship extraction simultaneously. The core idea of ​​this method is to combine the two tasks in a unified framework and make full use of the complementary information between entities and relationships to improve the accuracy and efficiency of information extraction. The advantages of joint extraction are obvious. First, since entity recognition and relationship extraction are performed in the same model, the model can share information and reduce redundant calculations, thereby improving efficiency. Secondly, the problem of decreased performance of relationship extraction due to entity recognition errors is reduced, and the overall accuracy is improved. In addition, the joint extraction method can make full use of contextual information, enhance the model's understanding of complex relationships, and optimize the parameters of the entire model through end-to-end learning.

[0026] Research shows that joint extraction methods based on pre-trained language models, such as Bidirectional Encoder Representations from Transformers (BERT) and Robustly Optimized BERT Pretraining Approach (RoBERTa), have made significant progress. These models have strong language understanding capabilities through pre-training on large-scale corpora, thus showing excellent performance in entity and relationship recognition tasks. In addition, some studies have proposed methods to combine Graph Neural Network (GNN) with Transformer to further enhance the ability to capture complex structured information.

[0027] However, the current entity-relationship joint extraction still faces many challenges. For example, when dealing with complex scenarios with multiple entities and multiple relationships, the accuracy and robustness of the model still need to be improved. In addition, for low-resource languages ​​and domain-specific texts, how to design efficient entity-relationship joint extraction methods also needs to be studied and solved.

[0028] Therefore, the embodiments of the present application provide a new entity relationship joint extraction method and device, which can improve the accuracy and robustness of the model.

[0029] Figure 1 A schematic block diagram of an entity relationship joint extraction method 100 provided in an embodiment of the present application is shown. Optionally, the entity relationship joint extraction method 100 can be executed by a processor of a computer, for example, by a central processing unit (CPU). For another example, it can also be executed by a CPU and a graphics processing unit (GPU). The embodiment of the present application does not limit the execution subject. Figure 1 As shown, the entity relationship joint extraction method 100 includes part or all of the following contents.

[0030] S110, performing word embedding processing on the initial text to be subjected to triple extraction, and obtaining a text vector of the target text in the initial text.

[0031] S120, input the text vector into a fully connected neural network and a Sigmod function, calculate the probability that each position in the target text can be used as the starting position of the head entity and the probability that the head entity ends, and determine the head entity based on the probability that each position in the target text can be used as the starting position of the head entity and the probability that the head entity ends.

[0032] S130, fusing the text vector and the head entity vector to generate a fusion vector, inputting the fusion vector into a convolutional neural network to obtain a convolution feature vector of the target text, and inputting the convolution feature vector into a Softmax function to calculate the existence probability of each relationship in the target text, wherein the head entity vector is a vector corresponding to the head entity obtained from the text vector.

[0033] S140, selecting the K relationships with the highest existence probability and extracting them from the global relationship parameter matrix M oh and the second global relationship parameter matrix M ot The parameter vectors of the K relations are respectively selected to generate two sub-parameter matrices, the fusion vector is passed into the two sub-parameter matrices, and the probability that each position in the target text can be used as the starting position of the tail entity and the probability of the ending position of the tail entity are calculated by the Sigmod function, thereby constructing a complete triple, where K is a positive integer.

[0034] First of all, it should be noted that the embodiments of the present application are mainly based on relation triple extraction. The so-called triple is a data structure, which consists of three parts, usually expressed as (entity 1, relationship, entity 2), where entity 1 can also be called the head entity, and entity 2 can also be called the tail entity. It is widely used in many fields such as knowledge representation and information processing to describe the relationship between two entities concisely and clearly. Relation triple extraction is a key task in the field of information extraction. It mainly extracts triple information presented in the form of (entity 1, relationship, entity 2) from unstructured data such as text.

[0035] In S110, first, the initial text to be subjected to triple extraction needs to be obtained. For example, it can be obtained from network resources, collected documents, and manual input. Then, the obtained initial text can be subjected to word embedding processing. For example, the obtained initial text can be first subjected to vocabulary statistics, all the characters that appear can be determined, and a vocabulary list can be constructed. Secondly, a word embedding model can be selected and trained, such as the Word2Vec or GloVe model trained on large-scale corpus (such as Wikipedia, news articles, etc.). These models have learned rich semantic information, directly download the corresponding word vector file, and then map the characters in the initial text to these pre-trained vectors. After obtaining the word vector, for each character in the initial text, extract the vector of part or all of the text, and you can get the text vector of the target text.

[0036] In triple extraction, the vector processed by word embedding can better help the model understand the semantics of the text. At the same time, word embedding can effectively deal with problems such as polysemy, reflect different semantics in the vector space through context information, and improve the accuracy of triple extraction.

[0037] Furthermore, S110 may also include: preprocessing the initial text and inputting it into a Bert pre-training model for word embedding processing to obtain a text vector of the target text.

[0038] Usually, the initial text often contains various noises and complex formats. For example, the text may contain HTML tags, special characters, spelling errors, or non-standard abbreviations. Preprocessing can clean up these irrelevant information, making the data input into the model cleaner and more standardized, thereby improving the performance and accuracy of the model. The Bert pre-training model is unsupervised pre-trained on a large-scale text corpus. The pre-training process enables the model to learn rich language knowledge, including the semantic representation of vocabulary, grammatical structure, common language patterns, etc. For example, it knows how words of different parts of speech are usually combined, the relationship between different components in a sentence, etc. When used for the downstream task of triple extraction, this pre-training knowledge can be effectively transferred, reducing the dependence on large-scale labeled data, and can achieve good performance after fine-tuning on a relatively small task-related dataset.

[0039] It should be understood that when the initial text itself is relatively clean and standardized, the initial text can also be directly input into the Bert pre-training model, that is, preprocessing of the initial text is not a necessary step.

[0040] It should also be understood that in addition to the Bert pre-trained model, other models can also be used to replace the word embedding processing of the initial text. For example, the Generative Pretrained Transformer (GPT) model, the Enhanced Representation through kNowledge IntEgration (ERNIE) model, etc. The embodiment of the present application does not limit the model for word embedding processing.

[0041] As mentioned above, the target text may be part or all of the initial text. When the target text is part of the initial text, the text vector of the target text may be obtained by preprocessing the initial text and inputting it into the Bert pre-trained model for word embedding processing to obtain the word vector of each word in the initial text, and sequentially combining the word vectors of n consecutively obtained words to obtain the text vector of the target text with a length of n, where n is a positive integer greater than 1.

[0042] For example, the initial text is "Xiao Ming likes to read all kinds of books. He often goes to the library to borrow books. There are many kinds of books in the library, which makes him very happy." After preprocessing the initial text and inputting it into the Bert pre-training model, the vector corresponding to each word in the initial text can be obtained. The vectors corresponding to 10 consecutive words, that is, the vectors corresponding to each word in the first sentence of the initial text "Xiao Ming likes to read all kinds of books", are combined to obtain the text vector of the target text "Xiao Ming likes to read all kinds of books".

[0043] In S120, after obtaining the text vector of the target text, the text vector can be further input into a fully connected neural network and a Sigmod function to calculate the probability that each position in the target text can be used as the starting position of the head entity and the probability that each position in the target text can be used as the ending position of the head entity.

[0044] Specifically, S120 may further include: inputting the text vector into the head entity starting position prediction fully connected layer and Sigmod function, and calculating the probability that each position in the target text can be used as the head entity starting position; inputting the text vector into the head entity ending position prediction fully connected layer and Sigmod function in sequence, and calculating the probability that each position in the target text can be used as the head entity ending position.

[0045] That is to say, in the neural network architecture for relation triple extraction, two fully connected layers are usually required to predict the probability related to the head entity position. Each neuron in the fully connected layer is connected to all neurons in the input layer. It is linearly transformed through the weight matrix and the bias vector. Assuming that the text vector of the target text is x, the weight matrix is ​​W, and the bias vector is b, the output after the linear transformation is y=Wx+b. Afterwards, in order to enhance the nonlinear expression ability, an activation function such as the ReLU function can be added in the middle, that is, z=ReLu(y). After a series of calculations, a vector with the same dimension as the length of the text vector of the target text is finally output. Each element of this vector represents the probability estimate of the corresponding position in the text as the target position of the head entity (for example, the starting position of the head entity or the ending position of the head entity). The probability value is usually obtained by passing the output vector after the fully connected layer through the Sigmoid function.

[0046] After obtaining the probability that each position in the target text can be the starting position of the head entity and the probability that each position in the target text can be the ending position of the head entity, the starting position and the ending position of the head entity can be determined based on a preset threshold, and then the head entity can be determined.

[0047] Specifically, determining the head entity based on the probability that each position in the target text can be the starting position of the head entity and the probability that each position in the target text can be the ending position of the head entity may include: combining the characters from the positions where the probability that each position in the target text can be the starting position of the head entity is higher than a first threshold to the characters from the positions where the probability that each position in the target text can be the ending position of the head entity is higher than a second threshold to obtain the head entity. Herein, the first threshold and the second threshold may be the same or different, and the embodiments of the present application do not limit this.

[0048] For example, for the target text "Beijing is the capital of China", the probabilities of "North", "Beijing", "is", "China", "of", "the", "capital", and "city" being the starting positions of the head entity are obtained as (0.8, 0.5, 0.4, 0.3, 0.25, 0.20, 0.15, 0.1) respectively. Assuming the first threshold is 0.6, then the starting position of the head entity can be determined as the first position of the target text, and the corresponding character is "North". Similarly, the probabilities of these 8 characters being the ending positions of the head entity are obtained as (0.5, 0.7, 0.25, 0.3, 0.35, 0.4, 0.5, 0.1) respectively. Similarly, assuming the second threshold is also 0.6, then the ending position of the head entity can be determined as the second position of the target text, and the corresponding character is "Beijing". Then, the head entity in the target text can be determined as "Beijing".

[0049] It should be understood that the first threshold and the second threshold are empirical values and can be preset. The embodiments of the present application do not limit the specific values of the first threshold and the second threshold.

[0050] After obtaining the head entity in the target text, the head entity vector can then be obtained from the text vector of the target text. It should be noted that the text vector of the target text is actually a combination of the vectors corresponding to each character in the target text. Then, after obtaining the head entity, the head entity vector can be directly extracted from the text vector of the target text according to the positional relationship of each character in the head entity.

[0051] In S130, after obtaining the head entity vector, the text vector of the target text and the head entity vector can be fused to generate a fusion vector. For example, the fusion can be performed by splicing, which is simple and easy to implement, and can completely retain the information of the text vector and the head entity vector. For another example, the fusion can be performed by addition, which can fuse the information of the two vectors to a certain extent, reflecting the mutual correlation between the target text and the head entity. For another example, the fusion can be performed by multiplication or dot product, and the multiplication method can emphasize the common part of the text vector and the head entity vector, which has a certain effect on highlighting the correlation between the two. The dot product method is mainly used to measure the similarity of two vectors, and the obtained scalar value can be used as a fused feature to represent the degree of correlation between the two. In other embodiments, the fusion can also be performed based on a neural network. This method can automatically learn the most effective fusion method between the text vector and the head entity vector, and can adapt to different target texts and head entities through training.

[0052] Furthermore, the obtained fusion vector can be input into the convolutional neural network to obtain the convolutional feature vector of the target text. Although the fusion vector already contains the information of the text vector and the head entity vector, this information is original or preliminarily fused. Convolutional neural networks can automatically learn semantic features in text. For example, for the task of extracting relational triples, the convolutional neural network can automatically learn the semantic pattern representing the entity relationship from the fusion vector. It slides on the fusion vector through the convolution kernel to detect local semantic information at different positions. This automatic learning ability avoids complex artificial feature engineering and enables the model to adaptively mine the potential semantics in the text. In addition, since the convolutional neural network has multiple convolutional layers and pooling layers, multi-level semantic mining can be performed. In the initial convolutional layer, simple word combination features or features related to local grammatical structure may be extracted. As the layers deepen, subsequent convolutional layers can build more complex semantic features based on the previously extracted features, such as the entity information of the sentence or the complex semantic associations between entities.

[0053] After the fusion vector is input into the convolutional neural network to obtain the convolutional feature vector of the target text, the convolutional feature vector can be further input into the Softmax function to calculate the existence probability of each relationship in the target text. Similar to predicting the probability of the starting position or the ending position of the head entity in the target text, if the dimension of the convolutional feature vector is inconsistent with the number of relationship categories, it is also necessary to convert the dimension of the convolutional feature vector to the dimension corresponding to the number of relationship categories through a fully connected layer before inputting it into the Softmax function. Then, the output after the fully connected layer is input into the Softmax function to obtain the existence probability of each relationship in the target text.

[0054] For example, taking the target text "Beijing is the capital of China" as an example, the relationship categories include "capital" and "provincial capital". After the convolutional feature vector of the target text is input into the fully connected layer and the Softmax function in turn, the probabilities of the two relationships are (0.8, 0.5) respectively.

[0055] After obtaining the existence probability of each relationship in the target text, at least one relationship can be further screened out from all relationships. In one embodiment, the existence probabilities of all relationships can be arranged in order from high to low based on a preset K value, and the K relationships with the highest existence probability can be taken. Assuming that in the above example of "Beijing is the capital of China", K is 1, then the relationship "capital" with the highest probability can be selected from all relationships "capital" and "provincial capital". In another embodiment, the existence probability of each relationship can also be directly compared with the preset threshold based on a preset threshold, and the relationship greater than the preset threshold is retained. Assuming that in the above example of "Beijing is the capital of China", the preset threshold is 0.4, then according to the probabilities (0.8, 0.5) corresponding to the two relationships "capital" and "provincial capital", it can be judged that both "capital" and "provincial capital" meet the requirements, and both relationships need to be subsequently processed.

[0056] After selecting K relations that meet the requirements, we can further obtain the global relation parameter matrix M oh and M ot Select the parameter vectors of the K relations respectively and generate two sub-parameter matrices. The global relation parameter matrix M oh and M ot It is an important parameter structure in processing relation triple extraction. oh Mainly used to represent the relationship between the head entity and the ot Mainly used for the relationship representation related to the tail entity. Each of their rows (or columns, depending on the specific definition and application) can be regarded as a vector representation of a relationship, which contains the features learned by the model about a specific relationship in the entire text semantic space. In other words, the global relationship parameter matrix M oh and M ot It is a combination of parameter vectors of all relations, from which the parameter vectors of the K relations that meet the requirements can be extracted to generate two corresponding sub-parameter matrices.

[0057] Furthermore, after obtaining two sub-parameter matrices corresponding to K relations, the fusion vector can be passed into the two sub-parameter matrices respectively, and the probability that each position in the target text can be used as the starting position of the tail entity and the probability that each position in the target text can be used as the ending position of the tail entity are obtained through the Sigmod function, thereby constructing a complete triple.

[0058] It should be understood that the calculation method of the probability that each position in the target text can be the starting position and the ending position of the tail entity is similar to the probability that each position in the target text can be the starting position and the ending position of the head entity. For the sake of brevity, it will not be repeated here.

[0059] In addition, after obtaining the probability that each position in the target text can be the starting position and the ending position of the tail entity, the starting position and the ending position of the tail entity can still be determined based on the preset threshold, and then the tail entity can be determined. After obtaining the head entity, the relationship and the tail entity in sequence, a complete triple can be formed, that is, the extraction of the relationship triple is completed.

[0060] The following will be combined Figure 2 A detailed embodiment of the entity relationship joint extraction method of the embodiment of the present application is described in detail.

[0061] like Figure 2 As shown, assuming that the initial text in the database is [word 1, word 2, word 3, ..., word m], the initial text is input into the Bert pre-training model, and the word vector of each word is obtained sequentially. Let x i ∈R d is the d-dimensional word vector x corresponding to the i-th word in the initial text i , combining the word vectors of consecutive n words can get a text vector of length n as [x 1 ,x 2 ,......,x n ], use x 1:n Represents. The text vector x 1:n The vectors are respectively passed into the fully connected layer for predicting the starting position of the head entity and the fully connected layer for predicting the ending position of the head entity, and linear transformation is performed to obtain the vectors and Then and The Sigmod function is used to calculate the probability of the n positions in the target text being the starting position and the ending position of the head entity. If the probability value of a certain position is not less than the preset value threshold, it is set to 1, indicating that the position is the starting position or the ending position of the head entity, and other positions are set to 0.

[0062] Among them, the text vector x 1:n for: is the cascade operator.

[0063] Further, the head entity position is obtained according to the training set, from the text vector x 1:nSpecifically, the vector of the head entity is the same as x 1:n Then, the text vector and the head entity vector are added to obtain the fusion vector v~; the fusion vector v~ is input into the convolutional neural network, where the convolutional neural network includes the filter w∈R hd , h is the window size of filter w, filter w uses h words as a window to generate a convolution feature vector, and fills the window part beyond the target text boundary with a 0 vector, where x i:i+h-1 After the filter w, the convolution feature vector c is generated i ; Filter w scans all characters {x 1:h , x 2:h+1 , ..., x x-h+1:n} and generate the corresponding convolution feature map c; perform the maximum pooling operation on the convolution feature map c and use the maximum value max{c} as the convolution feature vector of the target text Then the convolution feature vector The probabilities of different relationship categories are calculated by the Softmax function.

[0064] Among them, c i =f(w·x i:i+h-1 ),c={c 1 ,c 2 ,...,c n-h+1}, f is a nonlinear function; c∈R n-h+1 .

[0065] Furthermore, a predetermined value K of relations with the highest occurrence probability in the target text are selected to form a relation set S r , from the global relationship parameter matrix M oh ∈R p×d and M ot ∈R p×d , select S r The labels of the relations appearing in the sub-relation parameter matrix m oh ∈R k×d and m ot ∈R k×d ; The fusion vector Pass in two sub-relation parameter matrices and Sigmod function respectively to get the matrix M poh ∈R n×d and M pot ∈R n×d , for the matrix M poh ∈R n×d , each column of which corresponds to a relation r i ∈S r , which means that for the relationship r i The probability of extracting the starting position of the tail entity; for the matrix Mpot ∈R n×d , each column of which corresponds to a relation r i ∈S r , which means that for the relationship r i The probability of extracting the end position of the tail entity; for each target text, this method can be used to extract the head entity e through the above method h , for each relation r i ∈S r , extract e h The corresponding tail entity e t .

[0066] The entity relationship joint extraction method provided in the embodiment of the present application predicts the probability of various relationships and extracts some relationships to predict the tail entity, rather than using all relationships to predict the tail entity, thereby reducing the extraction of unnecessary relationships and reducing the amount of calculation. In addition, the entity is perceived by the relationship, which increases the degree of interaction between entity information and relationship information. Finally, the triples obtained by the entity relationship joint extraction method in the embodiment of the present application have high precision, recall rate, and F1 value.

[0067] Table 1 shows the comparison of the precision, recall rate and F1 value obtained by some traditional entity relationship joint extraction models and the entity relationship joint extraction model provided by the embodiment of the present application. From Table 1, it can be seen that the precision, recall rate and F1 value of the entity relationship joint extraction model provided by the embodiment of the present application are all higher.

[0068] Table 1

[0069] Model Accuracy Recall F1 Novel-Tagging 75.8 36.7 49.4 GraphRel 52.2 25.8 34.5 CMAN 77.3 68.1 72.4 TPLinker 78.4 70.5 74.2 This application model 79.3 68.3 73.8

[0070] The present application also provides a device for extracting entity relationships. Figure 3 As shown, the entity relationship joint extraction device 200 includes part or all of the following contents.

[0071] The word embedding processing module 210 is used to perform word embedding processing on the initial text to be subjected to triple extraction, and obtain the text vector of the target text in the initial text.

[0072] The head entity tagging module 220 is used to calculate the probability that each position in the target text can be used as the starting position of the head entity and the probability that each position in the target text can be used as the ending position of the head entity through the fully connected neural network and the Sigmod function of the text vector, and determine the head entity based on the probability that each position in the target text can be used as the starting position of the head entity and the probability that each position in the target text can be used as the ending position of the head entity.

[0073] The relationship identification module 230 is used to fuse the text vector and the head entity vector to generate a fusion vector, input the fusion vector into a convolutional neural network to obtain a convolution feature vector of the target text, and calculate the convolution feature vector through a Softmax function to obtain the existence probability of each relationship in the target text. The head entity vector is a vector corresponding to the head entity obtained from the text vector.

[0074] The tail entity tagging module 240 is used to select the K relationships with the highest existence probability and oh and M ot The parameter vectors of the K relations are respectively selected to generate two sub-parameter matrices, the fusion vector is passed into the two sub-parameter matrices, and the probability that each position in the target text can be used as the starting position of the tail entity and the probability that each position in the target text can be used as the ending position of the tail entity are calculated by the Sigmod function, thereby constructing a complete triple, where K is a positive integer.

[0075] Optionally, in an embodiment of the present application, the word embedding processing module 210 is specifically used to: pre-process the initial text, and input it into the Bert pre-trained model for word embedding processing to obtain the text vector of the target text.

[0076] Optionally, in an embodiment of the present application, the word embedding processing module 210 is specifically used to: preprocess the initial text, and input it into the Bert pre-trained model for word embedding processing to obtain a word vector for each word in the initial text; combine the word vectors of n words obtained successively in sequence to obtain a text vector of the target text with a length of n, where n is a positive integer greater than 1.

[0077] Optionally, in an embodiment of the present application, the head entity marking module 220 is specifically used to: input the text vector into the head entity starting position prediction fully connected layer and Sigmod function, and calculate the probability that each position in the target text can be used as the starting position of the head entity; input the text vector into the head entity ending position prediction fully connected layer and Sigmod function in sequence, and calculate the probability that each position in the target text can be used as the ending position of the head entity.

[0078] Optionally, in an embodiment of the present application, the head entity marking module 220 is specifically used to combine the characters at positions above a first threshold in the probability that each position in the target text can be the starting position of the head entity to the characters at positions above a second threshold in the probability that each position in the target text can be the ending position of the head entity to obtain the head entity.

[0079] Based on the same idea, the embodiment of the present application also provides another entity relationship joint extraction device. The entity relationship joint extraction device includes a memory for storing computer program instructions and a processor for executing the program instructions, wherein when the program instructions are executed by the processor, the entity relationship joint extraction device executes the above-mentioned various method embodiments.

[0080] It should be noted that the detailed contents of the device side embodiment can be found in the method side embodiment, and for the sake of brevity, they will not be repeated here.

[0081] Based on the same idea, an embodiment of the present application also provides a computer-readable storage medium, which includes a stored program, wherein when the program is running, the device where the computer-readable storage medium is located is controlled to execute the above-mentioned various method embodiments.

[0082] Although the present application has been described with reference to preferred embodiments, various modifications may be made thereto and parts thereof may be replaced with equivalents without departing from the scope of the present application. In particular, the various technical features mentioned in the various embodiments may be combined in any manner as long as there are no structural conflicts. The present application is not limited to the specific embodiments disclosed herein, but includes all technical solutions falling within the scope of the claims.

Claims

1. A method for joint extraction of entity relations, characterized in that: include: Performing word embedding processing on the initial text to be subjected to triple extraction to obtain a text vector of the target text in the initial text; Inputting the text vector into a fully connected neural network and a Sigmod function, calculating the probability that each position in the target text can be used as the starting position of a head entity and the probability that the head entity can be used as the ending position, and determining the head entity based on the probability that each position in the target text can be used as the starting position of a head entity and the probability that the head entity can be used as the ending position; The text vector and the head entity vector are fused to generate a fused vector, the fused vector is input into a convolutional neural network to obtain a convolutional feature vector of the target text, and the convolutional feature vector is input into a Softmax function to calculate the existence probability of each relationship in the target text, wherein the head entity vector is a vector corresponding to the head entity obtained from the text vector; Select the K relationships with the highest probability of existence and select them from the global relationship parameter matrix M oh and M ot The parameter vectors of the K relations are respectively selected to generate two sub-parameter matrices, the fusion vector is passed into the two sub-parameter matrices, and the probability that each position in the target text can be used as the starting position of the tail entity and the probability of the ending position of the tail entity are calculated by the Sigmod function, thereby constructing a complete triple, where K is a positive integer.

2. The entity relationship joint extraction method according to claim 1 is characterized in that: The step of performing word embedding processing on the initial text to be subjected to triple extraction to obtain a text vector of a target text in the initial text includes: The initial text is preprocessed and input into the Bert pre-trained model for word embedding processing to obtain the text vector of the target text.

3. The entity relationship joint extraction method according to claim 2 is characterized in that: The initial text is preprocessed and input into the Bert pre-trained model for word embedding processing to obtain the text vector of the target text, including: Preprocessing the initial text and inputting it into the Bert pre-training model for word embedding processing to obtain a word vector for each word in the initial text; The word vectors of n words obtained successively are combined to obtain a text vector of the target text with a length of n, where n is a positive integer greater than 1.

4. The entity relationship joint extraction method according to claim 1 is characterized in that: The step of inputting the text vector into a fully connected neural network and a Sigmod function, and calculating the probability that each position in the target text can be used as the starting position of a head entity and the probability that the head entity ends, includes: Input the text vector into the head entity starting position prediction fully connected layer and Sigmod function, and calculate the probability that each position in the target text can be used as the head entity starting position; The text vector is sequentially input into the head entity end position prediction fully connected layer and the Sigmod function, and the probability that each position in the target text can be used as the head entity end position is calculated.

5. The entity relationship joint extraction method according to claim 1, characterized in that: The determining of the head entity based on the probability that each position in the target text can be used as the starting position of the head entity and the probability that the head entity ends, comprises: The head entity is obtained by combining the characters at positions above a first threshold in the probability that each position in the target text can be the starting position of the head entity to the characters at positions above a second threshold in the probability that each position in the target text can be the ending position of the head entity.

6. An entity relationship joint extraction device, characterized in that: include: A word embedding processing module, used for performing word embedding processing on an initial text to be subjected to triple extraction, to obtain a text vector of a target text in the initial text; A head entity tagging module, used to input the text vector into a fully connected neural network and a Sigmod function, calculate the probability that each position in the target text can be used as the starting position of the head entity and the probability that the head entity ends, and determine the head entity based on the probability that each position in the target text can be used as the starting position of the head entity and the probability that the head entity ends; A relationship recognition module, used for fusing the text vector and the head entity vector to generate a fusion vector, inputting the fusion vector into a convolutional neural network to obtain a convolution feature vector of the target text, and inputting the convolution feature vector into a Softmax function to calculate the existence probability of each relationship in the target text, wherein the head entity vector is a vector corresponding to the head entity obtained from the text vector; The tail entity tagging module is used to select the K relations with the highest existence probability and extract them from the global relation parameter matrix M oh and M ot The parameter vectors of the K relations are respectively selected to generate two sub-parameter matrices, the fusion vector is passed into the two sub-parameter matrices, and the probability that each position in the target text can be used as the starting position of the tail entity and the probability of the ending position of the tail entity are calculated by the Sigmod function, thereby constructing a complete triple, where K is a positive integer.

7. The entity relationship joint extraction device according to claim 6, characterized in that: The word embedding processing module is specifically used for: The initial text is preprocessed and input into the Bert pre-trained model for word embedding processing to obtain the text vector of the target text.

8. The entity relationship joint extraction device according to claim 7, characterized in that: The word embedding processing module is specifically used for: Preprocessing the initial text and inputting it into the Bert pre-training model for word embedding processing to obtain a word vector for each word in the initial text; The word vectors of n words obtained successively are combined to obtain a text vector of the target text with a length of n, where n is a positive integer greater than 1.

9. The entity relationship joint extraction device according to claim 6, characterized in that: The header entity marking module is specifically used for: Input the text vector into the head entity starting position prediction fully connected layer and Sigmod function, and calculate the probability that each position in the target text can be used as the head entity starting position; The text vector is sequentially input into the head entity end position prediction fully connected layer and the Sigmod function, and the probability that each position in the target text can be used as the head entity end position is calculated.

10. The entity relationship joint extraction device according to claim 6, characterized in that: The header entity marking module is specifically used for: The head entity is obtained by combining the characters at positions above a first threshold in the probability that each position in the target text can be the starting position of the head entity to the characters at positions above a second threshold in the probability that each position in the target text can be the ending position of the head entity.

Images