Bi-affine overlapping relationship extraction method and system based on potential relationship prediction

Abstract

The present invention proposes a method and system for extracting double affine overlapping relations based on potential relationship prediction, the method comprising: preprocessing sentences for overlapping relationship extraction into an index form; converting words in the sentence into indexed lengths to generate a three-dimensional target matrix; marking the three-dimensional target matrix by handshaking and pairing each word in the sentence with itself, the preceding word, and the following word; inputting the indexed sentence and the marked three-dimensional target matrix into a Bert model for training to obtain a hidden layer vector containing a potential relationship; performing a linear transformation on the hidden layer vector to obtain a matrix having the same shape as the three-dimensional target matrix; obtaining a final model after training is completed and verified; using the final model to realize double affine overlapping relationship extraction, and based on the method, an extraction system is also proposed. The present invention can effectively utilize the potential relationships in a sentence to extract the entities corresponding to each relationship through potential relationship prediction and double affine transformation.

Description

Technical Field

[0001] The present invention belongs to the technical field of knowledge graph construction, and in particular relates to a method and system for extracting double affine overlapping relationships based on potential relationship prediction. Background Art

[0002] Relation extraction is an important subtask in the process of knowledge graph construction. Relation extraction mainly extracts triple information containing subject, object and corresponding relationship from unstructured or semi-structured text. The joint learning model can perform entity recognition and relationship classification at the same time, which can closely link these two subtasks. Therefore, the model can learn the deeper connection between entity pairs and relationships. Considering that the category of the relationship may have a potential positive impact on the recognition of the entities that make up the triples in the sentence, the bidirectionality of the relationship between the subject and the object, and the application of multi-label classification methods to the simultaneous extraction of entity pairs and relationships.

[0003] Traditional pipeline-based methods first extract the subject and object entities in the sentence, and then classify the relationships between the extracted candidate entity pairs. In this way, the model can only learn the shallow connection between entity pairs and relationships. Summary of the invention

[0004] In order to solve the above technical problems, the present invention proposes a dual affine overlapping relationship extraction method and system based on potential relationship prediction, which can effectively utilize the potential relationships in the sentence to extract the entities corresponding to each relationship, so that the model can extract all triples in the sentence at one time, providing good support for the construction of knowledge graphs.

[0005] To achieve the above object, the present invention adopts the following technical solutions:

[0006] The double affine overlapping relationship extraction method based on potential relationship prediction includes the following steps:

[0007] Preprocess the sentences used for overlapping relationship extraction into index form; convert the words in each sentence into the length after indexing, and generate a three-dimensional target matrix with an initial value of zero;

[0008] Marking the three-dimensional target matrix by performing a handshake pairing between each word in the sentence and itself, each preceding word, and each succeeding word;

[0009] The indexed sentences and the labeled three-dimensional target matrix are input into the Bert model for fine-tuning training to obtain a hidden layer vector containing potential relationships; the hidden layer vector is linearly transformed to obtain a matrix with the same shape as the three-dimensional target matrix; the final Bert model is obtained after training is completed and verified; the final Bert model is used to realize dual affine overlapping relationship extraction.

[0010] Furthermore, the formal method of preprocessing the sentences for overlapping relationship extraction into indexes is:

[0011] A tokenizer is used to convert sentences used for overlapping relation extraction into index form.

[0012] Furthermore, the three-dimensional target matrix is ​​[maximum sentence length*maximum sentence length*(number of relationship categories+4)].

[0013] Furthermore, the process of marking the three-dimensional target matrix by performing a handshake pairing on each word in the sentence with itself and each preceding word and each succeeding word includes:

[0014] If the subject entity in the triplet comes first and the object entity comes last, the following marking operation is performed on the upper triangular part of the target matrix;

[0015] If the object entity in the triplet comes first and the subject entity comes last, the following marking operation is performed on the lower triangular part of the target matrix.

[0016] Furthermore, if the subject entity in the triplet is in front and the object entity is in the back, the following marking operation is performed on the upper triangular part of the target matrix specifically including:

[0017] If the two paired words are the head of the subject entity and the head of the object entity in the triple, the last position of the third dimension of the target matrix is ​​marked as 1, and the relationship category corresponding to the subject entity and the object entity is marked as 1 in the corresponding position of the third dimension;

[0018] If the two paired words are the head of the subject entity and the tail of the object entity in the triple, mark the second to last position of the third dimension of the target matrix as 1, and mark the corresponding position of the relationship category corresponding to the subject entity and the object entity as 1 in the third dimension;

[0019] If the two paired words are the tail of the subject entity and the head of the object entity in the triple, mark the third to last position of the third dimension of the target matrix as 1, and mark the corresponding position of the relationship category corresponding to the subject entity and the object entity in the third dimension as 1;

[0020] If the two paired words are the tail of the subject entity and the tail of the object entity in the triple, the fourth to last position of the third dimension of the target matrix is ​​marked as 1, and the relationship categories corresponding to the subject entity and the object entity are marked as 1 in the corresponding positions of the third dimension.

[0021] Furthermore, if the object entity in the triple is in front and the subject entity is in the back, the following marking operation is performed on the lower triangular part of the target matrix specifically including:

[0022] If the two paired words are the head of the object entity and the head of the subject entity in the triple, the last position of the third dimension of the target matrix is ​​marked as 1, and the relationship category corresponding to the object entity and the subject entity is marked as 1 in the corresponding position of the third dimension;

[0023] If the two paired words are the head of the object entity and the tail of the subject entity in the triple, mark the second to last position of the third dimension of the target matrix as 1, and mark the corresponding position of the relationship category corresponding to the object entity and the subject entity as 1 in the third dimension;

[0024] If the two paired words are the tail of the object entity and the head of the subject entity in the triple, mark the third to last position of the third dimension of the target matrix as 1, and mark the corresponding position of the relationship category corresponding to the object entity and the subject entity in the third dimension as 1;

[0025] If the two paired words are the tail of the object entity and the tail of the subject entity in the triple, the fourth to last position of the third dimension of the target matrix is ​​marked as 1, and the relationship category corresponding to the object entity and the subject entity is marked as 1 in the corresponding position of the third dimension.

[0026] Furthermore, the process of inputting the indexed sentence and the labeled three-dimensional target matrix into the Bert model for fine-tuning training to obtain the hidden layer vector containing the potential relationship is:

[0027] Obtain the hidden vector of the last layer of Bert, and use the hidden vector of the last layer to predict the potential relationship of the sentence;

[0028] After obtaining the potential relationship, an embedding coding operation is performed on each obtained relationship, and then the potential relationship vectors are added, and then the added potential relationship vectors are added to the hidden vector corresponding to each word in the last layer of Bert during the training process to obtain a hidden layer vector containing the potential relationship.

[0029] Furthermore, the linear transformation of the hidden layer vector to obtain a matrix having the same shape as the three-dimensional target matrix includes:

[0030] First, two linear transformations are used to obtain the vector representation of each word in the sentence as the subject entity and the vector representation of each word as the object entity.

[0031] Inputting the vector representation of the subject entity and the vector representation of the object entity into the dual affine model respectively to obtain a dual affine model matrix; the dual affine model matrix is ​​expressed as [maximum sentence length*maximum sentence length*hidden layer size];

[0032] The bi-affine model matrix undergoes another linear transformation and is activated using an activation function to obtain a matrix having the same shape as the three-dimensional target matrix; the matrix having the same shape as the three-dimensional target matrix is ​​expressed as [maximum sentence length*maximum sentence length*(number of relationship categories+4)].

[0033] Furthermore, the process of using the final Bert model to extract the double affine overlapping relationship is as follows:

[0034] Input the test data into the final Bert model to obtain a final three-dimensional matrix; the final three-dimensional matrix is ​​expressed as [maximum sentence length*maximum sentence length*(number of relationship categories+4)];

[0035] Decoding the upper triangular part of the final three-dimensional matrix, the subject entity can be determined by using the dependency relationship of subject entity head-object entity head and subject entity tail-object entity tail and subject entity head-object entity tail and subject entity tail-object entity tail; the object entity can be determined by using subject entity head-object entity head and subject entity head-object entity tail and subject entity tail-object entity head and subject entity tail-object entity tail;

[0036] Then, the relationship category index on the dependency relationship can be used to determine the relationship category corresponding to the subject and the object; and a positive <subject, relationship, object> triple is obtained;

[0037] Similarly, by decoding the lower triangular part of the matrix representation, we can obtain the reverse <object, relationship, entity> triplet.

[0038] The present invention also proposes a dual affine overlapping relationship extraction system based on potential relationship prediction, including a preprocessing module, a marking module and a training extraction module;

[0039] The preprocessing module is used to preprocess the sentences used for overlapping relationship extraction into an index form; convert the words in each sentence into the length after the index, and generate a three-dimensional target matrix with an initial value of zero;

[0040] The marking module is used to mark the three-dimensional target matrix by performing a handshake pairing between each word in the sentence and itself, each preceding word, and each following word;

[0041] The training extraction module is used to input the indexed sentences and the labeled three-dimensional target matrix into the Bert model for fine-tuning training to obtain a hidden layer vector containing a potential relationship; linearly transform the hidden layer vector to obtain a matrix with the same shape as the three-dimensional target matrix; obtain the final Bert model after training is completed and verified; and use the final Bert model to implement dual affine overlapping relationship extraction.

[0042] The effects provided in the content of the invention are only the effects of the embodiments, not all the effects of the invention. One of the above technical solutions has the following advantages or beneficial effects:

[0043] The present invention proposes a method and system for extracting dual affine overlapping relations based on potential relationship prediction, the method comprising: preprocessing sentences for overlapping relationship extraction into an index form; converting words in each sentence into an indexed length to generate a three-dimensional target matrix with an initial value of zero; marking the three-dimensional target matrix by handshaking each word in the sentence with itself and each preceding word and each succeeding word; inputting the indexed sentence and the marked three-dimensional target matrix into a Bert model for fine-tuning training to obtain a hidden layer vector containing a potential relationship; performing a linear transformation on the hidden layer vector to obtain a matrix with the same shape as the three-dimensional target matrix; obtaining a final Bert model after training is completed and verified; using the final Bert model to implement dual affine overlapping relationship extraction, a dual affine overlapping relationship extraction method based on potential relationship prediction, and a dual affine overlapping relationship extraction system based on potential relationship prediction are also proposed. Compared with the prior art that can only extract one triple from a sentence at a time, the present invention can effectively utilize the potential relationships in the sentence to extract the entities corresponding to each relationship through potential relationship prediction and double affine transformation, so that the model can extract all triples in the sentence at one time, providing good support for the construction of the knowledge graph. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] like Figure 1 This is a flow chart of a method for extracting double affine overlapping relations based on potential relation prediction according to Embodiment 1 of the present invention;

[0045] like Figure 2 Schematic diagram of a dual affine overlapping relationship extraction system based on potential relationship prediction according to Embodiment 2 of the present invention. DETAILED DESCRIPTION

[0046] In order to clearly illustrate the technical features of the present solution, the present invention is described in detail below through specific implementation methods and in conjunction with the accompanying drawings. The disclosure below provides many different embodiments or examples for realizing different structures of the present invention. In order to simplify the disclosure of the present invention, the components and settings of specific examples are described below. In addition, the present invention may repeat reference numbers and / or letters in different examples. This repetition is for the purpose of simplification and clarity, and does not itself indicate the relationship between the various embodiments and / or settings discussed. It should be noted that the components illustrated in the accompanying drawings are not necessarily drawn to scale. The present invention omits the description of known components and processing techniques and processes to avoid unnecessary limitations on the present invention.

[0047] Example 1

[0048] Embodiment 1 of the present invention proposes a dual affine overlapping relationship extraction method based on potential relationship prediction, applies the predicted potential relationship in the sentence to the recognition of entity pairs, and then uses the dual affine mechanism to pair the word vector representations in the sentence to form a three-dimensional target matrix representation. Finally, by decoding the generated three-dimensional target matrix under four constraints, all triple information in the sentence can be obtained.

[0049] The overall steps are: pre-process the sentences used for overlapping relationship extraction into index form; convert the words in each sentence into the length after indexing, and generate a three-dimensional target matrix with an initial value of zero;

[0050] Marking the three-dimensional target matrix by performing a handshake pairing between each word in the sentence and itself, each preceding word, and each succeeding word;

[0051] The indexed sentences and the labeled three-dimensional target matrix are input into the Bert model for fine-tuning training to obtain the hidden layer vector containing the potential relationship; the hidden layer vector is linearly transformed to obtain a matrix with the same shape as the three-dimensional target matrix; after training and verification, the final Bert model is obtained; the final Bert model is used to realize the extraction of dual affine overlapping relationships.

[0052] like Figure 1 This is a flow chart of a method for extracting double affine overlapping relations based on potential relation prediction according to Embodiment 1 of the present invention;

[0053] Preprocess each sentence to be input, and use Bert's tokenizer to convert the sentence to be input into the model into an index form, where tokenizer is a word segmenter.

[0054] According to the length of the words in each sentence after being processed and converted into indexes, a three-dimensional target matrix with a shape of [maximum sentence length * maximum sentence length * (number of relationship categories + 4)] and an initial value of all 0 is generated.

[0055] The three-dimensional target matrix is ​​labeled, pairing each word in the sentence with itself and each previous and next word in a handshake. If the subject entity in the triple is in front and the object entity is in the back, the following marking operation is performed on the upper triangular part of the target matrix: if the two paired words are the head of the subject entity and the head of the object entity in the triple, the last position of the third dimension of the target matrix is ​​marked as 1, and the relationship category corresponding to the subject entity and the object entity is marked as 1 in the corresponding position of the third dimension; if the two paired words are the head of the subject entity and the tail of the object entity in the triple, the second to last position of the third dimension of the target matrix is ​​marked as 1, and the relationship category corresponding to the subject entity and the object entity is marked as 1 in the corresponding position of the third dimension; if the two paired words are the tail of the subject entity and the head of the object entity in the triple, the third to last position of the third dimension of the target matrix is ​​marked as 1, and the relationship category corresponding to the subject entity and the object entity is marked as 1 in the corresponding position of the third dimension; if the two paired words are the tail of the subject entity and the tail of the object entity in the triple, the fourth to last position of the third dimension of the target matrix is ​​marked as 1, and the relationship category corresponding to the subject entity and the object entity is marked as 1 in the corresponding position of the third dimension.

[0056] On the contrary, if the object entity in the triple is in front and the subject entity is in the back, the following marking operation is performed on the lower triangular part of the target matrix: if the two paired words are the head of the object entity and the head of the subject entity in the triple, the last position of the third dimension of the target matrix is ​​marked as 1, and the relationship category corresponding to the object entity and the subject entity is marked as 1 in the corresponding position of the third dimension; if the two paired words are the head of the object entity and the tail of the subject entity in the triple, the second to last position of the third dimension of the target matrix is ​​marked as 1, and the object entity and the subject entity are marked as 1. The relationship category corresponding to the entity is marked as 1 at the corresponding position of the third dimension; if the two paired words are the tail of the object entity and the head of the subject entity in the triple, the third to last position of the third dimension of the target matrix is ​​marked as 1, and the relationship category corresponding to the object entity and the subject entity is marked as 1 at the corresponding position of the third dimension; if the two paired words are the tail of the object entity and the tail of the subject entity in the triple, the fourth to last position of the third dimension of the target matrix is ​​marked as 1, and the relationship category corresponding to the object entity and the subject entity is marked as 1 at the corresponding position of the third dimension.

[0057] After data preprocessing, we input the indexed sentences and the processed target matrix into the Bert model for fine-tuning training. During the training process, we obtain the hidden vector of the last layer of Bert and use the hidden vector of the last layer to predict the potential relationship of the sentence. After obtaining the potential relationship, we perform embedding coding on each relationship obtained, and then add the potential relationship vectors. Then, we add it to the hidden vector corresponding to each word in the last layer of Bert during the training process, and we get the hidden layer vector containing the potential relationship.

[0058] After obtaining the hidden layer vector with potential relationship, two linear transformations are used to obtain the vector representation of each word in the sentence as the subject entity and the vector representation of the object entity respectively. Then these two vector representations are input into the bi-affine model to obtain a matrix representation with the shape of [maximum sentence length * maximum sentence length * hidden layer size].

[0059] After obtaining the matrix representation after the double affine transformation, perform another linear transformation, and then use the sigmoid activation function to activate it, and then obtain a matrix representation with the same shape as the target matrix, which is [maximum sentence length * maximum sentence length * (number of relationship categories + 4)].

[0060] After the supervised training of the model is completed, the best performing model can be obtained by verifying it with the validation set to obtain the final model.

[0061] Input the test data into the model and get a three-dimensional matrix representation of [maximum sentence length * maximum sentence length * (number of relationship categories + 4)]. Decode the upper triangular part of the obtained matrix representation, and determine the subject entity by using the dependency relationships of subject entity head-object entity head and subject entity tail-object entity tail and subject entity head-object entity tail and subject entity tail-object entity tail; determine the object entity by using subject entity head-object entity head and subject entity head-object entity tail and subject entity tail-object entity head and subject entity tail-object entity tail; and then determine the relationship category corresponding to the subject and the object by the relationship category index on the dependency relationship, because the positive <subject, relationship, object> triplet can be obtained. Similarly, decode the lower triangular part of the matrix representation and get the reverse <object, relationship, entity> triplet.

[0062] The double affine overlapping relationship extraction method based on potential relationship prediction proposed in Example 1 of the present invention utilizes the potential relationship information in the sentence to improve the accuracy of triple entity pair recognition in the sentence. Using the double affine mechanism, each word in the sentence is paired as the subject and the object in two cases to form a three-dimensional target matrix. The directionality of the relationship between the subject and the object can be determined by the upper and lower triangular parts of the matrix. The third dimension of the three-dimensional target matrix combines the entity pairs and relationship categories in the form of four constraints, and then uses a multi-label classification method to extract the subject and object and classify the relationship. Compared with the research in the prior art that can only extract one triple from a sentence at a time, Example 1 of the present invention can effectively utilize the potential relationship in the sentence to extract the entity corresponding to each relationship through potential relationship prediction and double affine transformation, so that the model can extract all triples in the sentence at one time, providing good support for the construction of the knowledge graph.

[0063] Example 2

[0064] Based on the double affine overlapping relationship extraction method based on potential relationship prediction proposed in Example 1 of the present invention. Example 2 of the present invention also proposes a double affine overlapping relationship extraction system based on potential relationship prediction, which includes a preprocessing module, a labeling module and a training extraction module;

[0065] The preprocessing module is used to preprocess the sentences used for overlapping relationship extraction into an indexed form; convert the words in each sentence into the length after the index, and generate a three-dimensional target matrix with an initial value of zero;

[0066] The marking module is used to mark the three-dimensional target matrix by performing a handshake pairing on each word in the sentence with itself, each preceding word, and each following word;

[0067] The training and extraction module is used to input the indexed sentences and the labeled three-dimensional target matrix into the Bert model for fine-tuning training to obtain a hidden layer vector containing potential relationships; linearly transform the hidden layer vector to obtain a matrix with the same shape as the three-dimensional target matrix; obtain the final Bert model after training is completed and verified; and use the final Bert model to realize dual affine overlapping relationship extraction.

[0068] The process of implementing the preprocessing module is: using a word segmenter to convert sentences used for overlapping relationship extraction into an index form.

[0069] The three-dimensional target matrix is ​​[maximum sentence length * maximum sentence length * (number of relationship categories + 4)].

[0070] The process of marking module implementation includes:

[0071] If the subject entity in the triplet comes first and the object entity comes second, the following marking operation is performed on the upper triangular part of the target matrix; if the object entity in the triplet comes first and the subject entity comes second, the following marking operation is performed on the lower triangular part of the target matrix.

[0072] If the subject entity in the triplet comes first and the object entity comes last, the following marking operations are performed on the upper triangular part of the target matrix:

[0073] If the two paired words are the head of the subject entity and the head of the object entity in the triple, the last position of the third dimension of the target matrix is ​​marked as 1, and the relationship category corresponding to the subject entity and the object entity is marked as 1 in the corresponding position of the third dimension;

[0074] If the two paired words are the head of the subject entity and the tail of the object entity in the triple, mark the second to last position of the third dimension of the target matrix as 1, and mark the corresponding position of the relationship category corresponding to the subject entity and the object entity as 1 in the third dimension;

[0075] If the two paired words are the tail of the subject entity and the head of the object entity in the triple, mark the third to last position of the third dimension of the target matrix as 1, and mark the corresponding position of the relationship category corresponding to the subject entity and the object entity in the third dimension as 1;

[0076] If the two paired words are the tail of the subject entity and the tail of the object entity in the triple, the fourth to last position of the third dimension of the target matrix is ​​marked as 1, and the relationship categories corresponding to the subject entity and the object entity are marked as 1 in the corresponding positions of the third dimension.

[0077] If the object entity in the triplet comes first and the subject entity comes last, the following marking operations are performed on the lower triangular part of the target matrix:

[0078] If the two paired words are the head of the object entity and the head of the subject entity in the triple, the last position of the third dimension of the target matrix is ​​marked as 1, and the relationship category corresponding to the object entity and the subject entity is marked as 1 in the corresponding position of the third dimension;

[0079] If the two paired words are the head of the object entity and the tail of the subject entity in the triple, mark the second to last position of the third dimension of the target matrix as 1, and mark the corresponding position of the relationship category corresponding to the object entity and the subject entity as 1 in the third dimension;

[0080] If the two paired words are the tail of the object entity and the head of the subject entity in the triple, mark the third to last position of the third dimension of the target matrix as 1, and mark the corresponding position of the relationship category corresponding to the object entity and the subject entity in the third dimension as 1;

[0081] If the two paired words are the tail of the object entity and the tail of the subject entity in the triple, the fourth to last position of the third dimension of the target matrix is ​​marked as 1, and the relationship category corresponding to the object entity and the subject entity is marked as 1 in the corresponding position of the third dimension.

[0082] The process of implementing the training extraction module is as follows: obtaining the hidden vector of the last layer of Bert, and using the hidden vector of the last layer to predict the potential relationship of the sentence;

[0083] After obtaining the potential relationship, an embedding coding operation is performed on each obtained relationship, and then the potential relationship vectors are added, and then the added potential relationship vectors are added to the hidden vector corresponding to each word in the last layer of Bert during the training process to obtain a hidden layer vector containing the potential relationship.

[0084] Linearly transforming the hidden layer vector to obtain a matrix having the same shape as the three-dimensional target matrix includes:

[0085] First, two linear transformations are used to obtain the vector representation of each word in the sentence as the subject entity and the vector representation of each word as the object entity.

[0086] Inputting the vector representation of the subject entity and the vector representation of the object entity into the dual affine model respectively to obtain a dual affine model matrix; the dual affine model matrix is ​​expressed as [maximum sentence length*maximum sentence length*hidden layer size];

[0087] The bi-affine model matrix undergoes another linear transformation and is activated using an activation function to obtain a matrix having the same shape as the three-dimensional target matrix; the matrix having the same shape as the three-dimensional target matrix is ​​expressed as [maximum sentence length*maximum sentence length*(number of relationship categories+4)].

[0088] Input the test data into the final Bert model to obtain a final three-dimensional matrix; the final three-dimensional matrix is ​​expressed as [maximum sentence length*maximum sentence length*(number of relationship categories+4)];

[0089] Decode the upper triangular part of the final three-dimensional matrix, and determine the subject entity by using the dependency relationships of subject entity head-object entity head and subject entity tail-object entity tail, as well as subject entity head-object entity tail and subject entity tail-object entity tail; determine the object entity by using subject entity head-object entity head and subject entity head-object entity tail, as well as subject entity tail-object entity head and subject entity tail-object entity tail;

[0090] Then, the relationship category index on the dependency relationship can be used to determine the relationship category corresponding to the subject and the object; and a positive <subject, relationship, object> triple is obtained;

[0091] Similarly, by decoding the lower triangular part of the matrix representation, we can obtain the reverse <object, relationship, entity> triplet.

[0092] The dual affine overlapping relationship extraction system based on potential relationship prediction proposed in Example 2 of the present invention utilizes the potential relationship information in the sentence to improve the accuracy of triple entity pair recognition in the sentence. Using the dual affine mechanism, each word in the sentence is paired as the subject and the object in two cases to form a three-dimensional target matrix. The directionality of the relationship between the subject and the object can be determined by the upper and lower triangular parts of the matrix. The third dimension of the three-dimensional target matrix combines the entity pairs and relationship categories in the form of four constraints, and then uses a multi-label classification method to extract the subject and object and classify the relationship. Compared with the research in the prior art that can only extract one triple from a sentence at a time, Example 2 of the present invention can effectively utilize the potential relationship in the sentence to extract the entity corresponding to each relationship through potential relationship prediction and dual affine transformation, so that the model can extract all triples in the sentence at one time, providing good support for the construction of the knowledge graph.

[0093] It should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply that there is any such actual relationship or order between these entities or operations. Moreover, the term "include", "comprise" or any other variant thereof is intended to cover non-exclusive inclusion, so that the process, method, article or equipment that includes a series of elements are inherent to the elements. In the absence of more restrictions, the elements limited by the sentence "comprise one..." do not exclude the presence of other identical elements in the process, method, article or equipment that includes the elements. In addition, the above-mentioned technical solution provided in the embodiment of the present application is consistent with the corresponding technical solution in the prior art in principle, and the part is not described in detail, so as not to repeat too much.

[0094] Although the above describes the specific implementation of the present invention in conjunction with the accompanying drawings, it is not intended to limit the scope of protection of the present invention. For those skilled in the art, other different forms of modifications or deformations can be made on the basis of the above description. It is not necessary and impossible to list all the implementation methods here. On the basis of the technical solution of the present invention, various modifications or deformations that can be made by those skilled in the art without creative work are still within the scope of protection of the present invention.

Claims

1. A dual affine overlapping relationship extraction method based on potential relationship prediction, characterized in that: The following steps are involved: Preprocess the sentences used for overlapping relationship extraction into index form; convert the words in each sentence into the length after indexing, and generate a three-dimensional target matrix with an initial value of zero; Marking the three-dimensional target matrix by performing a handshake pairing between each word in the sentence and itself, each preceding word, and each succeeding word; The process of marking the three-dimensional target matrix by performing a handshake pairing on each word in the sentence with itself and each preceding word and each succeeding word includes: if the subject entity in the triple is in front and the object entity is in the back, then the following marking operation is performed on the upper triangular part of the target matrix; if the object entity in the triple is in front and the subject entity is in the back, then the following marking operation is performed on the lower triangular part of the target matrix; If the subject entity in the triplet is in front and the object entity is in the back, the following marking operation is performed on the upper triangular part of the target matrix, specifically including: If the two paired words are the head of the subject entity and the head of the object entity in the triple, the last position of the third dimension of the target matrix is ​​marked as 1, and the relationship category corresponding to the subject entity and the object entity is marked as 1 in the corresponding position of the third dimension; If the two paired words are the head of the subject entity and the tail of the object entity in the triple, mark the second to last position of the third dimension of the target matrix as 1, and mark the corresponding position of the relationship category corresponding to the subject entity and the object entity as 1 in the third dimension; If the two paired words are the tail of the subject entity and the head of the object entity in the triple, mark the third to last position of the third dimension of the target matrix as 1, and mark the corresponding position of the relationship category corresponding to the subject entity and the object entity in the third dimension as 1; If the two paired words are the tail of the subject entity and the tail of the object entity in the triple, mark the fourth to last position of the third dimension of the target matrix as 1, and mark the corresponding position of the relationship category corresponding to the subject entity and the object entity in the third dimension as 1; If the object entity in the triple is in front and the subject entity is in the back, the following marking operation is performed on the lower triangular part of the target matrix, specifically including: If the two paired words are the head of the object entity and the head of the subject entity in the triple, the last position of the third dimension of the target matrix is ​​marked as 1, and the relationship category corresponding to the object entity and the subject entity is marked as 1 in the corresponding position of the third dimension; If the two paired words are the head of the object entity and the tail of the subject entity in the triple, mark the second to last position of the third dimension of the target matrix as 1, and mark the corresponding position of the relationship category corresponding to the object entity and the subject entity as 1 in the third dimension; If the two paired words are the tail of the object entity and the head of the subject entity in the triple, mark the third to last position of the third dimension of the target matrix as 1, and mark the corresponding position of the relationship category corresponding to the object entity and the subject entity in the third dimension as 1; If the two paired words are the tail of the object entity and the tail of the subject entity in the triple, mark the fourth to last position of the third dimension of the target matrix as 1, and mark the corresponding position of the relationship category corresponding to the object entity and the subject entity in the third dimension as 1; The indexed sentences and the labeled three-dimensional target matrix are input into the Bert model for fine-tuning training to obtain a hidden layer vector containing potential relationships; the hidden layer vector is linearly transformed to obtain a matrix with the same shape as the three-dimensional target matrix; the final Bert model is obtained after training is completed and verified; the final Bert model is used to realize dual affine overlapping relationship extraction.

2. The method for extracting double affine overlapping relations based on potential relation prediction according to claim 1, characterized in that: The formal method for preprocessing sentences for overlapping relation extraction into indexes is: A tokenizer is used to convert sentences used for overlapping relation extraction into index form.

3. The method for extracting double affine overlapping relations based on potential relation prediction according to claim 1, characterized in that: The three-dimensional target matrix is ​​[maximum sentence length*maximum sentence length*(number of relationship categories+4)].

4. The method for extracting double affine overlapping relations based on potential relation prediction according to claim 1, characterized in that: The process of inputting the indexed sentence and the labeled three-dimensional target matrix into the Bert model for fine-tuning training to obtain the hidden layer vector containing the potential relationship is: Obtain the hidden vector of the last layer of Bert, and use the hidden vector of the last layer to predict the potential relationship of the sentence; After obtaining the potential relationship, an embedding coding operation is performed on each obtained relationship, and then the potential relationship vectors are added, and then the added potential relationship vectors are added to the hidden vector corresponding to each word in the last layer of Bert during the training process to obtain a hidden layer vector containing the potential relationship.

5. The method for extracting double affine overlapping relations based on potential relation prediction according to claim 4 is characterized in that: The linear transformation of the hidden layer vector to obtain a matrix having the same shape as the three-dimensional target matrix includes: First, two linear transformations are used to obtain the vector representation of each word in the sentence as the subject entity and the vector representation of each word as the object entity. Inputting the vector representation of the subject entity and the vector representation of the object entity into the dual affine model respectively to obtain a dual affine model matrix; the dual affine model matrix is ​​expressed as [maximum sentence length*maximum sentence length*hidden layer size]; The bi-affine model matrix undergoes another linear transformation and is activated using an activation function to obtain a matrix having the same shape as the three-dimensional target matrix; the matrix having the same shape as the three-dimensional target matrix is ​​expressed as [maximum sentence length*maximum sentence length*(number of relationship categories+4)].

6. The method for extracting double affine overlapping relations based on potential relation prediction according to claim 5, characterized in that: The process of using the final Bert model to extract double affine overlapping relations is as follows: Input the test data into the final Bert model to obtain the final three-dimensional matrix; The final three-dimensional matrix is ​​expressed as [maximum sentence length*maximum sentence length*(number of relationship categories+4)]; Decoding the upper triangular part of the final three-dimensional matrix, the subject entity can be determined by using the dependency relationship of subject entity head-object entity head and subject entity tail-object entity tail and subject entity head-object entity tail and subject entity tail-object entity tail; the object entity can be determined by using subject entity head-object entity head and subject entity head-object entity tail and subject entity tail-object entity head and subject entity tail-object entity tail; Then, the relationship category corresponding to the subject and the object can be determined through the relationship category index on the dependency relationship; Get a positive <subject, relation, object> triple; Similarly, by decoding the lower triangular part of the matrix representation, we can obtain the reverse <object, relationship, entity> triplet.

7. A dual affine overlapping relationship extraction system based on potential relationship prediction, used to execute the dual affine overlapping relationship extraction method based on potential relationship prediction according to any one of claims 1 to 6, characterized in that: Includes preprocessing module, labeling module and training extraction module; The preprocessing module is used to preprocess the sentences used for overlapping relationship extraction into an index form; convert the words in each sentence into the length after the index, and generate a three-dimensional target matrix with an initial value of zero; The marking module is used to mark the three-dimensional target matrix by performing a handshake pairing on each word in the sentence with itself, each preceding word, and each following word; The training extraction module is used to input the indexed sentence and the labeled three-dimensional target matrix into the Bert model for fine-tuning training to obtain a hidden layer vector containing a potential relationship; linearly transform the hidden layer vector to obtain a matrix with the same shape as the three-dimensional target matrix; After training is completed and verified, the final Bert model is obtained; the final Bert model is used to implement dual affine overlapping relationship extraction.

Images