Phrase-enhanced open-domain relation joint extraction method and system

Abstract

The application provides an open domain relation joint extraction method and system based on phrase enhancement. A BERT pre-training language model is used to encode characters in a sentence and extract a sentence feature vector representation. A first entity phrase labeling model is used to decode the sentence feature vector representation and extract all first entity phrases that can constitute a relation triple. A relation phrase vocabulary enhancement dictionary is constructed, and phrase information contained in an external relation phrase vocabulary table is fused into the sentence feature vector representation. A relation phrase and tail entity sequence labeling model is used to extract all relation phrases and tail entity phrases corresponding to the first entity phrase, and a candidate open domain relation triple set for the first entity is constructed. According to the confidence of the open domain relation triple, open domain relation triples with a confidence higher than a set threshold are selected from the candidate open domain relation triple set as the open domain relation triple of the first entity. The application can better integrate information between relation phrases and entity pair phrases.

Description

Technical Field

[0001] This invention relates to the field of natural language processing, specifically to a method and system for joint extraction of open-domain relations based on phrase enhancement. Background Technology

[0002] Open-domain relation extraction is a hot research area in information extraction. Unlike limited-domain relation extraction, open-domain relation extraction does not require pre-defining relation categories; instead, it directly extracts relation triples from unstructured text, which helps in discovering new relationships between entities. Open-domain relation extraction has broad application prospects in knowledge graph construction, question answering systems, and information retrieval.

[0003] With the rapid development of deep learning technology and the emergence of various open-source open-domain information extraction systems, open-domain relation extraction methods based on deep learning models have gradually become the mainstream trend. Traditional open-domain relation extraction methods based on deep learning usually adopt sequence labeling to divide the extraction process of open-domain relation triples in a sentence into two sub-tasks: first, identify relation phrases in a given sentence, and then extract the corresponding entity pair phrases based on the identified relation phrases. However, traditional methods have three main problems: (1) they ignore the inherent connection and dependency between the two sub-tasks of relation phrase identification and entity pair identification, resulting in low relation extraction accuracy; (2) the errors generated by the two sub-tasks may accumulate, and errors in relation phrase identification will affect the effect of entity pair identification; (3) character-level relation phrase identification methods have poor ability to identify relation phrase boundaries, resulting in redundant or missing information in the identified relation phrases, making it difficult to express the actual relation meaning.

[0004] To address the aforementioned issues, this invention innovatively proposes a phrase-enhanced open-domain relation joint extraction method. This method jointly models and learns two tasks: relation phrase identification and entity pair identification, achieving end-to-end extraction of open-domain relation triples. This approach better integrates information between relation phrases and entity pair phrases. Furthermore, this method introduces an external relation phrase dictionary to enhance the boundary learning ability of open-domain relation phrases, thereby improving the semantic conciseness and completeness of the relation phrases. Summary of the Invention

[0005] The purpose of this invention is to propose a joint open-domain relation extraction method and system based on phrase enhancement, so as to solve the problems of low extraction accuracy, redundant or missing relation phrase information, and inaccurate relation semantic expression that are common in existing open-domain relation extraction methods.

[0006] The technical solution to achieve the objective of this invention is: a method for joint extraction of open-domain relations based on phrase enhancement, comprising the following steps:

[0007] Step 1. Feature Extraction: Encode the characters in the sentence using the BERT pre-trained language model and extract the sentence feature vector representation;

[0008] Step 2. First Entity Phrase Extraction: Based on the first entity phrase annotation model, decode the sentence feature vector representation and extract all first entity phrases that may form relation triples;

[0009] Step 3. Relational phrase vocabulary construction: Construct an enhanced relational phrase vocabulary lexicon excluding the first entity phrase, and integrate the phrase information contained in the external relational phrase vocabulary into the sentence feature vector representation;

[0010] Step 4. Extraction of relation phrases and tail entity phrases: Based on the relation phrase and tail entity sequence labeling model, extract all relation phrases and tail entity phrases corresponding to the first entity phrase, and construct a set of candidate open domain relation triples for the first entity.

[0011] Step 5. Filtering open domain relation triples: Based on the confidence level of open domain relation triples, select open domain relation triples with a confidence level higher than a set threshold from the candidate open domain relation triple set as the open domain relation triples of the first entity.

[0012] Further, in step 1, the characters in the sentence are encoded using a BERT pre-trained language model to extract the sentence feature vector representation. The specific method is as follows:

[0013]

[0014] in, This represents the extracted sentence feature vector. This represents the one-hot vector matrix of words in the input sentence. Representative word embedding matrix, represents the position embedding matrix, where p represents the position index in the input sentence;

[0015]

[0016] in, This represents the hidden state vector, i.e., the input sentence at the [number]th [position]. The context representation of a layer, This represents the number of Transformer blocks.

[0017] Further, in step 2, based on the first entity phrase annotation model, the sentence feature vector representation is decoded, and all possible first entity phrases that could form relation triples are extracted. The specific method is as follows:

[0018] The formulas for calculating the start and end positions of the first entity phrase are as follows:

[0019]

[0020] in, , These represent the probabilities that the i-th word in the input sentence sequence is the start and end position of the first entity phrase, respectively. In the model sequence annotation, if the probability value is greater than the threshold, the position is set to 1; otherwise, it is set to 0. This represents the encoded sequence of the i-th word. , The weights representing the start and end positions of the first entity phrase. , Indicates the deviation between the beginning and end positions of the first entity phrase. This represents the sigmoid activation function.

[0021] Furthermore, in step 3, an enhanced relational phrase vocabulary lexicon is constructed, excluding the initial entity phrase, and the phrase information contained in the external relational phrase vocabulary is integrated into the sentence feature vector representation. The specific method is as follows:

[0022] Step 3.1, Constructing a relational phrase lexicon: Collect Wikipedia corpus data, process the data using dependency parsing and part-of-speech analysis, extract relational phrases from the Wikipedia data, and construct a relational phrase lexicon.

[0023] Step 3.2, Matching Word Classification: By matching each character in the input sentence except for the first entity phrase using the constructed relational phrase dictionary, the matched structural words are classified into four categories: "BMES". For an input sentence sequence with the first entity phrase removed... The four categories of “BMES” are defined as follows:

[0024]

[0025] in, The dictionary represents the constructed relational phrase vocabulary enhancement dictionary; B, M, E, and S respectively represent the... The word at the beginning position, with For words in the middle position, with For the word at the end, a single The word w represents all possible matching words from the B, M, E, and S word sets;

[0026] Step 3.3, Word Set Compression: After obtaining the four word sets "BMES", the content of each word set is compressed into a vector of fixed dimensions to obtain the relational phrase vocabulary information. The compression formula is:

[0027]

[0028] in, , Represents a set of words. Representative terms are embedded in the lookup table. Representative words Frequency of occurrence in the Wikipedia dataset;

[0029] Step 3.4: Embed the relational SMS vocabulary information into the sentence feature vector, using the following formula:

[0030]

[0031] in, Represents the weighting function. This indicates the concatenation of the four word sets "BMES". This represents the sentence feature vector that embeds relational SMS vocabulary information.

[0032] Further, in step 4, based on the relation phrase and tail entity sequence labeling model, all relation phrases and tail entity phrases corresponding to the first entity phrase are extracted, and a set of candidate open-domain relation triples for the first entity is constructed. The specific method is as follows:

[0033] The formula for extracting the possible tail entity phrase positions corresponding to the first entity is:

[0034]

[0035] in, , and represent the probabilities that the i-th word in the input sentence sequence is the beginning and end position of the tail entity phrase, respectively. Represents the k-th candidate first entity. This represents the encoded sequence of the i-th word. , The weights representing the start and end positions of the tail entity phrase. , Indicates the deviation between the beginning and end positions of the final entity phrase. This represents the sigmoid activation function;

[0036] The formula for extracting the possible relational phrase positions corresponding to the first entity is:

[0037]

[0038] in , and represent the probabilities that the i-th word in the input sentence sequence is the beginning and end position of a relational phrase, respectively. Represents the k-th candidate first entity. This represents the m-th tail entity that may correspond to the candidate first entity. This represents the encoded sequence of the i-th word. , The weights representing the start and end positions of the relational phrase. , Indicates the deviation between the beginning and end positions of the relative phrase. This represents the sigmoid activation function.

[0039] Furthermore, in step 5, for cases where the first entity has multiple relationships, multiple relationship triples are retained based on the filtering results.

[0040] A phrase-enhanced open-domain relation joint extraction system is provided, which realizes phrase-enhanced open-domain relation joint extraction based on the aforementioned phrase-enhanced open-domain relation joint extraction method.

[0041] A computer device includes a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it performs phrase-enhanced open-domain relation joint extraction based on the aforementioned phrase-enhanced open-domain relation joint extraction method.

[0042] A computer-readable storage medium having a computer program stored thereon, wherein when the computer program is executed by a processor, it performs phrase-enhanced open-domain relation joint extraction based on the aforementioned phrase-enhanced open-domain relation joint extraction method.

[0043] Compared with existing technologies, the significant advantages of this invention are: 1) It utilizes the inherent connections and dependencies between the two sub-tasks of relation phrase recognition and entity pair recognition, and jointly models and learns these two tasks to achieve end-to-end open-domain relation triple extraction, which can better integrate the information between relation phrases and entity pair phrases. 2) It introduces an external relation phrase dictionary to enhance the boundary learning ability of open-domain relation phrases and improve the semantic conciseness and completeness of relation phrases. Attached Figure Description

[0044] Figure 1 This is a framework diagram of an open-domain relation joint extraction method based on phrase enhancement;

[0045] Figure 2 This is a flowchart of an open-domain relation joint extraction method based on phrase enhancement. Detailed Implementation

[0046] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0047] Figure 1 This is a framework diagram of a phrase-enhanced open-domain relation joint extraction method according to the present invention. The phrase-enhanced open-domain relation joint extraction method includes:

[0048] Step 1. Feature Extraction. The original sentence is encoded using a BERT pre-trained model to extract sentence feature vectors.

[0049]

[0050] in, This represents the one-hot vector matrix of words in the input sentence. Representative word embedding matrix, represents the position embedding matrix, and p represents the position index in the input sentence.

[0051]

[0052] in, This represents the hidden state vector, i.e., the input sentence at the [number]th [position]. The context representation of a layer, Represents the Transformer block The quantity of (.).

[0053] Step 2. Extract candidate first entity phrases from the sentence. By decoding the sentence feature vector generated by the BERT encoder, extract all possible first entity phrases from the input sentence. The formulas for calculating the start and end positions of the first entity phrases are:

[0054]

[0055] in, , These represent the probabilities of the i-th word in the input sentence sequence being the start position and structural position of the first entity phrase, respectively. In the model sequence annotation, if the probability value is greater than the threshold, the position is set to 1; otherwise, it is set to 0. This represents the encoded sequence of the i-th word. Indicates weight, Indicates deviation, This represents the sigmoid activation function.

[0056] Step 3. Relational Phrase Vocabulary Construction. To address issues such as redundant relational phrases and erroneous relational information in open-domain relation extraction results, an enhanced relational phrase vocabulary is proposed to refine the extracted relational expressions, making the extracted relational triples more concise and clear. The construction of the relational phrase vocabulary mainly includes the following steps:

[0057] (1) Constructing a dictionary of relational phrases: Collect Wikipedia corpus data, process the data using methods such as dependency parsing and part-of-speech analysis, and extract relational phrases from the Wikipedia data to construct a dictionary of relational phrases.

[0058] (2) Matching word classification: By constructing a relational phrase dictionary set, each character in the input sentence except for the first entity phrase is matched, and the matched structural words are divided into four categories: "BMES". For an input sentence sequence with the first entity phrase removed, The four categories of “BMES” are defined as follows:

[0059]

[0060] in, The dictionary set to be constructed is represented by B, M, E, and S respectively. The word at the beginning position, with For words in the middle position, with For the word at the end, a single The word w represents all possible matching words from the B, M, E, and S word sets.

[0061] (3) Word Set Compression: After obtaining the four word sets “BMES”, the content of each word set is compressed into a vector of fixed dimension. The compression formula is:

[0062]

[0063] in, , Represents a set of words. Representative terms are embedded in the lookup table. Representative words Frequency of occurrence in the Wikipedia dataset.

[0064] (4) Embedding relational SMS vocabulary information into character features: The formula for embedding relational SMS vocabulary information into the character features of a sentence is as follows:

[0065]

[0066] in, Represents the weighting function. This indicates the concatenation of the four word sets "BMES". This represents the sentence feature vector that embeds relational SMS vocabulary information.

[0067] Step 4. Extract the relational phrases and tail entity phrases corresponding to the first entity phrase. The formula for extracting the possible tail entity phrase positions corresponding to the first entity is:

[0068]

[0069] in, , and represent the probabilities that the i-th word in the input sentence sequence is the start and end position of the tail entity phrase, respectively. Represents the k-th candidate first entity. This represents the encoded sequence of the i-th word. Indicates weight, Indicates deviation, This represents the sigmoid activation function.

[0070] The formula for extracting the possible relational phrase positions corresponding to the first entity is:

[0071]

[0072] in , and represent the probabilities that the i-th word in the input sentence sequence is the start and end position of a relational phrase, respectively. Represents the k-th candidate first entity. This represents the m-th tail entity that may correspond to the candidate first entity. This represents the encoded sequence of the i-th word. Indicates weight, Indicates deviation, This represents the sigmoid activation function.

[0073] Step 5. In the candidate triplet set extracted from the first entity, filter the relation triplets based on the confidence level. Retain the triplets with a confidence level greater than the threshold of 0.5 as the open domain relation triplets corresponding to the first entity.

[0074] This invention also proposes a phrase-enhanced open-domain relation joint extraction system, which realizes phrase-enhanced open-domain relation joint extraction based on the aforementioned phrase-enhanced open-domain relation joint extraction method.

[0075] A computer device includes a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it performs phrase-enhanced open-domain relation joint extraction based on the aforementioned phrase-enhanced open-domain relation joint extraction method.

[0076] A computer-readable storage medium having a computer program stored thereon, wherein when the computer program is executed by a processor, it performs phrase-enhanced open-domain relation joint extraction based on the aforementioned phrase-enhanced open-domain relation joint extraction method.

[0077] The technical solution of the present invention will be illustrated below with examples.

[0078] Given the input sentence s = {Xiaoming, the class monitor of a university in a certain place, led a group to visit a museum}

[0079] Step 1. Data encoding and extraction of input sentence feature information. The feature vector extracted from the sentence using the BERT pre-trained model is [0.087197 -0.083435 0.057956 0.143120 -0.000068 0.123272 0.022439 -0.323317 -0.231756 -0.009262 -0.211264 -0.021698 0.246367 0.179090 0.054117 0.077638 -0.054555 -0.050630 0.072361 0.103788];

[0080] Step 2. Extract candidate first entity phrases from the sentence. By decoding the sentence feature vector generated by the BERT encoder, all possible first entity phrases extracted according to formulas (3) and (4) are {a certain place, Xiaoming}.

[0081] Step 3. Constructing a vocabulary of relational phrases. First, the dictionary set constructed from relational phrases includes {a certain place, university class monitor, class monitor, visit, lead a team, lead a team to visit...}. This dictionary set is then compressed and vectorized into [-0.13128 -0.452 0.043399 -0.99798 -0.21053 -0.95868 -0.24609 0.48413 0.18178 0.475 -0.22305 0.30064 0.43496 -0.3605 0.20245 -0.52594 -0.34708 0.0075873 -1.0497 0.18673 0.57369 0.43814 0.098659 0.3877 -0.2258 0.41911]. [0.043602 -0.7352 -0.53583 0.19276 -0.21961], embedding the dictionary into the sentence feature vector yields the sentence vector for enhanced relational phrases as [0.13357 0.41839 1.3138 0.35678 -0.32172 -1.2257 -0.26635 0.36716 -0.27586 -0.53246 0.16786 -0.11253 -0.99959 -0.60706 -0.89271 0.65156 -0.88784 0.049233 0.67111 -0.27553 -2.4005 -0.36989 0.29136 1.3498]. 1.7353 0.27 0.021299 0.14422 0.023784 0.33643 -0.35476 1.0921 1.4845.

[0082] Step 4. Extract the relation phrases and tail entity phrases corresponding to the first entity phrase to obtain the candidate relation triple set corresponding to the first entity phrase. For example, the candidate relation triple set obtained for the first entity 'Xiaoming' is {(Xiaoming, visit, museum), (Xiaoming, lead, visit), (Xiaoming, lead, museum)}.

[0083] Step 5. Filter the candidate relation triples extracted from the first entity based on their confidence scores. For example, in the candidate relation triples extracted from the first entity 'Xiaoming', the confidence scores of (Xiaoming, visit, museum), (Xiaoming, visit, museum), and (Xiaoming, lead, museum) are 0.81, 0, 47, and 0.39 respectively. Therefore, the relation triple extracted for this first entity is (Xiaoming, visit, museum).

[0084] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0085] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of this application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these modifications and improvements all fall within the protection scope of this application. Therefore, the protection scope of this application should be determined by the appended claims.

Claims

1. A method for joint extraction of open-domain relations based on phrase enhancement, characterized in that, Includes the following steps: Step 1. Feature Extraction: Encode the characters in the sentence using the BERT pre-trained language model and extract the sentence feature vector representation; Step 2. First Entity Phrase Extraction: Based on the first entity phrase annotation model, decode the sentence feature vector representation and extract all first entity phrases that may form relation triples; Step 3. Construction of the relational phrase vocabulary enhancement dictionary: Construct a relational phrase vocabulary enhancement dictionary excluding the first entity phrase, and integrate the phrase information contained in the relational phrase vocabulary enhancement dictionary into the sentence feature vector representation; Step 4. Extraction of relation phrases and tail entity phrases: Based on the relation phrase and tail entity sequence labeling model, extract all relation phrases and tail entity phrases corresponding to the first entity phrase, and construct a set of candidate open domain relation triples for the first entity. Step 5. Filtering open domain relation triples: Based on the confidence level of open domain relation triples, select open domain relation triples with a confidence level higher than a set threshold from the candidate open domain relation triple set as the open domain relation triples of the first entity; in: Step 3, the specific method is as follows: Step 3.1, Constructing a relational phrase lexicon: Collect Wikipedia corpus data, process the data using dependency parsing and part-of-speech analysis, extract relational phrases from the Wikipedia data, and construct a relational phrase lexicon. Step 3.2, matching word classification: By constructing a relational phrase vocabulary enhancement dictionary, each character in the input sentence except for the first entity phrase is matched, and the matched structural words are classified into four categories: BMES. Step 3.3, word set compression: After obtaining the four word sets of BMES, the contents of each word set are compressed into a vector of fixed dimensions to obtain the relational phrase vocabulary augmentation dictionary information; Step 3.4: Embed the vocabulary enhancement dictionary information of relational phrases into the sentence feature vector; Step 4, the specific method is as follows: The formula for extracting the possible tail entity phrase positions corresponding to the first entity is: ； in, , and represent the probabilities that the i-th word in the input sentence sequence is the beginning and end position of the tail entity phrase, respectively. Represents the k-th candidate first entity. This represents the encoded sequence of the i-th word. , The weights representing the start and end positions of the tail entity phrase. , Indicates the deviation between the beginning and end positions of the final entity phrase. This represents the sigmoid activation function; The formula for extracting the possible relational phrase positions corresponding to the first entity is: ； in , and represent the probabilities that the i-th word in the input sentence sequence is the beginning and end position of a relational phrase, respectively. Represents the k-th candidate first entity. This represents the m-th tail entity that may correspond to the candidate first entity. This represents the encoded sequence of the i-th word. , The weights representing the start and end positions of the relational phrase. , Indicates the deviation between the beginning and end positions of the relative phrase. This represents the sigmoid activation function.

2. The open-domain relation joint extraction method based on phrase enhancement according to claim 1, characterized in that, Step 1: Encode the characters in the sentence using the BERT pre-trained language model and extract the sentence feature vector representation. The specific method is as follows: （1）； in, This represents the extracted sentence feature vector, where T represents the one-hot vector matrix of words in the input sentence. Representative word embedding matrix, represents the position embedding matrix, where p represents the position index in the input sentence; （2）； in, The hidden state vector is represented by the input sentence at the [number]th [position]. The context representation of a layer, This represents the number of Transformer blocks.

3. The open-domain relation joint extraction method based on phrase enhancement according to claim 1, characterized in that, Step 2: Based on the first entity phrase annotation model, decode the sentence feature vector representation and extract all possible first entity phrases that could form relation triples. The specific method is as follows: The formulas for calculating the start and end positions of the first entity phrase are as follows: ； in, , These represent the probabilities that the i-th word in the input sentence sequence is the start and end position of the first entity phrase, respectively. In the model sequence annotation, if the probability value is greater than the threshold, the position is set to 1; otherwise, it is set to 0. This represents the encoded sequence of the i-th word. , The weights representing the start and end positions of the first entity phrase. , Indicates the deviation between the beginning and end positions of the first entity phrase. This represents the sigmoid activation function.

4. The open-domain relation joint extraction method based on phrase enhancement according to claim 1, characterized in that, Step 5: For cases where the first entity has multiple relationships, retain multiple relationship triples based on the filtering results.

5. A joint open-domain relation extraction system based on phrase enhancement, characterized in that, Based on the phrase-enhanced open-domain relation joint extraction method according to any one of claims 1-4, the phrase-enhanced open-domain relation joint extraction is realized.

6. A computer device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, it performs phrase-enhanced open-domain relation joint extraction based on the phrase-enhanced open-domain relation joint extraction method according to any one of claims 1-4.

7. A computer-readable storage medium having a computer program stored thereon, wherein when the computer program is executed by a processor, it performs phrase-enhanced open-domain relation joint extraction based on the phrase-enhanced open-domain relation joint extraction method according to any one of claims 1-4.

Images